Multi-plane network switch

Abstract

The present disclosure generally relates to a multi-plane network switch. A network switch includes a plurality of data crossbars and a plurality of header planes for controlling the data crossbars. In an exemplary embodiment, an apparatus includes: a plurality of header planes, each of the header planes including an input queue, an output queue, a request crossbar connected to the input queue and the output queue, and an authorization crossbar connected to the input queue and the output queue; a receiver; and a header plane selector. The header plane selector is configured to: receive a transmission request for a packet from a receiver; selecting a header plane among the header planes based on a header of the packet; and queuing the transmission request at an input queue of the header plane.

Description

Background Art

[0001] Network switching is a fundamental concept in computer networking that involves forwarding data packets between devices within a network. Network switches analyze the destination of incoming packets and use that information to make forwarding decisions, thereby performing data transfer within the network. Network switches reduce network congestion and increase overall efficiency compared to older hub-based networks. Bandwidth and speed are important factors in network switching performance, as consistently high bandwidth results in faster communication between devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0002] When with Figure 1 Aspects of the present disclosure are best understood from the following detailed description when read together.

[0003] Figure 1 is a schematic diagram of a network system according to some embodiments.

[0004] Figure 2 is a block diagram of a network switch according to some embodiments.

[0005] Figure 3 is a block diagram of a network switch according to some embodiments.

[0006] Figure 4 is a schematic diagram of a packet forwarding method according to some implementations.

[0007] Figure 5 is a schematic diagram of a plane selection method according to some embodiments.

[0008] Figure 6 is a block diagram of a network switch according to some embodiments.

[0009] Corresponding numerals and symbols in the different figures generally refer to corresponding parts unless otherwise indicated.The figures are drawn to clearly illustrate the relevant aspects of the disclosure and are not necessarily drawn to scale. DETAILED DESCRIPTION

[0010] The following disclosure provides many different examples for implementing different features. Specific examples of components and arrangements are described below to simplify the disclosure. Of course, these are merely examples and are not intended to be limiting.

[0011] Network switches are used for data packet forwarding in a local network. Packets can be large (e.g., in the case of Ethernet frames) or small (e.g., in the case of high performance computing (HPC) network frames). For example, a large packet can be larger than 256 bytes. Although network switches can generally operate at a line rate when forwarding large packets, they may have difficulty maintaining a line rate when forwarding small packets. This can be at least partially attributed to the effect of clock speed. Increasing the clock speed is a basic method to maintain a line rate when forwarding small packets, but as the demand for higher line rates increases, this may not be feasible.

[0012] The present disclosure describes a network switch including a plurality of data crossbar switches. A packet is transmitted from an input buffer of an input port to an output buffer of an output port via the data crossbar switch. Specifically, a plurality of packets are transmitted in parallel via the data crossbar switch. In practice, a plurality of packets can be transmitted from an input buffer to an output buffer per clock cycle. Therefore, compared to a network switch using a single data crossbar switch, the number of packets transmitted per clock cycle can be increased, which allows the total amount of packets transmitted to be increased without the need to increase the clock speed. Therefore, even when processing small packets, the linear rate of the network switch can be maintained without the disadvantage of increasing the clock speed.

[0013] In an exemplary embodiment, each port of a network switch includes components of multiple header planes that control the parallel transmission of packets through a data crossbar switch. Each header plane can independently transmit packets. Using multiple header planes allows for more efficient control of multiple data crossbar switches, which can be more advantageous than using a single data crossbar switch or using multiple data crossbar switches without multiple header planes. A first header plane in the header planes can control the transmission of a first packet from an input buffer to an output buffer through the data crossbar switch, and a second header plane in the header planes can control the transmission of a second packet from an input buffer to an output buffer through the data crossbar switch. The packet can be part of a packet stream, in which case a single header plane in the header planes can control the transmission of the packet from an input buffer to an output buffer through multiple data crossbar switches. When transmitting a packet, an input queue of the header plane can receive a sequence number of the packet and can transmit the sequence number to an output buffer along with the packet. The packet can be read from the output buffer and transmitted by the egress port in the order of its sequence number. In this way, the ordering of packets in the packet flow can be maintained even if congestion in the data crossbar causes packets to arrive at the output buffer out of order.

[0014] Figure 11 is a schematic diagram of a network system 100 according to some embodiments. The network system 100 may be a high-performance network as part of a high-performance computing (HPC) environment. In the network system 100, client devices 102 act as sending nodes and access servers 104. The servers 104 interact with each other and may interact with storage devices 106. The servers 104 and / or storage devices 106 act as receiving nodes and provide software / services to the client devices 102. The servers 104 and / or storage devices 106 may be part of an HPC cluster. The servers 104 and storage devices 106 are merely examples of devices that may be accessed by the client devices 102.

[0015] The client device 102 interacts with the server 104 via a network 108. The network 108 may be any wired or wireless network suitable for facilitating communication between the client device 102 and the server 104. The client device 102 and / or the server 104 include a network interface for accessing the network 108. For example, the client device 102 and / or the server 104 may include a network interface card (NIC), a wireless interface card, etc. The network 108 may include a controller, an access point, a switch, a router, etc. for forwarding traffic between the client device 102 and the server 104. The client device 102 may be any electronic device suitable for executing executable code. The server 104 may be a bare metal machine suitable for hosting cloud components (e.g., virtual machines, containers, etc.). The storage device 106 may be a network attached storage (NAS) device.

[0016] The server 104 and the storage device 106 are interconnected via one or more network switches (including the network switch 110). The server 104 and the storage device 106 communicate with each other via packets transmitted between the sending node and the receiving node. The server 104 and the storage device 106 are only examples of devices that can be interconnected via the network switch 110. Other devices can be connected to the network switch 110. The packets can be routed by the network switch 110. The network switch 110 can have a high linear rate, such as 800Gbps, 1600Gbps or higher linear rates. When the server 104 and / or the storage device 106 are part of an HPC cluster, a high linear rate can be expected. Using a wider data bus or a higher clock speed is a technique to increase the linear rate, however, when forwarding small packets, such techniques may not be suitable. As described in more detail later, the network switch 110 uses multiple header planes and multiple data crossbar switches to allow the network switch 110 to forward multiple packets per clock cycle. Advantageously, this helps the network switch 110 maintain a high linear rate even when forwarding small packets.

[0017] Figure 2is a block diagram of a network switch 200 according to some embodiments. The network switch 200 is a block diagram of a network switch 200 according to some embodiments. Figure 1 An example of a network switch 110 is depicted. The network switch 200 includes ports 202 and a switch fabric 204. The ports 202 serve as connection points for nodes (eg, servers, storage devices, etc.). The switch fabric 204 manages and forwards data packets between the ports 202.

[0018] Each port 202 includes an input port 206 and an output port 208. The input port 206 is the entry point through which packets enter the network switch 200. The output port 208 is the exit point responsible for transmitting the packet to its designated destination. When a packet arrives at the input port 206, the network switch 200 checks the destination address of the packet to determine the appropriate output port 208 for transmission. This process is called switching or forwarding, and includes performing a lookup in the forwarding table of the network switch 200 to find a forwarding path for the packet.

[0019] In a one-to-many configuration, each input port 206 receives packets from a node connected to the input port 206 and sends the packets to an output port 208. The input port 206 controls how packets are sent to the output port 208. Similarly, in a many-to-one configuration, each output port 208 receives packets from an input port 206 and sends packets to a node connected to the output port 208. The output port 208 controls how packets are received from the input port 206.

[0020] The ports 202 (including input ports 206 and output ports 208) are interconnected via a switch fabric 204, which provides the necessary path for packets to move from the input ports 206 to the output ports 208. The switch fabric 204 links the input ports 206 and the output ports 208. Depending on the architecture of the network switch 200, the switch fabric 204 can be based on a crossbar matrix, a shared bus, a shared memory, etc. In an embodiment, the switch fabric 204 includes a plurality of crossbar switches, which are used for both control and data transmission between the input ports 206 and the output ports 208.

[0021] The components of the network switch 200 may be implemented as integrated circuits, such as in one or more integrated circuit dies and / or one or more integrated circuit packages. For example, the network switch 200 may include a processor, an application specific integrated circuit, a field programmable gate array, a memory, and combinations thereof, etc. One or more modules within the network switch 200 may be partially or completely embodied as software and / or hardware for performing any of the functions described herein. For example, the buffers, queues, crossbars, transmitters, receivers, structures, etc. described herein may each be embodied as a logic block of an integrated circuit. In some embodiments, the components of the network switch 200 are implemented on multiple integrated circuit dies / packages, which may be assembled together in the same base.

[0022] As described in more detail later, the switch fabric 204 includes a plurality of data crossbar switches. Packets are transmitted in parallel from the input port 206 to the output port 208 through the data crossbar switches of the switch fabric 204. In addition, each port of the network switch 200 includes components of a plurality of header planes that control the parallel transmission of packets through the data crossbar switches. Therefore, multiple packets can be transmitted from the input port 206 to the output port 208 per clock cycle. Therefore, even when the packets are small, the linear rate of the network switch 200 can be maintained.

[0023] Figure 3 is a block diagram of a network switch 300 according to some embodiments. The network switch 300 is a block diagram of a network switch 300 according to some embodiments. Figure 2 An example of a network switch 200 is depicted. Additional components of the network switch 300 are shown, including components of a switch fabric 304, input ports 306, and output ports 308. The network switch 300 includes a plurality of header planes 310, including a first header plane 310A and a second header plane 310B, that control the parallel transmission of packets from the input ports 306 to the output ports 308. The logic flow during forwarding of packets from the input ports 306 to the output ports 308 is shown. Although packet forwarding from one input port 306 to one output port 308 will be described, it should be understood that each input port 306 can send packets to multiple output ports 308, and each output port 308 can receive packets from multiple input ports 306.

[0024] First, the components of the network switch 300 will be described. The input port 306 includes a receiver 312, an input buffer 314, a header plane selector 316, and a plurality of input queues 318 (including a first input queue 318A and a second input queue 318B). The receiver 312 receives packets from a source node connected via the input port 306. The destination of the packet is a destination node connected via the output port 308. The source node and / or the destination node may be directly connected to the network switch 300, or one or more network components (e.g., additional switches) may exist between the network switch 300 and the source node / destination node. The input buffer 314 is connected to the receiver 312. The received packets are stored in the input buffer 314.

[0025] The header plane selector 316 is also connected to the receiver 312. As used herein, the header plane selector 316 is a circuit, firmware or other component for selecting a component of the network switch 300 to control the packet to be transmitted through the data crossbar switch. The input queue 318 is connected to the header plane selector 316 and the input buffer 314 respectively. Therefore, the input queue 318 is connected to the receiver 312 via the header plane selector 316. Each input queue 318 is an input controller that controls the output buffers of the packets transmitted from the input buffer 314 to the respective output ports 308. The request for the packet to be sent to the respective output ports 308 is queued by the receiver 312 at the input queue 318. Each input queue 318 arbitrates in the request queued therein, and selects the request to be processed. As described in more detail later, the header plane selector 316 load balances the request on the entire input queue 318. The input queue 318 determines which output port 308 the packet for the selected request should be forwarded to. For example, an Ethernet lookup unit (not separately shown) may extract the appropriate headers from the packet and use them to perform address translation for the packet's destination node.Input queue 318 may receive the lookup results from the Ethernet lookup unit.

[0026] The output port 308 includes a plurality of output queues 322 (including a first output queue 322A and a second output queue 322B), an output buffer 324, and a transmitter 326. Each output queue 322 is an output controller that controls the reception of packets in the output buffer 324 from the input buffer of each input port 306. Requests to receive packets from each input port 306 are queued at the output queue 322. Each output queue 322 arbitrates among the requests in its queue and selects the request to be processed. The output buffer 324 is connected to the output queue 322. The received packets are stored in the output buffer 324. The transmitter 326 is connected to the output buffer 324. The transmitter 326 reads packets from the output buffer 324 and transmits the packets to its destination node. Therefore, by controlling the reception of the packets in the output buffer 324, the output queue 322 effectively controls the transmitter 326 to read packets from the output buffer 324.

[0027] The network switch 300 may support packet pull. Packet pull, such as Virtual Output Queuing (VOQ), is an architectural technique that allows a network switch to maintain a linear rate in the event of congestion by reducing head-of-line blocking. The input queue 318 may maintain a virtual queue for each of the output ports 308. Thus, if congestion occurs at an output port 308, the virtual queue for that output port 308 may be blocked, but virtual queues for other output ports 308 may continue to be processed. Thus, the network switch 300 may support congestion management and / or traffic shaping features.

[0028] The switch fabric 304 includes a plurality of crossbar switches separated from each other. In this example, the switch fabric 304 includes a plurality of request crossbar switches 332 (including a first request crossbar switch 332A and a second request crossbar switch 332B), a plurality of grant crossbar switches 334 (including a first grant crossbar switch 334A and a second grant crossbar switch 334B), and a plurality of data crossbar switches 336 (including a first data crossbar switch 336X and a second data crossbar switch 336Y). The request crossbar switch 332 and the grant crossbar switch 334 are connected to the input queue 318 of each input port 306, and to the corresponding output queue 322 of each output port 308. As described in more detail later, the transmission request will be sent through the request crossbar switch 332, and the transmission grant will be sent through the grant crossbar switch 334. The data crossbar switch 336 is connected to the input buffer 314 of each input port 306, and to the output buffer 324 of each output port 308. As described in more detail later, the packet will be transmitted through the data crossbar switch 336.

[0029] The data crossbars 336 can each have the same width (thus providing the same bandwidth), or can have different widths. In some embodiments, the data crossbars 336 include a first data crossbar of a first (e.g., small) width and a second data crossbar of a second (e.g., large) width. The first data crossbar 336 can be used to forward small packets (requiring less bandwidth), while the second data crossbar 336 can be used to forward large packets (requiring higher bandwidth).

[0030] The network switch 300 includes a plurality of header planes 310 (including a first header plane 310A and a second header plane 310B) for controlling the transmission of packets from the input port 306 to the output port 308. The first header plane 310A controls the transmission of a first packet from the input buffer 314 to the output buffer 324 through one of the data crossbar switches 336, while the second header plane 310B controls the transmission of a second packet from the input buffer 314 to the output buffer 324 through another of the data crossbar switches 336. The first header plane 310A can transmit packets independently of the second header plane 310B. The output queues 322 of the header planes 310 can coordinate traffic shaping via crediting.

[0031] The first header plane 310A includes a first input queue 318A for each input port 306, a first output queue 322A for each output port 308, a first request crossbar 332A, and a first grant crossbar 334A. The first request crossbar 332A and the first grant crossbar 334A are connected to the first input queue 318A for each input port 306 and the first output queue 322A for each output port 308. The first header plane 310A processes a first request to transfer a first packet from the input buffer 314 to the output buffer 324. Either the first data crossbar 336X or the second data crossbar 336Y can be used by the first header plane 310A to transfer the packet. As described in more detail later, the selection of the data crossbar 336 for packet transfer by the first header plane 310A can be based on the load of the data crossbar 336.

[0032] The second header plane 310B includes a second input queue 318B for each input port 306, a second output queue 322B for each output port 308, a second request crossbar 332B, and a second grant crossbar 334B. The second request crossbar 332B and the second grant crossbar 334B are connected to the second input queue 318B for each input port 306 and the second output queue 322B for each output port 308. The second header plane 310B processes a second request to transfer a second packet from the input buffer 314 to the output buffer 324. Either the first data crossbar 336X or the second data crossbar 336Y can be used by the second header plane 310B to transfer the packet. As described in more detail later, the selection of the data crossbar 336 for packet transfer by the second header plane 310B can be based on the load of the data crossbar 336.

[0033] The header plane selector 316 load balances the requests from the receiver 312 across the header plane 310. When the receiver 312 stores the packet in the input buffer 314, it also sends a request to the header plane selector 316. The header plane selector 316 selects one of the header planes 310 to process the request. The request is then forwarded to the input queue 318 of the selected header plane 310. The header plane selector 316 may include a first-in-first-out (FIFO) buffer that is used to queue the request before the request is distributed to the header plane 310. Because multiple packets are forwarded per clock cycle, the FIFO buffer of the header plane selector 316 is large, for example, deep enough to save requests for packet bursts. In some embodiments, the FIFO buffer of the header plane selector 316 is deep enough to save two requests per header plane 310.

[0034] The logic flow for forwarding a packet from an input port 306 to an output port 308 will now be described. A receiver 312 receives a packet and stores the packet in an input buffer 314. A request to transmit a packet is sent to a header plane selector 316. The header plane selector 316 selects a header plane 310. The criteria for header plane selection will be described later. The header plane selector 316 then queues the request at an input queue 318 of the selected header plane 310, which then selects the request for processing. The input queue 318 sends a transmit request for the packet to an output queue 322 of the selected header plane 310 via a request crossbar 332 of the selected header plane 310. The transmit request includes a description of the packet; for example, the request may include information from a header of the packet.

[0035] The transmit request is queued at the output queue 322 of the selected header plane 310, which then selects the transmit request for processing. The output queue 322 decides whether and when to grant the transmit request. For example, the output queue 322 can decide which transmit request to grant next based on the packet description of the transmit request and based on the current state of the output buffer 324. In response to the transmit request being granted, the output queue 322 sends the transmit grant to the input queue 318 through the grant crossbar 334 of the selected header plane 310.

[0036] The transmit grant is a notification instructing the input queue 318 to move a packet from the input buffer 314 to the output buffer 324. In response to receiving the transmit grant, the input queue 318 selects a data crossbar 336 and then transmits the packet from the input buffer 314 to the output buffer 324 via the selected data crossbar 336. The criteria for data crossbar selection will be described later. The transmitter 326 then reads the packet from the output buffer 324. Thus, the output queue 322 controls the reading of the packet by the transmitter 326 (and therefore determines which packets are sent via the transmitter 326) by the grant of the transmit request.

[0037] The packets are transmitted in parallel through the data crossbar switch 336. For example, the first packet and the second packet can both be transmitted from the input buffer 314 to the output buffer 324 through the data crossbar switch 336. The transmission of the first packet through the first data crossbar switch 336X and the transmission of the second packet through the second data crossbar switch 336Y can be performed in parallel. Because the data crossbar switch 336 is connected to each input buffer 314 and each output buffer 324, the packets sent in parallel through the data crossbar switch 336 can be sent to the same output (as in this example) or different outputs. The data crossbar switch 336 can be selected by the input queue 318 or other modules based on the load of the data crossbar switch 336. For example, the input queue 318 or other modules can select any data crossbar switch 336 with the least pending transmission request. In addition, the input queue 318 or other modules can select different data crossbar switches 336 for different packet transmissions. For example, the input queue 318 or other modules can select the first data crossbar switch 336X for the first packet transmission, and then select the second data crossbar switch 336Y for the second packet transmission.

[0038] When a header plane 310 is selected (by the header plane selector 316) to process a request to transmit a packet, the request may traverse the selected header plane 310 without switching to another header plane 310 midway through processing. For example, if the header plane selector 316 selects the first header plane 310A, the first input queue 318A sends a transmit request to the first output queue 322A via the first request crossbar 332A, and the first output queue 322A sends a transmit grant to the first input queue 318A via the first grant crossbar 334A. Similarly, if the header plane selector 316 selects the second header plane 310B, the second input queue 318B sends a transmit request to the second output queue 322B via the second request crossbar 332B, and the second output queue 322B sends a transmit grant to the second input queue 318B via the second grant crossbar 334B.

[0039] The header plane selector 316 can select the header plane 310 based on one or more criteria. In some embodiments, requests are distributed to the header plane 310 to achieve load balancing. For example, requests can be distributed in a round-robin manner. In some embodiments, requests for all packets of a specific frame type can be distributed to a specific header plane 310. For example, one header plane 310 can be used for all Ethernet frame requests, while another header plane 310 can be used for all HPC network frame requests. In some embodiments, the header plane 310 is selected based on one or more headers of the packet. For example, the same header plane 310 can be used for all packets of the same packet flow. One or more fields in the header can be hashed to calculate a flow hash, which is an integer value. Examples of packet header fields that can be hashed include source MAC / IP address, destination MAC / IP address, flow label, etc. The header plane selector 316 can then select the header plane 310 based on the flow hash. Each packet of a packet flow can have the same flow hash, and in this way, the same header plane 310 will be selected for each packet of the packet flow. As described later, using the same header plane 310 for each packet of the packet flow helps to ensure that the packets maintain proper ordering. The criteria for selecting a header plane 310 for a packet request can depend on whether the packet is an ordered packet or an unordered packet. For example, unordered packet requests can be distributed to the header plane 310 in a polling manner, while ordered packet requests can be distributed to the header plane 310 based on information contained in the header of the packet. Other variations are contemplated. For example, the header plane 310 can be selected based on the content of the packet itself (which can be evaluated via deep packet inspection), alone or in addition to any of the above criteria.

[0040] A packet stream may be forwarded from an input port 306 to an output port 308. A packet stream is an ordered sequence of packets. When a packet stream is forwarded, a single header plane 310 is used to control the forwarding of each packet of the packet stream. The single header plane 310 uses multiple data crossbar switches 336 to forward packets of the packet stream. When forwarding a packet stream, the ordering of the packets in the packet stream should be maintained so that the transmitter 326 reads the packets of the packet stream from the output buffer 324 in the desired order. However, when multiple data crossbar switches 336 are used to forward packets of a packet stream, the packets of the packet stream may arrive at the output port 308 out of order due to congestion in the switching structure 304.

[0041] The same header plane 310 can be used to process each request to transmit a packet in a packet stream. Therefore, all transmission requests of a packet stream can traverse the selected header plane 310 without switching to another header plane 310 in the middle of processing. When processing an ordered packet request, a header plane 310 is selected based on information contained in the header of the packet. Specifically, a flow hash is calculated for the packets of the packet stream (as described above). The packets of the packet stream can each have the same flow hash. The header plane selector 316 distributes requests with the same flow hash to the same header plane 310. Therefore, the transmission of all packets of the packet stream is controlled by the same header plane 310.

[0042] In some embodiments, the output queue 322 of the selected header plane 310 assigns a sequence number to each packet in the packet stream based on the order of the packets in the packet stream. Specifically, each output queue 322 assigns a sequence number to the transmission request of the packets in the packet stream in the order in which the requests are received. Each output queue 322 may have a sequencer (or counter) for each input port 306, and whenever the output queue 322 grants a transmission request, it uses the value of the sequencer as the sequence number and increments the sequencer. When the output queue 322 grants a transmission request, the transmission grant sent to the input queue 318 of the selected header plane 310 includes the sequence number of the packet for the request. When the packet is transmitted from the input buffer 314 to the output buffer 324, the sequence number of the packet is included with the packet. The packet and the sequence number of the packet are both stored in the output buffer 324. The transmitter 326 reads the packet from the output buffer 324 in the order of the sequence number of the packet. For example, when output buffer 324 contains both a first packet and a second packet in a packet stream, a packet with a lower sequence number may be read from output buffer 324 before a packet with a higher sequence number. Thus, transmitter 326 reads packets of the packet stream from output buffer 324 in a desired order, even when the packets are delivered to output buffer 324 out of order.

[0043] When processing a request for an out-of-order packet, the output queue 322 of the selected header plane 310 can indicate to the input queue 318 of the selected header plane 310 that the packet is an out-of-order packet (e.g., not an ordered packet). For example, the output queue 322 can assign a pseudo sequence number to the packet. The pseudo sequence number can be a predetermined value indicating that the packet is not part of a flow. When a packet in the output buffer 324 has a pseudo sequence number, the transmitter 326 can read the packet from the output buffer 324 at any time.

[0044] Figure 4 is a schematic diagram of a packet forwarding method 400 according to some embodiments. Figure 3 The packet forwarding method 400 is described below. The packet forwarding method 400 may be performed by the network switch 300 during forwarding of a packet from the input port 306 to the output port 308.

[0045] The network switch 300 performs step 402 to select a header plane that controls the transmission of packets from an input buffer to an output buffer, the header plane including an input queue and an output queue. Specifically, the selected header plane 310 controls the transmission of packets from an input buffer 314 to an output buffer 324. The selected header plane 310 includes an input queue 318 and an output queue 322. The packets are stored in the input buffer 314 by the receiver 312. The header plane selector 316 selects the selected header plane 310 based on the aforementioned criteria (such as the frame type and / or header of the packet). As part of the header plane selection, the header plane selector 316 can calculate a flow hash of the packet by hashing a field in the header of the packet. The field can be any of the aforementioned fields.

[0046] The network switch 300 executes step 404 to send the transmit request from the input queue of the header plane to the output queue of the header plane. Specifically, the transmit request is sent from the input queue 318 of the selected header plane 310 to the output queue 322 of the selected header plane 310. The transmit request may include a description of the packet, such as information from a header of the packet.

[0047] The network switch 300 performs step 406, and receives a transmission authorization from the output queue of the header plane at the input queue of the header plane, and the transmission authorization includes the sequence number of the packet. Specifically, at the input queue 318 of the selected header plane 310, the transmission authorization is received from the output queue 322 of the selected header plane 310. The output queue 322 sends the transmission authorization to the input queue 318 in response to its authorization of the transmission request. When the transmission request includes a description of the packet, the output queue 322 can authorize the transmission request based on the description of the packet. The transmission authorization may include the sequence number of the packet. The sequence number can be assigned as described above. If the packet is an unordered packet (e.g., not part of a packet flow), the sequence number can be a pseudo sequence number. If the packet is an ordered packet (e.g., part of a packet flow), the sequence number can be assigned to the packet based on the order of the packet in the packet flow. The transmission request for the packet can indicate whether it is an ordered packet or an unordered packet. The output queue 322 can assign the sequence number based on the order in which it receives the request. Since the requests are sent to the output queue 322 in the order in which they appear in the packet stream, sequence numbers can naturally be assigned to the packets in order.

[0048] The network switch 300 performs an optional step of selecting a data crossbar switch 336 from the plurality of data crossbar switches 336 of the switch fabric 304. As previously described, the selected data crossbar switch 336 may be selected based on a load of the selected data crossbar switch 336.

[0049] The network switch 300 executes step 408 to transfer the packet and the sequence number from the input buffer to the output buffer. Specifically, the packet and the sequence number of the packet are transferred from the input buffer 314 to the output buffer 324. The packet and its sequence number may be transferred through the previously selected data crossbar switch 336.

[0050] The network switch 300 performs step 410 to read the packet from the output buffer based on the sequence number. Specifically, the packet is read from the output buffer 324 based on the sequence number of the packet. The transmitter 326 reads the packet from the output buffer 324. When the packet is part of a packet stream, the packets of the packet stream are read from the output buffer 324 in the order of their sequence numbers. When the packet is an out-of-order packet, the transmitter 326 can read the packet at any time.

[0051] Figure 5 is a schematic diagram of a plane selection method 500 according to some embodiments. Figure 3 1 and 2 to describe the plane selection method 500. The plane selection method 500 may be performed by the network switch 300 (particularly, the header plane selector 316) to control forwarding of received packets.

[0052] The network switch 300 performs step 502 to receive a transmit request for a packet from the receiver 312. When the receiver 312 stores the received packet in the input buffer 314, the receiver 312 may send the transmit request to the header plane selector 316.

[0053] The network switch 300 performs step 504 to select a header plane 310 in the header plane 310 based on one or more headers of the packet. For example, the header plane 310 can be selected by calculating a flow hash of the packet by hashing fields in the header of the packet, and then selecting the header plane 310 based on the flow hash. The header fields of the hashed packet can be from an Ethernet header, an IP header, a TCP / UDP header, a combination thereof, and the like. If the packet is an encapsulated packet, the hashed header fields can be from the encapsulation header, as well as from an internal header of the encapsulated packet. The header plane 310 can also be selected based on other criteria in the aforementioned criteria, such as the frame type of the packet.

[0054] The network switch 300 executes step 506 to queue the transmit request at the input queue 318 of the selected header plane 310. The transmit request may then be sent to the output queue 322 of the selected header plane 310. Subsequently, a transmit grant may be sent back to the input queue 318 of the selected header plane 310, which may then transmit the packet to the output buffer 324 via one of the data crossbars 336, as previously described.

[0055] The technology of the present disclosure can achieve advantages. By utilizing multiple data crossbar switches 336 for packet transmission, multiple packets can be transmitted in parallel through the data crossbar switch 336, which can improve performance compared to using a single data crossbar switch. Thus, multiple packets can be transmitted from the input buffer 314 to the output buffer 324 per clock cycle. Therefore, even when forwarding small packets (including ordered packets and unordered packets), the linear rate of the network switch 300 can be maintained. Utilizing multiple header planes 310 allows independent control of the parallel transmission of packets through the data crossbar switch 336. Packet streams can be distributed on multiple data crossbar switches 336 at the packet level for forwarding packet streams.

[0056] In the foregoing example, the network switch 300 is described as having two data crossbars 336 and two header planes 310. However, it should be understood that the network switch 300 can have any desired number of data crossbars 336 and any desired number of header planes 310. Specifically, the network switch 300 can include more data crossbars 336 and / or more header planes 310 than previously described, which can allow even more packets to be transmitted per clock cycle. In addition, the switching techniques described herein can be applied to other types of networking, such as wireless networking.

[0057] Figure 6 is a block diagram of a network switch 600 according to some embodiments. The network switch 600 is a block diagram of a network switch 600 according to some embodiments. Figure 2 An example of a network switch 200 described herein. The network switch 600 may include a processor 602 and a memory 604. The memory 604 may be a non-transitory computer-readable medium storing a program for execution by the processor. In this embodiment, one or more modules within the network switch 600 may be partially or completely embodied as software for performing any of the functions described herein. For example, the memory 604 may include: instructions 612 for selecting a header plane that controls the transmission of packets from an input buffer to an output buffer, the header plane including an input queue and an output queue; instructions 614 for sending a transmission request from the input queue of the header plane to the output queue of the header plane; instructions 616 for receiving a transmission grant from the output queue of the header plane at the input queue of the header plane, the transmission grant including a sequence number of the packet; instructions 618 for transmitting the packet and the sequence number from the input buffer to the output buffer; and instructions 620 for reading the packet from the output buffer based on the sequence number.

[0058] In an exemplary embodiment, a device includes: an output buffer; a first data crossbar switch connected to the output buffer; a second data crossbar switch connected to the output buffer, the second data crossbar switch being separate from the first data crossbar switch; an input buffer connected to the first data crossbar switch and the second data crossbar switch; a first input queue configured to transmit a first packet from the input buffer to the output buffer through the first data crossbar switch; and a second input queue configured to transmit a second packet from the input buffer to the output buffer through the second data crossbar switch. In some embodiments of the device, the first packet is transmitted through the first data crossbar switch in parallel with the second packet being transmitted through the second data crossbar switch. In some embodiments, the device further includes: a header plane selector configured to distribute a first transmission request for the first packet to the first input queue and a second transmission request for the second packet to the second input queue. In some embodiments of the device, the header plane selector distributes the first transmission request and the second transmission request based on a first header of the first packet and a second header of the second packet. In some embodiments of the device, the header plane selector distributes the first transmission request and the second transmission request based on the first frame type of the first packet and the second frame type of the second packet. In some embodiments of the device, the first packet is part of a packet stream, and the first input queue is further configured to transmit each packet of the packet stream from the input buffer to the output buffer. In some embodiments, the device also includes: a first output queue, the first output queue is configured to receive a first transmission request for the first packet from the first input queue and send a first transmission grant to the first input queue; and a second output queue, the second output queue is configured to receive a second transmission request for the second packet from the second input queue and send a second transmission grant to the second input queue.In some embodiments, the device further comprises: a first request crossbar switch connected to a first input queue and a first output queue, the first input queue being configured to send a first transfer request to the first output queue through the first request crossbar switch; a second request crossbar switch connected to a second input queue and a second output queue, the second request crossbar switch being separated from the first request crossbar switch, the second input queue being configured to send a second transfer request to the second output queue through the second request crossbar switch; a first grant crossbar switch connected to the first input queue and the first output queue, the first output queue being configured to send a first transfer grant to the first input queue through the first grant crossbar switch; and a second grant crossbar switch connected to the second input queue and the second output queue, the second grant crossbar switch being separated from the first grant crossbar switch, the second output queue being configured to send a second transfer grant to the second input queue through the second grant crossbar switch. In some embodiments of the device, the first transfer grant comprises a first sequence number, the first output queue comprises a first sequencer configured to generate the first sequence number, the second transfer grant comprises a second sequence number, and the second output queue comprises a second sequencer configured to generate the second sequence number.

[0059] In an exemplary embodiment, a device includes: a plurality of header planes, each of the header planes including an input queue, an output queue, a request crossbar connected to the input queue and the output queue, and a grant crossbar connected to the input queue and the output queue; a receiver; and a header plane selector, the header plane selector being configured to: receive a transmit request for a packet from the receiver; select a header plane in the header plane based on a header of the packet; and queue the transmit request at the input queue of the header plane. In some embodiments of the device, the input queue is configured to send a transmit request to the output queue through the request crossbar, and the output queue is configured to send a transmit grant to the input queue through the grant crossbar. In some embodiments of the device, the header plane selector is configured to select the header plane by calculating a flow hash of the packet by hashing fields in the header of the packet, and the header plane is selected based on the flow hash.

[0060] In an exemplary embodiment, a method includes: selecting a header plane that controls the transfer of packets from an input buffer to an output buffer, the header plane including an input queue and an output queue; sending a transfer request from the input queue of the header plane to the output queue of the header plane; receiving a transfer grant from the output queue of the header plane at the input queue of the header plane, the transfer grant including a sequence number of the packet; transferring the packet and the sequence number from the input buffer to the output buffer; and reading the packet from the output buffer based on the sequence number. In some embodiments, the method also includes: calculating a flow hash of the packet by hashing a field in a header of the packet, the header plane being selected based on the flow hash. In some embodiments of the method, the header plane is selected based on a frame type of the packet. In some embodiments of the method, the transfer request includes a description of the packet, and the transfer request is granted by the output queue based on the description of the packet. In some embodiments of the method, the packet is an unordered packet and the sequence number is a pseudo sequence number. In some embodiments of the method, the packet is part of a packet stream, the sequence number is assigned to the packet based on an ordering of the packets in the packet stream, and a plurality of data crossbars are used to transfer each packet of the packet stream from an input buffer to an output buffer. In some embodiments, the method further comprises: selecting a data crossbar from a plurality of data crossbars, wherein transferring the packet and the sequence number from the input buffer to the output buffer comprises transferring the packet and the sequence number through the data crossbar. In some embodiments of the method, the data crossbar is selected based on a load of the data crossbar. The method may be implemented as instructions stored on a non-transitory computer-readable medium.

[0061] The foregoing summarizes the features of several examples so that those skilled in the art can better understand the various aspects of the present disclosure. Various modifications and combinations of the illustrative examples and other examples will be apparent to those skilled in the art upon reference to the specification. Therefore, the appended claims are intended to cover any such modifications.

Claims

1. A device, comprising: Output buffer; a first data crossbar switch connected to the output buffer; a second data crossbar switch, the second data crossbar switch being connected to the output buffer, the second data crossbar switch being separated from the first data crossbar switch; an input buffer connected to the first data crossbar switch and the second data crossbar switch; a first input queue configured to transmit a first packet from the input buffer to the output buffer through the first data crossbar; and A second input queue is configured to transmit a second packet from the input buffer to the output buffer through the second data crossbar.

2. The apparatus of claim 1, wherein the first packet is transmitted through the first data crossbar in parallel with the second packet being transmitted through the second data crossbar.

3. The device according to claim 1, further comprising: A header plane selector is configured to distribute a first transmit request for the first packet to the first input queue and a second transmit request for the second packet to the second input queue. 4 . The apparatus of claim 3 , wherein the header plane selector distributes the first transmit request and the second transmit request based on a first header of the first packet and a second header of the second packet.

5. The apparatus of claim 3, wherein the header plane selector distributes the first transmit request and the second transmit request based on a first frame type of the first packet and a second frame type of the second packet.

6. The apparatus of claim 1, wherein the first packet is part of a packet flow, and the first input queue is further configured to transfer each packet in the packet flow from the input buffer to the output buffer.

7. The device according to claim 1, further comprising: a first output queue configured to receive a first transmit request for the first packet from the first input queue and to send a first transmit grant to the first input queue; and A second output queue is configured to receive a second transmit request for the second packet from the second input queue and to send a second transmit grant to the second input queue.

8. The device according to claim 7, further comprising: a first request crossbar switch, the first request crossbar switch being connected to the first input queue and the first output queue, the first input queue being configured to send the first transfer request to the first output queue through the first request crossbar switch; a second request crossbar switch, the second request crossbar switch being connected to the second input queue and the second output queue, the second request crossbar switch being separate from the first request crossbar switch, the second input queue being configured to send the second transfer request to the second output queue through the second request crossbar switch; a first grant crossbar switch, the first grant crossbar switch being connected to the first input queue and the first output queue, the first output queue being configured to send the first transmit grant to the first input queue through the first grant crossbar switch; and A second grant crossbar switch is connected to the second input queue and the second output queue, the second grant crossbar switch is separated from the first grant crossbar switch, and the second output queue is configured to send the second transmission grant to the second input queue through the second grant crossbar switch.

9. The apparatus of claim 7, wherein the first transmit grant comprises a first sequence number, the first output queue comprises a first sequencer configured to generate the first sequence number, the second transmit grant comprises a second sequence number, and the second output queue comprises a second sequencer configured to generate the second sequence number.

10. A device, comprising: a plurality of header planes, each of the header planes comprising an input queue, an output queue, a request crossbar connected to the input queue and the output queue, and a grant crossbar connected to the input queue and the output queue; Receiver; and A header plane selector, the header plane selector being configured to: receiving a transmit request for a packet from the receiver; selecting a header plane among the header planes based on a header of the packet; as well as The transmit request is queued at the input queue of the header plane.

11. The apparatus of claim 10, wherein the input queue is configured to send the transmit request to the output queue through the request crossbar, and the output queue is configured to send the transmit grant to the input queue through the grant crossbar.

12. The apparatus of claim 10, wherein the header plane selector is configured to select the header plane by computing a flow hash of the packet by hashing a field in the header of the packet, the header plane being selected based on the flow hash.

13. A method comprising: selecting a header plane, the header plane controlling the transfer of packets from an input buffer to an output buffer, the header plane comprising an input queue and an output queue; sending a transmit request from the input queue of the header plane to the output queue of the header plane; receiving, at the input queue of the header plane, a transmit grant from the output queue of the header plane, the transmit grant comprising a sequence number for the packet; transferring the packet and the sequence number from the input buffer to the output buffer; as well as The packet is read from the output buffer based on the sequence number.

14. The method according to claim 13, further comprising: A flow hash for the packet is calculated by hashing fields in a header of the packet, and the header plane is selected based on the flow hash.

15. The method of claim 13, wherein the header plane is selected based on a frame type of the packet.

16. The method of claim 13, wherein the transfer request includes a description of the packet, and the transfer request is granted by the output queue based on the description of the packet.

17. The method of claim 13, wherein the packets are out-of-order packets and the sequence numbers are pseudo sequence numbers.

18. The method of claim 13, wherein the packets are part of a packet stream, the sequence numbers are assigned to the packets based on an ordering of the packets in the packet stream, and a plurality of data crossbar switches are used to transfer each packet in the packet stream from the input buffer to the output buffer.

19. The method according to claim 13, further comprising: A data crossbar is selected from a plurality of data crossbars, wherein transferring the packet and the sequence number from the input buffer to the output buffer comprises transferring the packet and the sequence number through the data crossbar.

20. The method of claim 19, wherein the data crossbar is selected based on a load of the data crossbar.

Images