Traffic management in network device

Abstract

The invention relates to traffic management in a network device. An example traffic management system for a network device includes a first memory storing a queue of packets received by the network device; a second memory that stores data of a scheduler of the queue that schedules transmission of the packet to a port of the network device; and an accelerator circuit comprising a third memory for storing a data structure, the accelerator circuit coupled to the second memory and operable to: load a field of the data structure in the third memory with data from a first node of the scheduler in a first iteration of a traversal operation in response to a command; updating the field of the data structure in the third memory during the first iteration of the traversal operation; and storing update data from the field to the first node in the second memory during the first iteration of the traversal operation.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to traffic management in network devices. BACKGROUND

[0002] Network devices, such as switches, routers, bridges, etc., employ traffic management to maintain efficient network performance, ensure fair resource allocation, and provide quality of service (QoS) in the network. Traffic management involves various techniques to control the flow of packets through the network, deciding how to handle, prioritize, and route the packets. Traffic management includes packet classification, queue management, scheduling, congestion management, and rate limiting and traffic shaping techniques. Network switches can classify packets based on defined rules and policies. Packets can be placed in different queues based on their classification. Network switches can have multiple queues to handle different classifications of packets. Scheduling algorithms decide the order in which network switches process and send packets from the queues. Common scheduling algorithms include priority queuing, first-in-first-out (FIFO), weighted fair queuing (WFQ), deficit weighted round robin (DWRR), etc. Scheduling algorithms manage congestion and ensure that high-priority packets receive proper bandwidth and minimum delay. Rate limiting controls the rate at which senders can send packets to ensure that the network is not overwhelmed. Traffic shaping smooths out packet bursts to ensure that a more smooth flow of packets adheres to a specified rate.

[0003] Traffic management becomes difficult when serving high-speed ports of network switches, such as optical line terminals (OLTs) in passive optical networks (PONs). Such network switches can include many queues (e.g., thousands of queues), each of which needs to consider many parameters for scheduling, congestion management, rate limiting, and traffic shaping. Furthermore, the scheduler can be hierarchical and include many levels, each with many parameters to consider. For low-bandwidth ports, a central processing unit (CPU) executing software can be used to implement the traffic management system. For high-bandwidth ports, such as those in OLTs (e.g., 10 gigabits per second (Gbps) to 400 Gbps), a CPU executing software alone is insufficient to implement the traffic management system. SUMMARY

[0004] In an embodiment, a traffic management system for a network device is described. The traffic management system includes a first memory that stores a queue of packets received by the network device. The traffic management system includes a second memory that stores data of a scheduler of the queue that schedules transmission of the packets to a port of the network device. The traffic management system includes an accelerator circuit that includes a third memory for storing a data structure. The accelerator circuit is coupled to the second memory and is operable to load a field of the data structure in the third memory with data from a first node of the scheduler in a first iteration of a traversal operation in response to a command. The accelerator is operable to update the field of the data structure in the third memory during the first iteration of the traversal operation. The accelerator is operable to store updated data from the field to the first node in the second memory during the first iteration of the traversal operation.

[0005] In an embodiment, a network device is described. The network device includes a queue manager operable to maintain a queue of packets, a port, a controller coupled to the queue manager and the port, a first memory that stores nodes of a scheduler of the queue that schedules transmission of the packets to the port, and an accelerator circuit that includes a second memory. The accelerator circuit is coupled to the controller and the first memory and is operable to load the second memory with data from each of a plurality of nodes of the nodes in a plurality of iterations in a traversal operation in response to a command from the controller. The accelerator is operable to update the second memory during the traversal operation of at least one of the plurality of iterations. The accelerator is operable to store updated data from the second memory to at least one of the plurality of nodes in the first memory during the traversal operation.

[0006] In an embodiment, a method of traffic management in a network device is described. The method includes storing packets received by the network device in a queue. The method includes storing nodes of a scheduler to be selected for the queue that schedules transmission of the packets to a port of the network device in a first memory. The method includes updating, by an accelerator circuit in response to a command from a controller, the scheduler by loading a field of a data structure with data from each of a plurality of nodes of the nodes in a plurality of iterations in a traversal operation, updating the field of the data structure during the traversal operation of at least one of the plurality of iterations, and storing updated data to at least one of the plurality of nodes in the first memory during the traversal operation. BRIEF DESCRIPTION OF DRAWINGS

[0007] Figure 1is a block diagram depicting a communication system in accordance with some embodiments.

[0008] Figure 2 is a block diagram depicting a channel of a network device in accordance with some embodiments.

[0009] Figure 3 is a block diagram depicting a controller of a traffic management system in accordance with some embodiments.

[0010] Figure 4 is a block diagram depicting a general scheduling unit in accordance with some embodiments.

[0011] Figure 5 is a block diagram depicting a scheduler node in accordance with some embodiments.

[0012] Figure 6 is a block diagram depicting a scheduler in accordance with some embodiments.

[0013] Figure 7 is a block diagram depicting an accelerator in accordance with some embodiments.

[0014] Figure 8 is a block diagram depicting a packet descriptor in accordance with some embodiments.

[0015] Figure 9 is a block diagram depicting a logical view of a traffic management system for a network device in accordance with embodiments.

[0016] Figure 10 is a flow diagram depicting a method of traffic management in a network device in accordance with embodiments.

[0017] Figure 11 is a flow diagram depicting a method of traversing a scheduler hierarchy in accordance with some embodiments.

[0018] Figure 12 is a flow diagram depicting a method of traffic management in response to updating a queue in accordance with some embodiments.

[0019] Figure 13 is a flow diagram depicting a method of transmitting a packet in accordance with some embodiments.

[0020] Figure 14 is a flow diagram depicting a method of updating a scheduler in accordance with some embodiments. DETAILED DESCRIPTION

[0021] Figure 1is a block diagram depicting a communication system 100 in accordance with some embodiments. The communication system 100 includes a service equipment 10, a network 11, a computer 21, an optical line terminal (OLT) 12, an optical distribution network (ODN) 16, an optical network terminal (ONT) 18, and a customer premises equipment (CPE) 20. The network 11 can be an electrical network, such as an Ethernet network, etc. The service equipment 10 and the computer 21 are connected to the network 11. The OLT 12 includes a traffic management system (TMS) 14, ports 15, and a queue manager 22. The ports 15 can send and receive data units, which can be packets, frames, segments, etc. A packet can be any data unit (e.g., an Internet Protocol (IP) packet). A packet can be encapsulated within another data unit (e.g., an IP packet encapsulated in an Ethernet frame). A packet can be encapsulated in another data structure, such as a packet descriptor (PD). The TMS 14 manages traffic sent by the ports 15. Traffic can be the amount of data sent or received during any given time period. The queue manager 22 can manage queues of packets for transmission.

[0022] A port can be a physical entity (e.g., a circuit) or a virtual entity (e.g., a logical entity implemented using software executing on a processor) that provides an input / output (IO) interface of a network device. The ports 15 can include one or more passive optical network (PON) ports 15A and one or more network-to-network interface (NNI) ports 15B. The OLT 12 is connected to the network 11 via the NNI ports 15B (e.g., Ethernet ports). Each NNI port 15B sends and receives packets to and from the network 11 using electrical signals. The OLT 12 is connected to the ODN 16 via the PON ports 15A. Each PON port 15A sends and receives packets to and from the ODN 16 using optical signals.

[0023] The ODN 16 includes optical transmission media (e.g., optical fibers) that connect the OLT 12 to the ONU / ONT 18. In some embodiments, the communication system 100 implements a PON. A PON is “passive” in that the ODN 16 uses passive components and does not require any power for signal transmission, except for the OLT 12 and the ONTs 18. The ONT 18 (also referred to as an optical network unit (ONU)) converts optical signals transmitted by the OLT 12 and received from the ODN 16 to electrical signals. The CPE 20 includes a network device (e.g., a switch, a router, etc.) that processes packets received from the ONT 18 via electrical signals. The computer 21 executes management software 23, which can control aspects of the OLT 12, as further described herein.

[0024] Service equipment 10, network 11, computer 21, and OLT 12 can be located in a central office (CO). CPE 20 can be distributed across many locations (e.g., houses, businesses, etc.) and include network devices of many customers. Service equipment 10 can send packets to customers concurrently. TMS 14 can maintain many queues of packets, e.g., one or more queues for each of the customers served by service equipment 10. A queue can store a set of packets that can be retrieved in a determined order, e.g., insertion order. PON port 15A can be a high-speed port that sends packets to many customers over ODN 16 (e.g., on the order of Gbps, e.g., between 10 Gbps and 400 Gbps). TMS 14 employs hardware acceleration to implement traffic management for PON port 15A. Hardware acceleration involves a controller executing software and offloading some tasks to specific hardware circuitry. Hardware acceleration implemented by TMS 14 is described below.

[0025] Communication system 100 is merely one example system in which TMS 14 can be deployed for traffic management. While TMS 14 is shown in OLT 12, TMS 14 can be implemented in any type of network device. While TMS 14 is described as performing traffic management for PON port 15A, TMS 14 can perform traffic management for any type of network port, e.g., an Ethernet port. While TMS 14 is described in the context of a PON, TMS 14 can be implemented in a network device in a different type of network, e.g., an electrical network or a different type of optical network. In general, TMS 14 provides traffic management for a port of a network device. In some embodiments, TMS 14 provides traffic management for a high-speed port that can send data at a rate between 10 Gbps and 400 Gbps. However, TMS 14 can provide traffic management for ports of other speeds, e.g., those with data rates less than 10 Gbps or greater than 400 Gbps.

[0026] Figure 2 is a block diagram depicting channels of OLT 12 according to some embodiments. Channels of OLT 12 include queue manager 22, TMS 14, and port 15. OLT 12 can include device manager 24 coupled to each of queue manager 22, TMS 14, and port 15. Queue manager 22 is coupled to TMS 14. TMS 14 is coupled to port 15. Queue manager 22 receives incoming packets and generates packet descriptors (PDs). A PD can be a data structure that includes a packet and associated metadata. Metadata of a PD can be attributes of a packet, e.g., packet length, source / destination address, priority information, quality of service (QoS) information, timestamp, error check information, etc. Queue manager 22 includes a set of first-in-first-out memories (FIFOs) 301, 302,..., 30 N(where N is an integer greater than 1). The queue manager 22 inserts the PDs into the FIFOs 301...30 N implemented queues.

[0027] For example, the service appliance 10 can generate packets for multiple customers each having a unique IP address. Each of the FIFOs 301...30 N of the queue manager 22 can be associated with a destination IP address or other identifier of a customer. The queue manager 22 can insert the PDs into the FIFOs 301...30 N based on the destination IP or other customer identifier. In general, the FIFOs 301...30 N may be associated with any attribute of the PD metadata and can insert the PDs into the queues based on such attributes. The device manager 24 can send commands to the queue manager 22 to add or remove queues. The commands can be any data associated with an operation to be performed. Thus, as queues are added or removed (e.g., adding a new customer or removing an existing customer), the number N of FIFOs 301...30 N may change over time.

[0028] The TMS 14 includes a controller 26 and a general scheduling unit (GSU) 28. The controller 26 executes software to orchestrate scheduling PDs for transmission over the port 15. The device manager 24 configures the GSU 28 with a scheduler. A scheduler for a set of queues can be logic that schedules packets for transmission from the queues in a sequence. The scheduler can be hierarchical, including multiple scheduler nodes (examples described below). A node of the scheduler (also referred to as a scheduler node) can itself be a scheduler that selects an output from a set of inputs. The inputs of a leaf node of the scheduler are queues. The output of a root node of the scheduler supplies the currently selected queue. The root node is the last node in the hierarchy of the scheduler. When a queue is updated with a PD, the queue manager 22 can send a notification to the controller 26, and in response, the controller 26 can send a command to the GSU 28 to update the scheduler. The notification can be any data that indicates or requests an action. The port 15 can send a notification to the controller 26 to request a PD for transmission, and in response, the controller 26 can send a command to the GSU 28 to cause the PD to be transmitted and update the scheduler. The controller 26 outputs the PD to the port 15 for transmission.

[0029] Management software 23 executing on computer 21 can interact with device manager 24 to control aspects of queue manager 22, ports 15, and TMS 14. For example, management software 23 can interact with device manager 24 to add or remove queues managed by queue manager 22. Management software 23 can interact with device manager 24 to configure a scheduler in GSU 28. Management software 23 can interact with device manager 24 to update software of controller 26. Management software 23 can interact with device manager 24 to set parameters (e.g., transmission rates) of ports 15. Device manager 24 can include, for example, a CPU executing software that provides an application programming interface (API), a graphical user interface (GUI), etc. for controlling aspects of OLT 12.

[0030] Figure 2 One channel of OLT 12 is shown by way of example. In some embodiments, TMS 14 can serve multiple ports 15. In some embodiments, OLT 12 can include multiple instances of TMS 14 for multiple ports 15. In some embodiments, queue manager 22 can serve multiple instances of TMS 14. In some embodiments, OLT 12 can include multiple instances of queue manager 22, each serving one of multiple instances of TMS 14. While OLT 12 is described by way of example, other types of network devices can be used. Figure 2 The structures shown in FIG. 1 can be implemented in any type of network device, including any type of router, network switch, etc.

[0031] Figure 3is a block diagram depicting controller 26 of TMS 14 according to some embodiments. The controller includes CPU 32 coupled to memory 34. Memory 34 can be random access memory (RAM). Memory 34 stores software 36 and transmission PDs 44. Software 36 includes update queue task 38, transmission task 40, and timer task 42. Queue manager 22 wakes up update queue task 38 to inform controller 26 of a queue update. In response, update queue task 38 sends a command to GSU 28 to update the scheduler based on the queue update. Port 15 wakes up transmission task 40 to inform controller 26 of a request to transmit a PD. In response, transmission task 40 sends a command to GSU 28 to obtain a PD to be transmitted from a selected queue and perform an update to the scheduler. GSU 28 stores transmission PDs 44 in memory 34. Transmission task 40 can then send transmission PDs 44 to port 15 to transmit packets. Timer task 42 can periodically send a command to GSU 28 to update the scheduler (e.g., for periodic updates to scheduler parameters). In an embodiment, memory 34 includes firmware (e.g., non-volatile memory) that stores software 36, which CPU 32 can load into volatile memory (e.g., RAM) for execution.

[0032] Figure 4 is a block diagram depicting GSU 28 according to some embodiments. GSU 28 includes input / output (IO) circuitry 46, command FIFO 48, accelerator 50 (also referred to as accelerator circuitry), memory 52, and PD FIFOs 56i…56 N IO circuitry 46 provides an interface for GSU 28 to queue manager 22, controller 26, and device manager 24. Queue manager 22 is coupled to IO circuitry 46 to store PDs in PD FIFOs 56i…56 N PD FIFOs 56i…56 N correspond to FIFOs 301…30 N FIFOs 301…30 N and PD FIFOs 56i…56 N combine to implement N queues managed by queue manager 22. FIFOs 301…30 N each store a PD at the tail of a respective queue. PD FIFOs 56i…56 NEach stores a PD at the head of the corresponding queue. Memory 52 can be a RAM and stores scheduler nodes 54. Scheduler nodes 54 comprise the nodes of the scheduler implemented by GSU 28. Each scheduler node 54 includes data as further described below. Device manager 24 is coupled to IO circuit 46 to store scheduler nodes 54 in memory 52 when initializing GSU 28 with a scheduler. Controller 26 is coupled to IO circuit 46 to store commands in command FIFO 48.

[0033] Accelerator 50 (or accelerator circuit) can be a circuit operable to perform processing on behalf of controller 26 using digital logic. Accelerator 50 is coupled to command FIFO 48, memory 52, and PD FIFOs 56i…56 N Accelerator 50 can receive commands sent by controller 26 from command FIFO 48. Accelerator 50 can read scheduler nodes 54 from memory 52 as it traverses the hierarchy of the scheduler to perform updates of the scheduler and determine the currently selected queue. Accelerator 50 can read PDs from PD FIFOs 56i…56 N Accelerator 50 is coupled to IO circuit 46 to output PDs to controller 26.

[0034] Figure 5 is a block diagram depicting a scheduler node 54 according to some embodiments. Scheduler node 54 includes input data 502, output data 514, scheduling data 522, and traffic shaping data 528. Input data 502 includes data for one or more inputs 504 of scheduler node 54. Input 504 can include an input ID 505 storing an identifier of the input. For each input 504 from a queue, a queue ID 506 stores an identifier of the queue being serviced, and a number of PDs in queue 508 stores the number of PDs in the queue being serviced. For each input from a previous scheduler node in the hierarchy, input 504 can include an indication of whether the previous scheduler node has a PD to transmit (stored by indicator 510).

[0035] The output data 514 includes data for one or more outputs 516 of the scheduler node 54. Each output 516 includes a type 517, such as a committed information rate (CIR) output or an excess information rate (EIR) output. In some embodiments, the GSU 28 implements traffic shaping to ensure that each queue receives a determined amount of bandwidth. The CIR is the minimum bandwidth guaranteed to a queue. The EIR is excess bandwidth that can be allowed for a queue, but with potential limits. Queues with traffic within the CIR can be selected for outputs 516 with the type CIR. Queues with traffic within the EIR can be selected for outputs 516 with the type EIR. Each output 516 includes a selected input 518, which can be an identifier of a selected one of the inputs 504. Each output 516 can include a next node connectivity 520, which can store an identifier of a next scheduler node 54 in the hierarchy of schedulers.

[0036] The GSU 28 implements the scheduling algorithm in digital logic, as discussed further below. The scheduling algorithm can be any known scheduling algorithm or a combination of such known scheduling algorithms. In some embodiments, the GSU 28 implements a combination of strict priority scheduling and DWRR scheduling. Strict priority scheduling is selection of queues based on priority, referred to as strict priority. DWRR scheduling is designed to handle different traffic flows with different QoS requirements. In DWRR, in addition to a deficit counter, each queue is assigned a weight. The weight reflects the relative importance or priority of the queue. DWRR uses the deficit counter for each queue to track the amount of data allowed to be sent by the queue. The deficit counter is incremented at the start of each round in proportion to the weight of the queue. While strict priority and DWRR are described as example scheduling algorithms, it should be understood that the scheduler node 54 can implement different types of scheduling algorithms in various combinations.

[0037] Each input 504 can include a group assignment 512. For example, the input groups can be divided between strict priority groups and DWRR priority groups (e.g., high strict priority, high priority DWRR, low strict priority, low priority DWRR, etc.). The scheduling data 522 includes parameters 524 and state data 526 used by the scheduling algorithm implemented by the GSU 28. The parameters 524 can include, for example, increment values for DWRR counters, DWRR weights for inputs, etc. The state data 526 can include, for example, current values for DWRR counters, current input selections for DWRR groups, etc. The traffic shaping data 528 includes parameters 530 and state data 532 used by the traffic shaping algorithm implemented by the GSU 28. The parameters 530 can include, for example, maximum and increment values for CIR / EIR output shapers. The state data 532 can include, for example, current values for CIR / EIR output shapers.

[0038] Scheduler nodes 54 can be used for different types of nodes of the scheduler, e.g., leaf nodes with queues as input, intermediate nodes, or root nodes. Some data of scheduler nodes 54 can be valid for a given type of node, while other data can be invalid (e.g., set to null) for the type. While specific types of data are shown in the examples of Figure 5

[0039] Figure 6 is a block diagram depicting a scheduler 600 according to some embodiments. Scheduler 600 includes a hierarchy of scheduler nodes 54. Scheduler nodes 541...544 include leaf nodes of the scheduler. Inputs to scheduler nodes 541...544 include outputs of queues. Each scheduler node 541...544 has two outputs of type CIR and EIR. CIR / EIR outputs from scheduler nodes 541 and 542 are coupled to inputs of scheduler node 545, which is an intermediate node. CIR / EIR outputs from scheduler nodes 543 and 544 are coupled to inputs of scheduler node 546, which is another intermediate node. CIR / EIR outputs of scheduler nodes 545 and 546 are coupled to inputs of scheduler node 547, which is a root node of scheduler 600. Scheduler node 547 includes an output that provides a selected one of the queues. Scheduler node 547 can also have an input that selects directly from the outputs of the queues, as shown in the example. Note that scheduler 600 is formed by the data structures (e.g., next node connectivity 520) of scheduler nodes 54 in link memory 52.

[0040] Figure 7 is a block diagram depicting an accelerator 50 according to some embodiments. Accelerator 50 includes digital logic 702, memory 704, and IO circuitry 710. IO circuitry 710 provides an input / output interface for accelerator 50, e.g., for communication with memory 52, PDFIFOs 561...56 N Figure 5 ​​The fields can be part of a data structure 706. The memory 704 can be a RAM. For example, the data structure 706 can be stored in registers, with each field 708 being stored by a register. The digital logic 702 can receive commands from the command FIFO 48 through the IO circuit 710. In response to the commands, the digital logic 702 performs a walk operation by loading the memory 704 with a scheduler node 54, where the data of the scheduler node 54 is stored in the fields 708 of the data structure 706. A walk operation can be an operation that processes one or more scheduler nodes in succession. An iteration of the walk operation can process one scheduler node. The digital logic 702 can then read the data from the fields 708 and update the data in the fields 708 according to a scheduling and traffic shaping algorithm. The digital logic 702 can then update the data of the scheduler node 54 in the memory 52 with the updated data from the fields 708 via the IO circuit 710. The digital logic 702 can then load the memory 704 with the next scheduler node 54. The walk operation loads the scheduler nodes 54 into the memory in succession from the leaf nodes of the scheduler towards the root node. As a result of the walk operation, the digital logic 702 can store the currently selected queue 712 in the memory 704. The digital logic 702 can also read the PD from the PD FIFO 561…56 N The digital logic 702 can output the PD to the controller 26 through the IO circuit 710 and the IO circuit 46. The digital logic 702 can include asynchronous and synchronous digital circuits configured to perform the functions of the accelerator 50 described herein.

[0041] Figure 8 is a block diagram depicting a packet descriptor (PD) 802 according to some embodiments. The packet descriptor 802 includes one or more packets 804 and packet metadata 806. The packet metadata 806 can include attributes of the packets 804, such as packet length, source / destination address, priority information, quality of service (QoS) information, time stamp, error check information, etc.

[0042] Figure 9 is a block diagram depicting a logical view of a traffic management system 900 of a network device according to embodiments. The traffic management system 900 includes queues 9021…902 N , a scheduler 904, and ports 906. The queues 9021…902 N store PDs 802 generated from incoming packets (e.g., by the queue manager 22). The queues 9021…902 N are implemented by FIFOs 301…30 N and PD FIFOs 561…56 N , where the FIFOs 301…30 N store tail PDs, and the PD FIFOs 561…56N The head PDs are stored. The scheduler 904 schedules the PDs for transmission by the ports 906 by selecting among the queues 902i...902 N The scheduler 904 can be implemented in hardware by the GSU 28 under control of the controller 26, as described above.

[0043] Figure 10 is a flowchart depicting a method 1000 of traffic management in a network device according to an embodiment. The method 1000 begins at step 1002, where the management software 23 interacts with the device manager 24 of the network device to configure the GSU 28 with scheduler data. The management software 23 can interact with the device manager 24 to store the scheduler nodes 54 in the memory 52 of the GSU 28. The management software 23 can initialize the data of each scheduling node 54 (e.g., initialize the input data 502, the output data 514, the scheduling data 522, and the traffic shaping data 528).

[0044] At step 1004, the queue manager 22 fills the queues with input packets. In some embodiments, the queue manager 22 generates PDs from the input packets and queues the PDs. The PDs are assigned to different queues based on their attributes. The queue manager 22 can store the PDs in the FIFOs 30 and the PD FIFOs 56. Each FIFO 30 can store the tail of a respective queue, and each PDFIFO 56 can store the head of a respective queue.

[0045] At step 1006, the controller 26 receives notifications from the queue manager 22 and the ports 15 and generates commands for the GSU 28. The notifications from the queue manager 22 can indicate that a PD has been added to a PD FIFO 56 (e.g., a PD has been added to the head of a queue). The notifications from the ports 15 can indicate the number of PDs that are requested for transmission. The controller 26 can generate commands in response to the notifications. The update queue task 38 can generate commands for the GSU 28 in response to notifications from the queue manager 22. The transmit task 40 can generate commands for the GSU 28 in response to notifications from the ports 15. The controller 26 can autonomously generate some commands. For example, the timer task 42 can generate commands to periodically update the scheduler data.

[0046] At step 1008, the GSU 28 traverses the scheduler nodes 54 in response to each command. As Figure 6As shown in the example of FIG. 6, the scheduler 600 can include a hierarchy of scheduler nodes 54. The GSU 28 can perform a traversal operation in response to a command. The traversal operation can include the GSU 28 successively processing one or more scheduler nodes 54 in one or more iterations. Thus, at step 1010, the accelerator 50 successively loads scheduler nodes in each iteration of a traversal operation. In an iteration of a traversal operation, the accelerator 50 loads a field 708 of the data structure 706 in the memory 704 with data from a scheduler node 54. The traversal operation can include a single iteration or multiple iterations. In the case of multiple iterations, the accelerator 50 loads the field 708 of the data structure 706 with data of a different scheduler node at each iteration. At step 1012, the accelerator 50 can update one or more scheduler nodes 54 with update data during the traversal operation. The accelerator 50 can store the update data to the one or more scheduler nodes 54 in the memory 52. The update data can include new values, such as new values of the state data 526 of the scheduling data 522, new values of the state data 532 of the traffic shaping data 528, or new values of the selected input 518.

[0047] At step 1014, the GSU 28 selects a queue after each traversal operation. The accelerator 50 can store a currently selected queue 712 in the memory 704. The currently selected queue 712 can store a selected queue output by a root node of the scheduler, such as the scheduler node 547 of the scheduler 600. After each traversal operation, the value of the currently selected queue 712 can change or remain the same, depending on whether the traversal operation resulted in changing the selected queue of the root node in the scheduler. In general, the GSU 28 selects a queue after each traversal operation, whether the queue has changed from a previous traversal operation or remained the same as a previous traversal operation.

[0048] At step 1016, the controller responds to the notification received at step 1006. For example, in response to the notification from the port 15, the controller 26 can command the GSU 28 to provision a PD from the selected queue. The GSU 28 can store the PD as the transport PD 44 in the memory 34. The controller 26 can then send the transport PD 44 to the port 15. The port 15 can then transmit packets from the transport PD 44. In response to the notification from the queue manager 22, the controller can respond with an acknowledgement. The queue manager 22 can wait for the acknowledgement before sending additional notifications.

[0049] Figure 11 is a flow diagram depicting a method 1100 of traversing a scheduler hierarchy, according to some embodiments. The method 1100 begins at step 1102, where the accelerator 50 begins a traversal operation. For example, the accelerator 50 can begin a traversal operation in response to a command received from the controller 26 (e.g.,Figure 10 Step 1008) of the method 1000 shown in FIG. 10. At step 1104, the accelerator 50 determines the leaf scheduler node from the current command and selects the leaf scheduler node to be processed. The leaf scheduler node has input from the queue, as Figure 6 shown in the example of FIG. 11.

[0050] At step 1106, the accelerator 50 loads the data structure 706 with data from the selected scheduler node. The accelerator 50 can load the fields 708 (e.g., fields of input data 502, fields of output data 514, fields of scheduling data 522, and fields of traffic shaping data 528) with corresponding data from the scheduler node 54. At step 1108, the accelerator 50 updates the fields 708 of the data structure 706 based on the operation of the scheduler. As described above, the scheduler can implement a scheduling algorithm, such as a combination of strict scheduling and DWRR scheduling. The digital logic 702 of the accelerator 50 executes the scheduling algorithm and updates the fields 708 (e.g., updates fields associated with state data 526 of scheduling data 522, state data 532 of traffic shaping data 528, selected input 518, etc.). At step 1110, the accelerator 50 stores the updated data to the selected scheduler node (e.g., updates to scheduling data 522, traffic shaping data 528, selected input 518, etc.).

[0051] At step 1112, accelerator 50 determines whether an early exit condition exists for exiting the traversal operation. For some types of commands, the early exit condition can be that the output of the selected dispatch node does not change. That is, accelerator 50 can check whether the selected input 518 has changed as a result of the operation of the scheduler. If the early exit condition exists, method 1100 proceeds to step 1114. For other types of commands (e.g., transmit (TX) commands discussed below), there can be no early exit condition, and the scheduler can be traversed from leaf to root for each TX command. At step 1114, accelerator 50 determines whether the selected node is a root node of the scheduler. If not, method 1100 proceeds to step 1116, where accelerator 50 selects the next scheduler node in the scheduler hierarchy. Accelerator 50 can determine the next scheduler node based on the next node connectivity 520 of the selected scheduler node. Then, method 1100 returns to step 1106 and repeats. If at step 1114, the selected node is the root node, method 1100 proceeds to step 1118. At step 1118, accelerator 50 records the change in the queue selected by the scheduler, if any. That is, the output of the root node is the output of the scheduler, which can or can not have changed in response to the traversal operation. Accelerator 50 updates the currently selected queue 712 in response to the change in the output of the root node. Method 1100 proceeds from step 1118 to step 1120. If at step 1112, the output of the selected scheduler node has not changed, method 1100 proceeds to step 1120. At step 1120, accelerator 50 completes the traversal operation.

[0052] Figure 12 is a flow diagram depicting a method 1200 of updating traffic management in response to an updated queue according to some embodiments. Method 1200 begins at step 1202, where queue manager 22 sends a notification to controller 26 indicating an updated queue. Queue manager 22 can update the queue by inserting a new PD into PD FIFO 56. At step 1204, controller 26 sends a command to GSU 28 identifying the updated queue (e.g., by queue number). At step 1206, GSU 28 determines a leaf scheduler node having the updated queue as input and performs a traversal operation. The traversal operation can result in a change in the currently selected queue for transmission, as described above. The traversal operation of a queue update can include an early exit condition for exiting the traversal operation if the output of the selected dispatch node does not change. In other embodiments, there is no early exit condition.

[0053] Figure 13is a flowchart depicting a method 1300 of transmitting packets according to some embodiments. The method 1300 begins at step 1302, where the port 15 sends a notification to the controller 26 indicating a request for PDs to be transmitted. At step 1304, the controller 26 sends a command to the GSU 28 with a certain number of requested PDs. At step 1306, the GSU 28 dequeues the PDs from the currently selected queue and outputs the PDs to the controller 26. At step 1308, the GSU 28 determines the leaf scheduler node with the updated queue as input and performs a traversal operation. The traversal operation can result in a change of the currently selected queue for transmission, as described above. The traversal operation of the TX command can not have an early exit condition. Even if there is no change in the output of the node, the GSU 28 can traverse the scheduler from leaf to root (e.g., in order to update shapers and DWRR counters across related nodes). At step 1310, the GSU 28 determines whether there are more PDs to be transmitted. If so, the method 1300 returns to step 1306. Otherwise, the method 1300 proceeds to step 1312, where the GSU 28 ends the command.

[0054] The methods 1200 and 1300 can be understood with respect to the following example. Assume Figure 6 The scheduler 600 shown in FIG. 6. Starting with no valid PDs in the system. The controller 26 then receives an update notification from the queue manager 22 indicating that the queue 9021 connected to the scheduler node 541 has a new PD (e.g., the queue manager 22 inserted a new PD into the PD FIFO 561). In this case, the GSU 28 performs a traversal operation. In the first iteration, the accelerator 50 loads the scheduler node 541 and updates its data according to the scheduling algorithm. The result is that the selected input of the scheduler node 541 is the queue 9021 (e.g., changed from none to the queue 9021). In the second iteration, the accelerator 50 loads the scheduler node 545 and updates its data according to the scheduling algorithm. The result is that the selected input of the scheduler node 545 is the queue 9021 (e.g., changed from none to the queue 9021). In the third iteration, the accelerator 50 loads the scheduler node 547 and updates its data according to the scheduling algorithm. The result is that the selected input of the scheduler node 547 is the queue 9021 (changed from none to the queue 9021). Since the scheduler node 547 is the root node, the accelerator 50 completes the traversal operation.

[0055] Thereafter, the controller 26 receives an update notification from the queue manager 22 indicating that the queue 9021 has another new PD. The GSU 28 performs a traversal operation. In the first iteration, the accelerator 50 loads the scheduler node 541 and performs the scheduling algorithm. The result is that the selected input of the scheduler node 541 remains unchanged, i.e., the queue 9021. Thus, the accelerator 50 completes the traversal operation.

[0056] Thereafter, controller 26 receives an update notification from queue manager 22 indicating that queue 9022 has a new PD (e.g., queue manager 22 inserted a new PD into PD FIFO 562). Assume that queue 9022 is connected to scheduler node 542. In this case, GSU 28 performs a traversal operation. In the first iteration, accelerator 50 loads scheduler node 542 and updates its data according to the scheduling algorithm. The result is that the selected input of scheduler node 542 is queue 9022 (e.g., changed from none to queue 9022). In the second iteration, accelerator 50 loads scheduler node 545 and updates its data according to the scheduling algorithm. Assume that the input from scheduler node 542 has a higher priority than scheduler node 541. The result is that the selected input of scheduler node 545 is queue 9022 (e.g., changed from queue 9021 to queue 9022). In the third iteration, accelerator 50 loads scheduler node 547 and updates its data according to the scheduling algorithm. The result is that the selected input of scheduler node 547 is queue 9022 (changed from queue 9021 to queue 9022). Since scheduler node 547 is the root node, accelerator 50 completes the traversal operation.

[0057] Thereafter, controller 26 receives a notification from port 15 requesting the transmission of two PDs. Controller 26 sends a command to GSU 28 requesting two PDs. GSU 28 uses the currently selected queue 712 (set to queue 9022) to send PDs to controller 26. Since queue 9022 is connected to scheduler node 542, GSU 28 starts a traversal operation from scheduler node 542. In the first iteration, accelerator 50 loads scheduler node 542 and updates its data according to the scheduling algorithm. The result is that there are no more PDs in queue 9022, and the selected input of scheduler node 542 changes to none. In the second iteration, accelerator 50 loads scheduler node 545 and updates its data according to the scheduling algorithm. Since its only valid input is from queue 9021, the selected input of scheduler node 545 changes to queue 9021. In the third iteration, accelerator 50 loads scheduler node 547 and updates its data according to the scheduling algorithm. The selected input of the root changes to queue 9021. Since scheduler node 547 is the root node, accelerator 50 completes the traversal operation.

[0058] The GSU 28 determines that another PD has been requested by the command and sends the PD to the controller 26 using the currently selected queue, which is now set to queue 9021. Since queue 9021 is connected to scheduler node 541, the GSU 28 starts the traversal operation from scheduler node 541. In the first iteration, the accelerator 50 loads scheduler node 541 and updates its data according to the scheduling algorithm. The result is that the selected input of scheduler node 541 remains at queue 9021. Since there is no change in the output of scheduler node 541, the accelerator 50 completes the traversal operation. Since the GSU 28 has output two PDs as requested by the command, the GSU 28 signals the end of the command to the controller 26.

[0059] Figure 14 is a flowchart depicting a method of updating a scheduler according to some embodiments. The method 1400 begins at step 1402, where the controller 26 performs the timer task and sends a command to the GSU 28 to update the scheduler data. At step 1404, the GSU 28 determines the scheduler nodes for updating and performs the traversal operation, as described above. In this way, the controller 26 can periodically update the scheduler data.

[0060] While some processes and methods have been described as having various sequential stages, one or more embodiments could alternatively be conducted in an arrangement where stages are performed at least partially on an overlapping basis, or even in an arrangement where stages are performed on a substantially concurrent basis. Also, while some processes and methods have been described as having various sequential stages, one or more embodiments could alternatively be conducted in an arrangement where stages are performed at least partially on an overlapping basis, or even in an arrangement where stages are performed on a substantially concurrent basis.

[0061] As used herein, the phrase “at least one of” preceding a series of items, with the term “and” or “or” used to separate items in the list, modifies the entire list of items as opposed to each member of the list (i.e., each item). The phrase “at least one of” does not require selection of at least one of each item listed; rather, the phrase allows a meaning that includes at least one of any one of the items, and / or at least one of any combination of the items. By way of example, the phrases “at least one of A, B, and C” or “at least one of A, B, or C” each mean A alone, B alone, C alone, A and B together, A and C together, B and C together, or A and B and C together. In instances where it is intended that selection be “at least one of each of A, B, and C” or alternatively “at least one of A, at least one of B, and at least one of C,” it is specifically described as such.

[0062] As used herein, the term “coupling” and its derivatives include: (a) electrical and communication coupling; and (b) does not imply a direct connection but may include intermediary elements unless described as “direct coupling”.

[0063] Although one or more embodiments of the invention have been described in detail for ease of understanding, certain changes may be made within the scope of the claims. Therefore, the described embodiments should be considered illustrative rather than restrictive, and the scope of the claims is not limited to the details given herein, but may be modified within the scope of the claims and their equivalents. In the claims, unless expressly stated in the claims, elements and / or steps do not imply any particular order of operations.

[0064] The boundaries between components, operations, and data storage are somewhat arbitrary, and specific operations are described within the context of a particular illustrative configuration. Other functional assignments are conceivable and fall within the scope of this invention. Generally, structures and functions presented as individual components in the exemplary configuration can be implemented as combined structures or components. Similarly, structures and functions presented as single components can be implemented as single components. These and other variations, additions, and modifications fall within the scope of the appended claims.

Claims

1. A traffic management system for a network device, comprising: A first memory stores a queue of packets received by the network device; A second memory stores data of the scheduler of the queue that schedules the transmission of the packets to the ports of the network device; and An accelerator circuit includes a third memory for storing data structures, the accelerator circuit being coupled to a second memory and operable to: In response to a command, in the first iteration of the traversal operation, the fields of the data structure in the third memory are loaded with data from the first node of the scheduler; During the first iteration of the traversal operation, the field of the data structure in the third memory is updated; and During the first iteration of the traversal operation, the updated data from the field is stored in the first node of the second memory.

2. The flow management system according to claim 1, wherein the accelerator circuit is operable to: In the second iteration of the traversal operation, the data of the second node of the scheduler referenced by the first node is loaded into the field of the data structure in the third memory; During the second iteration of the traversal operation, the field of the data structure in the third memory is updated; and During the second iteration of the traversal operation, the updated data from the field is stored in the second node of the second memory.

3. The flow management system according to claim 2, wherein the accelerator circuit is operable to: It is determined that the second node of the scheduler is the root node; and The root node determines the selected one in the queue.

4. The traffic management system according to claim 1, further comprising: The controller includes a central processing unit (CPU) that executes software stored in a fourth memory, and the controller is operable to send the commands to the accelerator circuitry.

5. The traffic management system of claim 4, wherein the accelerator circuit is operable to select a first queue in the queue and output packets from the first queue to the controller in response to the traversal operation, and wherein the controller is operable to output the packets to the port.

6. The traffic management system of claim 5, wherein the controller is operable to receive a notification from the port and, in response to the notification, send the command to the accelerator circuit.

7. The traffic management system of claim 4, wherein the CPU executes a timer task of the software, and wherein the CPU sends the command to the accelerator circuit in response to the timer task.

8. The traffic management system of claim 4, wherein the network device includes a queue manager, the docking manager is coupled to the controller and the accelerator circuitry, wherein the queue in the first memory receives the packets from the queue manager, and wherein the controller is operable to send the command to the accelerator circuitry in response to a notification from the queue manager.

9. A network device comprising: A queue manager, which operates to maintain queues of groups; port; A controller, which is coupled to the queue manager and the port; A first memory stores a node of the scheduler that schedules the transmission of the packets to the port; and An accelerator circuit, comprising a second memory, is coupled to the controller and the first memory and is operable to: In response to a command from the controller, the second memory is loaded sequentially in multiple iterations during the traversal operation with data from each of the multiple nodes of the node; The second memory is updated during the traversal operation of at least one of the multiple iterations; and During the traversal operation, the updated data from the second memory is stored in at least one of the plurality of nodes in the first memory.

10. The network device of claim 9, wherein the controller includes a central processing unit (CPU) that executes software stored in a third memory.

11. The network apparatus of claim 9, wherein the port is operable to send a notification for transmitting a packet to the controller, wherein the controller is operable to send a command to the accelerator circuit in response to the notification, wherein the accelerator circuit is operable to identify a first queue in the queues selected by the scheduler, wherein the accelerator circuit is operable to dequeue a packet from the first queue and output the packet to the controller, and wherein the controller is operable to output the packet to the port.

12. The network apparatus of claim 11, wherein the accelerator circuitry is operable to perform the traversal operation in response to dequeuing the packet from the first queue.

13. The network device of claim 12, wherein the accelerator circuitry is operable to identify a second queue among the queues selected by the scheduler after the traversal operation.

14. The network apparatus of claim 11, further comprising: A first-in-first-out (FIFO) memory having an input coupled to the queue manager and an output coupled to the accelerator circuitry; The accelerator circuit dequeues the packet from the first queue by reading from the first FIFO in the FIFO.

15. A method for traffic management in a network device, comprising: The packets received by the network device are stored in a queue; The node of the scheduler that will select the queue for the transmission of the packet to the port of the network device is stored in the first memory; and The scheduler is updated by the accelerator circuit in response to a command from the controller by continuously loading fields of a data structure with data from each of the plurality of nodes in a traversal operation in multiple iterations, updating the fields of the data structure during at least one of the traversal operations in the multiple iterations, and storing the updated data in at least one of the plurality of nodes in the first memory during the traversal operation.

16. The method of claim 15, further comprising: The controller receives a notification from the port regarding the packets to be transmitted. The controller sends the command to the accelerator circuit in response to the notification; The accelerator circuit identifies the first queue among the queues selected by the scheduler; The accelerator circuit causes the packet to be dequeued from the first queue; The accelerator circuit outputs the packets to the controller; and The controller outputs the packet to the port.

17. The method of claim 16, wherein the accelerator circuit performs the update step in response to dequeuing the packet from the first queue.

18. The method of claim 17, further comprising: The accelerator circuit identifies a second queue from the queues selected by the scheduler after the traversal operation.

19. The method of claim 15, further comprising: The timer task is executed by the controller; and The controller sends the command to the accelerator circuit in response to the timer task.

20. The method of claim 15, further comprising: The controller receives notifications from the queue manager that manages the queues. and The controller sends the command to the accelerator circuit in response to the notification.

Images