Hardware architecture for packet cache eviction engine

Abstract

Aspects of the disclosure relate to a packet cache eviction engine for a reliable transport protocol of a network. The packet cache eviction engine can manage on-chip cache occupancy by evicting lower priority packets to off-chip memory and ensuring that higher priority packets remain on-chip to achieve higher performance and lower latency when processing packets in the network.

Description

[0001] Cross-reference to related applications

[0002] This application is a continuation of U.S. Patent Application No. 18 / 327,962, filed June 2, 2023, the disclosure of which is incorporated herein by reference. Background Technology

[0003] On-chip cache buffer space and off-chip memory bandwidth are critical resources for network interface cards (NICs) utilizing reliable transport protocols. Improper use of these resources can lead to poor packet processing performance and high latency in the network. Lower-priority packets should be evicted from off-chip memory, while high-priority packets should be stored in the on-chip cache. Given the complex nature of networks, managing the evicting and storage of packets in the on-chip cache can be challenging. Summary of the Invention

[0004] Various aspects of this disclosure relate to a packet cache eviction engine for a reliable transport protocol in a network. The packet cache eviction engine can be modular, programmable, space-saving, and / or efficiently manage on-chip buffer usage. The packet cache eviction engine can enable networks utilizing reliable transport protocols to achieve higher performance and lower latency.

[0005] One aspect of this disclosure provides a method for evicting packets from an on-chip cache to off-chip memory, the method comprising: determining, by one or more processors, that an occupancy counter of the on-chip cache is higher than a first threshold; selecting, by one or more processors, a queue with the lowest priority from a plurality of queues, each of the plurality of queues having a configurable priority; removing packets from the head of the selected queue by one or more processors to evict the packets from the on-chip cache; and reducing the occupancy counter by one or more processors based on the length of the removed packets.

[0006] In one example, the method further includes: one or more processors determining that the occupancy counter is below a second threshold; and one or more processors stopping the evicting of packets from the on-chip cache. In another example, the first threshold and the second threshold are equal. In yet another example, the occupancy counter counts the total packet length of packets stored in the on-chip cache. In yet another example, the method further includes: one or more processors selecting a plurality of queues with the lowest priority; and one or more processors selecting a queue from the plurality of queues based on a round-robin approach.

[0007] In yet another example, the method further includes installing, by the one or more processors, the packet to a queue, and increasing, by the one or more processors, the occupancy counter based on a length of the installed packet. In yet another example, the method further includes de-allocating, by the one or more processors, the packet from the queue, and decreasing, by the one or more processors, the occupancy counter based on a length of the de-allocated packet.

[0008] In yet another example, the method further includes updating, by the one or more processors, a priority of the packet by: removing the packet from a first queue; determining an updated priority of the packet using a configurable register; mapping the updated priority to a second queue associated with the updated priority; and installing the packet in the second queue. In yet another example, the method further includes updating, by the one or more processors, a priority of a plurality of queues by: reading a priority from a priority mapping table; determining an updated priority using a configurable register; and writing the updated priority to the priority mapping table.

[0009] In yet another example, the queues are implemented as a double linked list.

[0010] Another aspect of the disclosure provides a system comprising: one or more processors; and one or more storage devices coupled to the one or more processors and storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations for evicting a packet from an on-chip cache to an off-chip memory, the operations comprising: determining that an occupancy counter of the on-chip cache is above a first threshold; selecting a queue with a lowest priority from a plurality of queues, each queue of the plurality of queues having a configurable priority; removing the packet from a head of the selected queue to evict the packet from the on-chip cache; and decreasing the occupancy counter based on a length of the removed packet.

[0011] In one example, the operations further include determining that the occupancy counter is below a second threshold, and ceasing to evict the packet from the on-chip cache. In another example, the operations further include selecting a plurality of queues with a lowest priority, and selecting a queue of the plurality of queues based on a round robin.

[0012] In yet another example, the operations further include installing the packet to a queue, and increasing the occupancy counter based on a length of the installed packet. In yet another example, the operations further include de-allocating the packet from the queue, and decreasing the occupancy counter based on a length of the de-allocated packet.

[0013] In yet another example, the operations further include updating a priority of the packet by: removing the packet from the first queue; determining an updated priority of the packet using a configurable register; mapping the updated priority to a second queue associated with the updated priority; and installing the packet in the second queue. In yet another example, the operations further include updating priorities of a plurality of queues by: reading a priority from a priority mapping table; determining an updated priority using a configurable register; and writing the updated priority to the priority mapping table.

[0014] Yet another aspect of the disclosure provides a non-transitory computer-readable medium for storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations for evicting a packet from an on-chip cache to off-chip memory, the operations comprising: determining that an occupancy counter of the on-chip cache is above a first threshold; selecting a queue with a lowest priority from a plurality of queues, each queue of the plurality of queues having a configurable priority; removing a packet from a head of the selected queue to evict the packet from the on-chip cache; and decrementing the occupancy counter based on a length of the removed packet.

[0015] In one example, the operations further include: determining that the occupancy counter is below a second threshold; and stopping evicting packets from the on-chip cache. In another example, the operations further include: selecting a plurality of queues with a lowest priority; and selecting a queue of the plurality of queues based on a round robin. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 A block diagram depicting an example cache eviction engine for a network utilizing a reliable transport protocol is depicted in accordance with aspects of the disclosure.

[0017] Figure 2 A block diagram depicting an example doubly linked list is depicted in accordance with aspects of the disclosure.

[0018] Figure 3 A table depicting an example configurable register is depicted in accordance with aspects of the disclosure.

[0019] Figure 4 A block diagram depicting an example network environment with network devices is depicted in accordance with aspects of the disclosure.

[0020] Figure 5 A block diagram depicting an example network device for a network is depicted in accordance with aspects of the disclosure.

[0021] Figure 6 A flow diagram depicting an example process for evicting a packet from an on-chip cache to off-chip memory is depicted in accordance with aspects of the disclosure.

[0022] Figure 7 A flow diagram depicting an example process for installing a packet for an on-chip cache in accordance with aspects of the present disclosure is illustrated.

[0023] Figure 8 A flow diagram depicting an example process for uninstalling a packet for an on-chip cache in accordance with aspects of the present disclosure is illustrated.

[0024] Figure 9 A flow diagram depicting an example process for updating a priority of a packet in accordance with aspects of the present disclosure is illustrated. DETAILED DESCRIPTION

[0025] The present technology generally relates to a highly flexible and area-efficient packet cache eviction engine for a reliable transport protocol for a network. A reliable transport protocol can refer to a connection-oriented architecture that provides reliable packet delivery over a lossy and out-of-order network. Implementing a reliable transport protocol can involve storing received packets until an upper layer of the protocol acknowledges the packets. Having an on-chip cache supported by off-chip memory can enable higher performance even in the case of network congestion and dropouts. According to an aspect of the present disclosure, a packet cache eviction engine for a reliable transport protocol for a network is provided. The packet cache eviction engine is configured to manage on-chip cache occupancy by evicting lower priority packets to off-chip memory and ensuring that higher priority packets remain on-chip to enable higher performance and lower latency in processing packets in the network. The eviction engine manages on-chip cache occupancy based on packet priority using various factors such as resource usage time and / or host occupancy. The eviction engine evicts lower priority packets to off-chip memory and ensures that higher priority packets remain on-chip, thereby enabling higher performance and lower latency in processing packets.

[0026] An on-chip cache, such as a receive (RX) packet cache, can buffer packets received through a reliable transport protocol in both ingress and egress directions. The on-chip cache can be an SRAM and can manage on-chip data storage by writing, reading, evicting, and / or uninstalling packets. To minimize silicon area while still providing high performance, the on-chip cache can be supported by off-chip memory, such as DRAM. Since the on-chip cache is relatively small and optimized for latency and area reduction, an eviction engine can monitor on-chip cache buffer usage and evict packets from the on-chip cache to off-chip memory when the buffer usage exceeds a threshold. The eviction engine can determine when to evict packets and which packets to evict based on the relative priority of the packets in the on-chip cache, such as evicting earlier-in-time packets or packets known to be serviced at a later time.

[0027] The eviction engine can include one or more interfaces to communicate with the on-chip cache, allowing for a modular and programmable eviction engine. The on-chip cache can inform the eviction engine via a mount interface that a packet has been mounted in the cache. The eviction engine can inform the on-chip cache via a completion interface that the eviction engine has updated its queue with the mounted packet. When a buffer corresponding to a packet is freed in the on-chip cache, the on-chip cache can inform the eviction engine to delete the context of the packet from its queue using a deallocation interface. The eviction engine can inform the on-chip cache that the eviction engine has updated its queue to delete the packet using a completion interface or a separate deallocation completion interface. The on-chip cache can inform the eviction engine to update the priority of individual packets or the entire queue using a priority update interface. The eviction engine can inform the on-chip cache that the eviction engine has updated its queue to update the priority of individual packets or the entire queue using a completion interface or a separate priority update completion interface. The eviction engine can inform the on-chip cache which packets to evict from the on-chip cache using an eviction interface.

[0028] The eviction engine can include one or more interfaces to communicate with a configurable number of queues. The eviction engine can inform the queue to push a packet into the tail of the corresponding queue based on an input queue number using a push interface. The eviction engine can inform the queue to pop a packet from the head of the queue using a pop interface. The eviction engine can inform the queue to delete a packet from the queue using a delete interface.

[0029] The queues can be distributed to hosts connected to a network interface card (NIC) containing the on-chip cache. The hosts can correspond to processing devices (e.g., CPUs) sharing the NIC. Distributing the queues to different hosts maintains isolation between packets belonging to different hosts, allowing for prioritization or deprioritization of packets belonging to any particular host. The queues can be implemented as doubly linked lists, with next pointers, previous pointers, and packet information stored in on-chip memory separate from the on-chip cache, allowing for area and power efficiency.

[0030] A fixed number of queues can be generated for each host, where each queue can represent a priority level. Different priority levels can be assigned to the queues in a host based on the host and / or packet information, such as connection type and / or packet type. Packets can be mounted into different queues based on the host and / or packet information.

[0031] Queues allow for determining which packet in the queue is the oldest. The head of the queue can correspond to the oldest packet, and thus, the eviction candidate within that queue as the oldest packet corresponds to the least recently used packet. If a packet is recently accessed, the eviction engine can move that packet from its position in the queue to the tail of that same queue or any other queue, thereby changing its priority. Since each queue is associated with a priority, queues also allow for determining which queue has the lowest priority and thus is the candidate selected for packet eviction.

[0032] The eviction engine can also include a configurable number of priorities and priority levels programmed via configurable registers. The configurable registers can include eviction registers and priority level registers to configure the eviction engine to implement a particular eviction scheme, thereby providing flexibility in how packets are prioritized and / or evicted.

[0033] The counter can track the availability of on-chip cache, such as the total number of bytes in the on-chip cache that are occupied. Once the occupancy rate reaches a configurable threshold, the strict priority arbiter can select a queue to evict a packet from, starting with the lowest priority queue. If multiple queues have the same lowest priority, a round-robin arbiter or a random selection arbiter can arbitrate among those queues. A packet can be popped from the selected queue, which will be evicted from the on-chip cache to off-chip memory.

[0034] Each packet written to the on-chip cache can trigger an install event on an install interface to the eviction engine. Each installed packet can include a packet length (e.g., in bytes) and a unique index. As an example, the packet length can correspond to the size of the payload of the packet, the size of the payload of the packet plus the header, or the total size of the packet. Packets can be variable length or uniform length. Packets can also include a portion of the payload in one packet and a second portion of the payload in another packet. The eviction engine can install the packet at the unique index in the queue. The packet length can be added to the counter that tracks cache occupancy. After installation, the eviction engine can generate a completion event to the on-chip cache.

[0035] Each packet deallocated from the on-chip cache can trigger a deallocation event on a deallocation interface to uninstall the packet at the unique index in the queue. The queue can correspond to a list of packets connected in both forward and reverse directions, enabling packets to be deleted at any position and then spliced using backward and forward pointers. This maintains the order of the queue. The length of the deallocated packet can be subtracted from the counter. Once the packet is deleted, the packet is no longer an eviction candidate.

[0036] The priority update interface can modify the priority of individual packets or of all packets in a queue. The individual packet priority can be updated when the packet is at the head of the line and thus needs to be in the on-chip cache to minimize the latency of the entire packet processing pipeline, or when the packet is in a receiver not ready (RNR) that is known to have a high inherent latency and thus is preferred to be included in off-chip memory. The priority update interface can provide a unique index of the packet that needs to have its priority modified. The eviction engine can delete the packet from the queue, then use the adjusted priority level register to calculate the updated priority of the packet to increase or decrease the priority level of the corresponding host and / or packet by a configurable amount, use the priority mapping table to map the priority to a new queue, and install the packet in the new queue.

[0037] To modify the priority of all queues associated with a host, the eviction engine can read the priority of each queue of the host from the priority mapping table, use the configurable register to update the priority of each queue, and write the updated priority of the queue back into the priority mapping table. Using the configurable register to change the priority of individual packets or of all queues provides an efficient and flexible mechanism to update the priority while the on-chip cache is still processing packets.

[0038] The eviction engine includes a counter to track the total packet length of all packets in all queues. When the total packet length is above a configurable threshold, the eviction engine can initiate an eviction scan. The eviction engine can use an arbiter to select a group of queues that have the same lowest priority. The eviction engine can then use the arbiter to select individual queues in a round-robin fashion. The arbiter can send a pop request to each selected queue. The queue can pop a packet from the head of the queue. The eviction engine can present the popped packet to the on-chip cache as an eviction candidate. For each evicted packet, the occupancy counter is decremented by the length of the packet. The arbiter continues to select queues until the counter reaches below another configurable threshold. The configurable threshold to stop the eviction scan can be the same or different from the configurable threshold to start the eviction scan.

[0039] Evictions can also be triggered by user requests via the eviction interface. The on-chip cache can request eviction of a particular packet through the priority update interface. The eviction engine can immediately pop the packet from the queue and remove the evicted packet from the on-chip cache.

[0040] Figure 1A block diagram of an example cache eviction engine 100 for a network utilizing reliable transport protocols is depicted. The cache eviction engine 100 can be configured to manage on-chip cache occupancy using configurable packet priorities. The cache eviction engine 100 can evict lower priority packets to off-chip memory while maintaining higher priority packets in on-chip cache. Based on processing packets through the cache eviction engine 100, a network utilizing reliable transport protocols can have higher performance and lower latency. The cache eviction engine 100 can include a plurality of queues 102, a priority mapping table 104, and an arbiter 106. The cache eviction engine 100 can use control logic to manage packet flow, such as eviction, installation, and / or maintenance of packets in on-chip memory.

[0041] The queues 102 can correspond to a configurable number of queues in on-chip cache that contain packets and / or packet information. The cache eviction engine 100 can include any number of queues. The queues 102 can also each have a configurable priority. The queues 102 can be distributed to different hosts of the on-chip cache to maintain isolation between packets belonging to different hosts. Maintaining isolation allows prioritization or deprioritization of packets for any host of the on-chip cache. Since each queue is associated with a priority, the queues 102 also allow determination of which queue is a candidate for packet eviction based on which queue has the lowest priority.

[0042] Each host can include one or more queues 102, where each queue can represent a priority. The queues 102 for each host can have different priority levels based on information associated with the host and / or packets. The information associated with the host and / or packets can include connection types, such as in-order or out-of-order connections, and / or packet types, such as packets to be serviced immediately, packets that are delay scheduled, or packets that have different packet processing rates. Based on the information associated with the host and / or packets, the cache eviction engine 100 can install packets into different queues 102.

[0043] For example, each host can include a set of 4 queues, and each packet can contain information of which host the packet belongs to, such that the packet is enqueued into one of the 4 queues that belongs to its host. Each packet can additionally have an indication that specifies whether the packet belongs to an in-order connection or an out-of-order connection. A configurable register (which can be included within the priority mapping table 104) can specify whether packets that belong to in-order connections have higher priority. Based on this configurable register, in-order connection packets can be enqueued into one queue, while out-of-order connection packets can be enqueued into another queue.

[0044] As one example, the queues 102 can be implemented as doubly linked lists.Figure 2 A block diagram depicting an example doubly-linked list 200 is depicted. The doubly-linked list can include a backward pointer 202, a forward pointer 204, and packet information 206, which can be stored in on-chip memory such as RAM. The on-chip memory used to store the backward pointer 202, the forward pointer 204, and the packet information 206 can be separate from the on-chip cache to enable area and power efficiency. The cache eviction engine 100 can use a push interface to configure the doubly-linked list 200 to push a packet into the tail of a queue. The cache eviction engine 100 can use a pop interface to configure the doubly-linked list 200 to pop a packet from the head of a queue. Popping a packet can correspond to evicting the packet from the on-chip cache to off-chip memory. The cache eviction engine 100 can use a delete interface to configure the doubly-linked list 200 to remove a packet from any location in a queue. Deleting a packet can correspond to completely removing the packet, rather than evicting to off-chip memory, because the packet is no longer needed. The cache eviction engine 100 can use an update interface to configure the doubly-linked list 200 to move a packet from a location in one queue to another location in the same queue or any location in another queue.

[0045] The queues 102 can allow the cache eviction engine 100 to determine the staleness of packets in the queues. The staleness of a packet can correspond to its most recent use or access. For example, the oldest packet in a queue can correspond to the least recently used or accessed packet within that queue. The oldest packet in a queue can be located at the head of the queue as a candidate for eviction. If a packet is recently accessed or used, the cache eviction engine 100 can move the packet from its location in the queue to the tail of the same queue or the tail of another queue.

[0046] Referring back to Figure 1 The priority mapping table 104 is configured to maintain a mapping between the queues 102 and their respective priority levels. The cache priority mapping table 104 can include any number of queues and corresponding priority levels. The priority mapping table 104 can include configurable registers to allow flexibility in how the cache eviction engine 100 prioritizes and / or evicts packets. For example, a client computing device can configure registers in the priority mapping table 104 to implement a particular eviction scheme to include a particular number of priority levels for respective queues 102.

[0047] Figure 3A table depicting an example configurable register 300 is illustrated. The configurable register can include an eviction register 302 and a priority level register 304. The eviction register 302 can include a field for enabling eviction, such as evict en. The eviction register 302 can also include a field for whether the number of packets in the cache is above or equal to a start eviction threshold, such as evict th lid hi, and / or a field for whether the number of packets in the cache is below or equal to a stop eviction threshold, such as evict th lid lo. When the start eviction threshold is met, the eviction engine 100 can start evicting packets from the on-chip cache and can continue to evict packets until the number of packets falls below the stop eviction threshold. The start eviction threshold and the stop eviction threshold can be equal values or different values.

[0048] The priority level register 304 can include an initial priority level register, such as pri level init cfg, and / or an adjusted priority level register, such as pri level adj cfg. The initial priority level register can program the initial priority level for each queue. The initial priority level can include a field for the initial value of the lowest priority level queue for the host, such as host pri level init, and / or a field for the initial value of the highest priority level queue for the host, such as host pri level max. The adjusted priority level register can program the adjusted priority level for each queue. The adjusted priority level can include a field for the adjusted value of the lowest priority level queue for the host, such as host pri level adj, and / or a field for the adjusted value of the highest priority level queue for the host, such as host pri level max adj. <n>low, and / or a field for a fixed increment or gap of a subsequent priority level queue for the host, such as host <n>_gap. For example, assume host_0_low = 2 and host_0_gap = 3, then the initial priority assignment for host 0's queue would be Q0 = 2, Q1 = 5, Q2 = 8, Q3 = 11. This provides flexibility in priority level assignment without the need for additional configurable registers.

[0049] The adjusted priority level register can program priority adjustments based on cache occupancy. The cache occupancy can include a total number of packets enqueued and dequeued or a total number of data bytes enqueued and dequeued. The adjusted priority level register can include a field to increase or decrease the priority level of a corresponding host and / or packet by a configurable amount, such as case <n>inc and case <n>adj_value. For example, if case <n>If inc = 1, then the cache eviction engine 100 can increase the priority level of the host and / or packet by case <n>the amount of _adj_value, whereas if case <n>If inc = 0, then the cache eviction engine 100 can reduce the priority level by case <n>_adj_value.

[0050] Referring back Figure 1 , the arbiter 106 is configured to determine which queues should evict packets when the cache occupancy reaches a threshold, such as a start eviction threshold. The cache eviction engine 100 can include a counter 108 to track the cache occupancy to determine when the cache occupancy reaches a threshold. The arbiter 106 can include a strict priority arbiter 110 and one or more round robin arbiters 112. The strict priority arbiter 110 can be configured to select one or more queues 102 from which to evict packets. The strict priority arbiter 110 can start with the lowest priority queue and move to incrementally higher priority queues if the cache occupancy is still above the threshold. The strict priority arbiter 110 can use the priority mapping table 104 to determine the priority of the queues 102. If multiple queues have the same priority, such as the same lowest priority, the one or more round robin arbiters 112 can select among the multiple queues with the same priority in a round robin fashion. Alternatively or additionally, if multiple queues have the same priority, such as the same lowest priority, the arbiter can include one or more random selection arbiters (not shown) to randomly select among the multiple queues with the same priority. The arbiter 106 can pop one or more packets from the selected queue to evict it from the on-chip cache to off-chip memory.

[0051] The cache eviction engine 100 can communicate with the on-chip cache via one or more interfaces, such as a mount interface, a completion interface, a deallocation interface, a priority update interface, and an eviction interface. The cache eviction engine 100 can include a mount interface for the on-chip cache to notify the cache eviction engine 100 that a packet has been added to the cache. The cache eviction engine 100 can include a completion interface for the cache eviction engine 100 to notify the on-chip cache that a queue 102 has been updated with an added packet. The cache eviction engine 100 can include a deallocation interface for the on-chip cache to notify the cache eviction engine 100 to delete a packet from its queue, such as when a buffer corresponding to the packet has been freed in the on-chip cache. The cache eviction engine 100 can include a priority update interface for the on-chip cache to notify the cache eviction engine 100 to update the priority of a particular packet or all queues. The priority update interface can request a per-host or per-packet priority update. A per-host update can change the priority of a host’s queue, while a per-packet priority update can move a packet from one queue to another. The cache eviction engine 100 can include an eviction interface to notify the on-chip cache which packets to evict from the on-chip cache.

[0052] Figure 4 A block diagram depicting an example network environment 400 with a network device 402 is drawn. The network environment 400 includes a network 404 of interconnected network nodes 406, including the network device 402, which can be one of the network nodes 406. The network nodes 406 participate in the network 404 as data sources, data destinations or data sinks, and / or intermediate nodes such as switches, routers, and / or gateways that propagate data from sources toward destinations through the network 404. The network 404 includes the network device 402 and links 408 to various other participating network nodes 406.

[0053] The network 404 can facilitate interaction between participating devices. Example networks include the Internet, a local network, a network structure, or any other local or wide area network. The network 404 can be composed of multiple connected sub-networks or autonomous networks. The network 404 can be a local area network (LAN), such as a company intranet, a metropolitan area network (MAN), a wide area network (WAN), an inter-network such as the Internet, or a peer-to-peer network. Any type and / or form of data and / or communication network can be used for the network 404. The network can be public, private, or a combination of public and private. In general, the network 404 can be used to communicate information between computing devices, such as between the network nodes 406 and the network device 402.

[0054] The network device 402 can be a host device or server executing one or more virtual machines 410 on one or more processors of the network device 402. The network device 402 can include a network interface driver 412, off-chip memory 414, a network interface card 416, a real operating system (OS) 418, a hypervisor 420, and the virtual machines 410. The network device 402 can communicate with other network devices over the network 404. The network device 402 and other network devices can create or participate in one or more virtual networks to facilitate communication between the virtual machines 410. The one or more virtual networks can overlay the physical network that makes up the network 404. A data packet encapsulation system can be used to route data packets within one of the virtual networks that overlays the physical network. The network device 402 and the network nodes 406 can route data packets according to virtual internet protocol addresses (VIPs) and media access control (MAC) addresses. In some examples, the network device 402 can host one or more virtual machines 410 that transmit or receive data packets. In other examples, the network device 402 can be an intermediate node of a path between two virtual machines executing on two separate network devices. Thus, the network device 402 can act as a source, destination, or intermediate switch in one or more virtual networks.

[0055] Hypervisor 420 can manage the operation of virtual machines 410, including starting and terminating virtual machines 410, managing the allocation of off-chip memory 414 to virtual machines 410, and live migrating virtual machines 410 to other network devices.

[0056] Off-chip memory 414 can store data and / or computer-executable instructions related to the operation and use of network interface driver 412. For example, memory 414 can store lower-priority packets evicted from on-chip cache. By way of example, memory 414 can include high-bandwidth memory (HBM), dynamic random-access memory (DRAM), synchronous dynamic random-access memory (SDRAM), and / or flash memory.

[0057] Off-chip memory 414 can store computer-executable instructions of transport protocol module 422, such as a transport control protocol (TCP) module or a TCP layer of a network stack, to execute on a processor of network device 402. Functionality occurring within the TCP layer of the network stack can additionally or alternatively be performed in another network protocol module within a transport layer, a network layer, or a combined transport / network layer of the network stack. For example, the functionality can be implemented in a user datagram protocol (UDP) module, a reliable datagram protocol (RDP) module, a reliable user datagram protocol (RUDP) module, or a datagram congestion control protocol (DCCP) module. The network layer, the transport layer, or the combined transport layer / network layer can generally be referred to as a packet layer of the network stack.

[0058] Network interface driver 412 can include a network interface driver software module running on real OS 418. As one example, network interface driver 412 can be a set of computer-executable instructions stored in off-chip memory 414 that, when executed by a processor, help facilitate network communications. As another example, network interface driver 412 can be implemented as logic in a hardware processor or other integrated circuit, or as a combination of hardware and software logic. Network interface driver 412 can communicate with one of software virtual machines 410 directly or via hypervisor 420 and / or guest OS of hypervisor 420.

[0059] The network interface driver 412 can be included within a layer of the network stack of the real OS 418 of the network device 402. In one example, the network interface driver 412 can be included within the data link layer of the network stack and can communicate with software modules or applications included in the application layer of the network stack. In another example, the network interface driver 412 can be included within the link layer of the network stack and can communicate with TCP / IP modules included in the internet / transport layer of the network stack. In other examples, functionality occurring within a layer of the network stack can additionally or alternatively be configured to receive packets from another network or transport layer protocol module, such as a UDP module, RDP module, RUDP module, or DCCP module. Although not shown, the network interface driver 412 can be included as part of the network interface card 416.

[0060] The network interface card 416 can be configured to communicate data packets over a network protocol, such as a layer 2 protocol. The network interface card 416 can include a packet forwarder 424, a cache eviction engine 426, and an on-chip cache 428. The cache eviction engine 426 can correspond to the cache eviction engine 100 as Figure 1 depicted.

[0061] The on-chip memory 428 can store data and / or computer-executable instructions related to the operation and use of the network interface card 416. For example, the cache 428 can store a subset of higher priority packets from the off-chip memory 414. These packets can be fetched from or evicted to the off-chip memory 414. The on-chip cache 428 can buffer packets received through the transport protocol module 422 in both ingress and egress directions. The on-chip cache 428 can manage on-chip memory through writing, reading, evicting, and / or deallocating packets. As an example, the on-chip cache 428 can include HBM, DRAM, SRAM, and / or flash memory.

[0062] The packet forwarder 424 can be implemented in an application-specific integrated circuit (ASIC) or as software executing on a microprocessor. The packet forwarder 424 can receive packets for transmission from the virtual machines 410 and route them to one or more destination virtual machines executing on a remote network device. The packet forwarder 424 can also perform functionality that allows the network interface card 416 to act as a destination for packets received over the links 408. For example, the packet forwarder 424 can receive a data packet over one of the links 408, decapsulate the data packet, and forward the data packet to one or more destination virtual machines 410 executing on the network device 402. The packet forwarder 424 can transmit packets from the on-chip cache 428 as well as receive packets for storage in the on-chip cache 428.

[0063] The cache eviction engine 426 can be implemented in the ASIC or as software executing on a microprocessor. The cache eviction engine 426 can manage the installation, deallocation, priority updates, and / or eviction of packets from the on-chip cache 428. Because the size of the on-chip cache 428 can be small compared to the off-chip memory 414 for optimization to reduce latency and area, the eviction engine 426 can monitor the occupancy of the on-chip cache 428 and evict packets from the on-chip cache 428 to the off-chip memory 414 when the occupancy exceeds a threshold. The eviction engine 426 can determine when to evict packets and which packets to evict based on the relative priority of the packets in the on-chip cache 428, such as evicting packets that are earlier in time or packets that are known to be serviced at a later time.

[0064] Figure 5 A block diagram of an example network device 500 of a network is described. The network device 500 can correspond to the network device 402 as Figure 4 depicted. The network device 500 can include one or more processors 510 in communication with memory 520, at least one network interface controller 550 with a network interface port 560 for connecting to a network, input / output (I / O) components 570, and other components 580. The processor 510 can be in communication with the memory 520, the network interface controller 550, the I / O components 570, and / or the other components 580 via a bus 540. The processor 510 can incorporate or be connected to cache memory 530. In some examples, instructions are read from the memory 520 into the cache memory 530 and executed by the processor 510 from the cache memory 530.

[0065] The processor 510 can be any logic circuitry that executes instructions fetched from the memory 520 or cache 530. The processor 510 can be a microprocessor unit or a special purpose processor. The network device 500 can be based on any processor or set of processors capable of operating as described herein. The processor 510 can be a single core processor or a multi-core processor. The processor 510 can be multiple different processors. The processor 510 can be implemented as circuitry on one or more chips.

[0066] Memory 520 can be any device suitable for storing computer readable data. Memory 520 can be a device with fixed storage, or a device for reading a removable storage medium. Examples include all forms of non-volatile memory, media and memory devices, semiconductor memory devices (e.g., erasable programmable read-only memory (EPROM), electrically EPROM (EEPROM), SDRAM, and flash memory devices), magnetic disks, magneto-optical disks, and optical disks (e.g., CD ROM, DVD-ROM or DVD-RAM). Network device 500 can have any number of memory devices 520.

[0067] Cache memory 530 can be in the form of computer memory placed in close proximity to processor 510 to enable fast access times. Cache memory 530 can be part of processor 510 or located on the same chip as the processor. There can be multiple levels of cache 530, for example, a level 2 cache layer and a level 3 cache layer.

[0068] Network interface controller 550 can manage the exchange of data via network interface 560. Network interface controller 550 can handle the physical and data link layers of the Open Systems Interconnection (OSI) model for network communications. Some of the tasks of network interface controller 550 can be handled by one or more of processors 510. Network interface controller 550 can be incorporated in processor 510, such as circuitry on the same chip.

[0069] Network device 500 can have multiple network interfaces 560 controlled by a single controller 550 or multiple network interface controllers 550. Each network interface 560 can be a connection point for a physical network link (e.g., cat-5 or cat-6 Ethernet link). Network interface controller 550 can support wireless network connections, and interface port 560 can be a wireless (e.g., radio) receiver / transmitter. Network interface controller 550 can implement one or more network protocols, such as Ethernet. Network device 500 can exchange data with other network devices via physical or wireless links through network interface 560. Network interface 560 can link directly to another device or to another device via an intermediate device (e.g., hub, bridge, switch, or router), thereby connecting network device 500 to a data network, such as the Internet.

[0070] Network device 500 can include or provide an interface for one or more input or output (I / O) components 570. As examples, input devices can include a keyboard, microphone, touchscreen, sensor, and pointing device, such as a mouse or trackball. As examples, output devices can include a video display, speaker, and printer.

[0071] Other components 580 can include I / O interfaces, external serial device ports, and any additional coprocessors. For example, network device 500 can include interfaces, such as a universal serial bus (USB) interface, for connecting input devices, output devices, or additional memory devices (e.g., a portable flash drive or an external media drive). Network device 500 can include additional coprocessors, such as a math coprocessor, to assist processor 510 in performing high precision or complex calculations.

[0072] Figure 6 A flowchart depicting an example process 600 for evicting packets from an on-chip cache to off-chip memory is shown. Example process 600 can be performed on a system of one or more processors in one or more locations, such as a network device 500 as described above. Figure 1 The depicted example cache eviction engine 100.

[0073] As shown in block 610, cache eviction engine 100 can determine that a cache occupancy counter of the on-chip cache is above a first threshold. Cache eviction engine 100 can track a total packet length of all packets on all queues, which represents packets stored in the on-chip cache. As an example, the packet length can correspond to a size of a payload of a packet, a size of a payload of a packet plus a header, or a total size of a packet. When the total packet length is above the first threshold, cache eviction engine 100 can begin evicting packets. The first threshold can be configurable based on a size of the on-chip cache.

[0074] As shown in block 620, cache eviction engine 100 can select one or more queues with a lowest priority to evict one or more packets from the on-chip cache. Cache eviction engine 100 can use an arbiter to select a group of queues with the same lowest priority. As shown in block 630, for multiple queues of the one or more queues with the lowest priority, cache eviction engine 100 can select a queue of the multiple queues based on a round robin. Cache eviction engine 100 can use an arbiter to select each queue of the group of queues with the same lowest priority in a round robin manner.

[0075] As shown in block 640, cache eviction engine 100 can remove one or more packets from a head of a queue selected to evict one or more packets based on the lowest priority and / or the round robin. Cache eviction engine 100 can send a pop request to each selected queue to pop a packet from a head of the queue. Cache eviction engine 100 can report the popped packet to the on-chip cache, such that the popped packet can be evicted.

[0076] As represented by block 650, the cache eviction engine 100 can decrement the cache occupancy counter based on the evicted packet. The cache eviction engine 100 can decrement the cache occupancy counter by the length of each packet that is evicted.

[0077] The cache eviction engine 100 can continue to select queues and pop packets until, as represented by block 660, the cache eviction engine 100 can determine that the cache occupancy counter is below a second threshold. The second threshold can also be configurable based on the size of the on-chip cache. The first threshold and the second threshold can be equal values, or different values.

[0078] As represented by block 670, the cache eviction engine 100 can stop evicting packets. The cache eviction engine 100 can stop selecting queues and popping packets to be evicted based on the selected queues.

[0079] The cache eviction engine 100 can also evict packets based on a request received from a client computing device. The request can be to select a queue with the lowest priority, as described above, or can include a specific packet to evict. If the request includes a specific packet, the cache eviction engine 100 can immediately remove the packet from its queue and evict the packet from the on-chip cache.

[0080] Figure 7 A flow diagram depicting an example process 700 for installing packets for an on-chip cache is depicted. The example process 700 can be performed on a system of one or more processors in one or more locations, such as the example cache eviction engine 100 described above. Figure 1 The example cache eviction engine 100 described above.

[0081] As represented by block 710, the cache eviction engine 100 can install a packet to a queue. The packet can include a packet length, such as in bytes, and a unique index to identify the packet. The cache eviction engine 100 can determine a priority of the packet and install the packet in the queue based on the determined priority to add the packet to its appropriate priority level queue with the unique index.

[0082] As represented by block 720, the cache eviction engine 100 can increment the cache occupancy counter based on the installed packet. The cache eviction engine 100 can add the packet length to the cache occupancy counter to monitor the on-chip cache.

[0083] As represented by block 730, the cache eviction engine 100 can generate a completion notification to send to the on-chip cache to inform the on-chip cache that the packet has been installed in its queue and can thus be added to the on-chip cache.

[0084] Figure 8 A flowchart depicting an example process 800 for deallocating a packet for an on-chip cache. The example process 800 can be performed on a system of one or more processors in one or more locations, such as the example cache eviction engine 100 as Figure 1 depicted.

[0085] As shown in block 810, the cache eviction engine 100 can deallocate the packet from the queue. The cache eviction engine 100 can offload the packet at the unique index of the packet in the queue. The packet can be connected in both forward and reverse directions to enable deallocation of the packet from any location in the queue. The cache eviction engine 100 can use back and forward pointers to splice the queue to maintain the order of queuing.

[0086] As shown in block 820, the cache eviction engine 100 can decrement the cache occupancy counter based on the deallocated packet. As the cache eviction engine 100 monitors the on-chip cache, the cache eviction engine 100 can remove the packet length from the cache occupancy counter.

[0087] As shown in block 830, the cache eviction engine 100 can generate a completion notification to send to the on-chip cache to inform the on-chip cache that the packet has been deallocated from its queue and is therefore no longer a candidate for eviction.

[0088] Figure 9 A flowchart depicting an example process 900 for updating a priority of a packet. The example process 900 can be performed on a system of one or more processors in one or more locations, such as the example cache eviction engine 100 as Figure 1 depicted. As an example, the packet priority can be updated when the packet is at the head of a row but needs to be in the on-chip cache to minimize latency of the entire packet processing pipeline, or when the packet has a high inherent latency and should be included in off-chip memory.

[0089] As shown in block 910, the cache eviction engine 100 can delete a packet from a first queue based on a received unique index of the packet whose priority is being modified.

[0090] As shown in block 920, the cache eviction engine 100 can determine an updated priority of the packet. The cache eviction engine 100 can use an adjusted priority level register to calculate the updated priority to increase or decrease the priority level of the packet by a configurable amount.

[0091] As represented by block 930, the cache eviction engine 100 can map the updated priority to a second queue associated with the updated priority. The cache eviction engine 100 can use the priority mapping table to determine the second queue associated with the updated priority.

[0092] As represented by block 940, the cache eviction engine 100 can install the packet in the second queue.

[0093] The cache eviction engine 100 can also update the priority of all queues associated with the host, rather than the priority of a single packet. The cache eviction engine 100 can read the priority of each queue of the host from the priority mapping table. The cache eviction engine 100 can use a configurable register, such as the adjusted priority level register, to update the priority of each queue. The cache eviction engine 100 can write the updated priority of the queue back into the priority mapping table. Using a configurable register to change the priority of individual packets or all queues provides an efficient and flexible mechanism to update the priority while the on-chip cache is still processing packets.

[0094] Aspects of the disclosure can be implemented in digital electronic circuitry, in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of the same. Aspects of the disclosure can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer-readable storage medium for execution by, or to control the operation of, computer software, firmware, or hardware.

[0095] The phrase "configured to" is used in different contexts with respect to a portion of a computer system, hardware, or computer program. When it is said that a system is configured to perform one or more operations, it means that the system has installed on it appropriate software, firmware, and / or hardware that, when operating, causes the system to perform the one or more operations. When it is said that certain hardware is configured to perform one or more operations, it means that the hardware includes one or more circuits that, when operating, receive inputs and generate outputs in accordance with the inputs and corresponding to the one or more operations. When it is said that a computer program is configured to perform one or more operations, it means that the computer program includes one or more program instructions that, when executed by one or more computers, cause the one or more computers to perform the one or more operations.

[0096] The foregoing alternative examples are not mutually exclusive unless otherwise specified and can be implemented in various combinations to achieve unique advantages. As these and other variations and combinations of features discussed above can be susceptible to variation and / or modification, the foregoing description should not be taken as limiting the scope of the subject matter defined by the claims. Additionally, the examples described herein, as well as the provisions of the articles "a,” "an,” and "the” should not be construed so as to limit the subject matter defined by the claims to the particular examples presented. Rather, the described examples are intended to be illustrative only and in no way limiting of the scope of the subject matter defined by the claims. Additionally, the same reference numerals in different drawings can designate the same or similar elements.< / n> < / n> < / n> < / n> < / n> < / n> < / n> < / n>

Claims

1. A method for evicting a packet from an on-chip cache to an off-chip memory, characterized in that, The method comprises: determining, by one or more processors, that an occupancy counter of the on-chip cache is above a first threshold, wherein the occupancy counter counts a total packet length of packets stored in the on-chip cache; selecting, by the one or more processors, a queue with a lowest priority from a plurality of queues used to store packets in the on-chip cache, each queue of the plurality of queues having a configurable priority; removing, by the one or more processors, a packet from a head of the selected queue to evict the packet from the on-chip cache; and decrementing, by the one or more processors, the occupancy counter based on a length of the removed packet.

2. The method of claim 1, wherein, Further comprising: determining, by the one or more processors, that the occupancy counter is below a second threshold; and stopping, by the one or more processors, evictions of packets from the on-chip cache. wherein the first threshold and the second threshold are equal.

3. The method of claim 2, wherein, Further comprising:

4. The method of claim 1, wherein, selecting, by the one or more processors, a plurality of queues having the lowest priority; and selecting, by the one or more processors, a queue of the plurality of queues based on a round robin. Further comprising: installing, by the one or more processors, a packet to a queue; and 5. The method of claim 1, wherein, incrementing, by the one or more processors, the occupancy counter based on a length of the installed packet. Further comprising: deinstalling, by the one or more processors, a packet from a queue; and decrementing, by the one or more processors, the occupancy counter based on a length of the deinstalled packet.

6. The method of claim 1, wherein, Further comprising updating, by the one or more processors, a priority of a packet by: deleting the packet from a first queue; determining, using a configurable register, an updated priority of the packet; mapping the updated priority to a second queue associated with the updated priority; and 7. The method of claim 1, wherein, installing the packet in the second queue. Further comprising updating, by the one or more processors, priorities of a plurality of queues by: reading the priorities from a priority mapping table; determining, using a configurable register, updated priorities; and writing the updated priorities to the priority mapping table. wherein the queues are implemented as doubly linked lists.

8. The method of claim 1, wherein, Comprising: one or more processors; and one or more storage devices coupled to the one or more processors and storing instructions, which when executed by the one or more processors, cause the one or more processors to perform operations for evicting packets from an on-chip cache to off-chip memory, the operations comprising: determining that an occupancy counter of the on-chip cache is above a first threshold, wherein the occupancy counter counts a total packet length of packets stored in the on-chip cache; 9. The method of claim 1, wherein, selecting a queue with a lowest priority from a plurality of queues used to store packets in the on-chip cache, each queue of the plurality of queues having a configurable priority; 10. A system, characterized by removing a packet from a head of the selected queue to evict the packet from the on-chip cache; and decrementing the occupancy counter based on a length of the removed packet. ​ ​ ​ ​ ​ decrementing the occupancy counter based on a length of the packet that is removed.

11. The system of claim 10, wherein, wherein the operations further comprise: determining that the occupancy counter is below a second threshold; and stopping evictions of packets from the on-chip cache.

12. The system of claim 10, wherein, wherein the operations further comprise: selecting a plurality of queues having the lowest priority; and selecting a queue of the plurality of queues based on a round robin.

13. The system of claim 10, wherein, wherein the operations further comprise: installing a packet into a queue; and incrementing the occupancy counter based on a length of the packet that is installed.

14. The system of claim 10, wherein, wherein the operations further comprise: deinstalling a packet from a queue; and decrementing the occupancy counter based on a length of the packet that is deinstalled.

15. The system of claim 10, wherein, wherein the operations further comprise updating a priority of a packet by: deleting the packet from a first queue; determining an updated priority of the packet using a configurable register; mapping the updated priority to a second queue associated with the updated priority; and installing the packet in the second queue.

16. The system of claim 10, wherein, wherein the operations further comprise updating priorities of a plurality of queues by: reading the priorities from a priority mapping table; determining updated priorities using a configurable register; and writing the updated priorities to the priority mapping table.

17. A non-transitory computer-readable medium for storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations for evicting a packet from an on-chip cache to an off-chip memory, the operations comprising: the operations comprise: determining that an occupancy counter of the on-chip cache is above a first threshold, wherein the occupancy counter counts a total packet length of packets stored in the on-chip cache; selecting a queue having a lowest priority from a plurality of queues for storing packets in the on-chip cache, each queue of the plurality of queues having a configurable priority; removing a packet from a head of the selected queue to evict the packet from the on-chip cache; and decrementing the occupancy counter based on a length of the packet that is removed.

18. The non-transitory computer-readable medium of claim 17, wherein, wherein the operations further comprise: determining that the occupancy counter is below a second threshold; and stopping evictions of packets from the on-chip cache.

19. The non-transitory computer-readable medium of claim 17, wherein, wherein the operations further comprise: selecting a plurality of queues having the lowest priority; and selecting a queue of the plurality of queues based on a round robin. wherein the operations further comprise: installing a packet into a queue; and incrementing the occupancy counter based on a length of the packet that is installed. wherein the operations further comprise: deinstalling a packet from a queue; and decrementing the occupancy counter based on a length of the packet that is deinstalled. wherein the operations further comprise updating a priority of a packet by: deleting the packet from a first queue; determining an updated priority of the packet using a configurable register; mapping the updated priority to a second queue associated with the updated priority; and installing the packet in the second queue. wherein the operations further comprise updating priorities of a plurality of queues by: reading the priorities from a priority mapping table; determining updated priorities using a configurable register; and writing the updated priorities to the priority mapping table.

Images