Hardware architecture for packet cache eviction engine

Abstract

Aspects of the present disclosure relate to a packet cache eviction engine for a reliable transport protocol for a network. A packet cache eviction engine may manage on-chip cache occupancy by evicting lower priority packets to off-chip memory and ensuring that higher priority packets remain on-chip to achieve higher performance and lower latency when processing packets in a network.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is a continuation of U.S. patent application No. 18 / 327,962 filed on June 2, 2023, the disclosure of which is incorporated herein by reference. Background Art

[0003] On-chip cache buffer space and off-chip memory bandwidth are critical resources for network interface cards for networks utilizing reliable transport protocols. Improper use of these resources can result in lower performance and higher latency in processing packets in the network. Lower priority packets should be evicted to off-chip memory, while high priority packets should be stored in on-chip cache. Given the complex nature of networks, managing the eviction and storage of packets in on-chip cache can be difficult. Summary of the invention

[0004] Aspects of the present disclosure relate to a packet cache eviction engine for a reliable transport protocol for a network. The packet cache eviction engine can be modular, programmable, area efficient, and / or efficiently manage on-chip buffer usage. The packet cache eviction engine can enable higher performance and lower latency for networks utilizing a reliable transport protocol.

[0005] One aspect of the present disclosure provides a method for evicting a packet from an on-chip cache to an off-chip memory, the method comprising: determining, by one or more processors, that an occupancy counter of the on-chip cache is above a first threshold; selecting, by the one or more processors, a queue with a lowest priority from a plurality of queues, each of the plurality of queues having a configurable priority; removing, by the one or more processors, a packet from a head of the selected queue to evict the packet from the on-chip cache; and decrementing, by the one or more processors, the occupancy counter based on a length of the removed packet.

[0006] In one example, the method further includes: determining, by one or more processors, that the occupancy counter is below a second threshold; and stopping, by one or more processors, evicting packets from the on-chip cache. In another example, the first threshold and the second threshold are equal. In yet another example, the occupancy counter counts a total packet length of packets stored in the on-chip cache. In yet another example, the method further includes: selecting, by one or more processors, a plurality of queues having a lowest priority; and selecting, by one or more processors, a queue from the plurality of queues on a round-robin basis.

[0007] In yet another example, the method further comprises: installing, by one or more processors, a packet to a queue; and increasing, by one or more processors, an occupancy counter based on a length of the installed packet. In yet another example, the method further comprises: deallocating, by one or more processors, a packet from a queue; and decreasing, by one or more processors, an occupancy counter based on a length of the deallocated packet.

[0008] In yet another example, the method further includes updating, by one or more processors, the priority of the packet by: deleting the packet from the first queue; using a configurable register to determine the updated priority of the packet; mapping the updated priority to a second queue associated with the updated priority; and installing the packet in the second queue. In yet another example, the method further includes updating, by one or more processors, the priorities of the plurality of queues by: reading the priorities from a priority mapping table; using a configurable register to determine the updated priorities; and writing the updated priorities to the priority mapping table.

[0009] In yet another example, the queue is implemented as a doubly linked list.

[0010] Another aspect of the present disclosure provides a system, the system comprising: one or more processors; and one or more storage devices, the one or more storage devices being coupled to the one or more processors and storing instructions, which, when executed by the one or more processors, cause the one or more processors to perform operations for evicting packets from an on-chip cache to an off-chip memory, the operations comprising: determining that an occupancy counter of the on-chip cache is above a first threshold; selecting a queue with a lowest priority from a plurality of queues, each of the plurality of queues having a configurable priority; removing packets from a head of the selected queue to evict the packets from the on-chip cache; and decrementing the occupancy counter based on a length of the removed packet.

[0011] In one example, the operations further include: determining that the occupancy counter is below a second threshold; and stopping evicting packets from the on-chip cache. In another example, the operations further include: selecting a plurality of queues having a lowest priority; and selecting a queue from the plurality of queues based on a round robin.

[0012] In yet another example, the operations further include: installing the packet to the queue; and increasing the occupancy counter based on the length of the installed packet. In yet another example, the operations further include: deallocating the packet from the queue; and decreasing the occupancy counter based on the length of the deallocated packet.

[0013] In yet another example, the operations further include updating the priority of the packet by: deleting the packet from the first queue; using a configurable register to determine the updated priority of the packet; mapping the updated priority to a second queue associated with the updated priority; and installing the packet in the second queue. In yet another example, the operations further include updating the priorities of the plurality of queues by: reading the priorities from a priority mapping table; using a configurable register to determine the updated priority; and writing the updated priority to the priority mapping table.

[0014] Yet another aspect of the present disclosure provides a non-transitory computer-readable medium for storing instructions, which, when executed by one or more processors, cause the one or more processors to perform operations for evicting packets from an on-chip cache to an off-chip memory, the operations comprising: determining that an occupancy counter of the on-chip cache is above a first threshold; selecting a queue with a lowest priority from a plurality of queues, each of the plurality of queues having a configurable priority; removing packets from a head of the selected queue to evict the packets from the on-chip cache; and decrementing the occupancy counter based on a length of the removed packet.

[0015] In one example, the operations further include: determining that the occupancy counter is below a second threshold; and stopping evicting packets from the on-chip cache. In another example, the operations further include: selecting a plurality of queues having a lowest priority; and selecting a queue from the plurality of queues based on a round robin. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 Depicted is a block diagram of an example cache eviction engine for a network utilizing a reliable transport protocol in accordance with aspects of the present disclosure.

[0017] Figure 2 Depicted is a block diagram of an example doubly linked list in accordance with aspects of the present disclosure.

[0018] Figure 3 A table depicting example configurable registers according to aspects of the present disclosure.

[0019] Figure 4 Depicted is a block diagram of an example network environment with network devices in accordance with aspects of the present disclosure.

[0020] Figure 5 Depicted is a block diagram of an example network device for use with a network in accordance with aspects of the present disclosure.

[0021] Figure 6 Depicted is a flow diagram of an example process for evicting packets from an on-chip cache to off-chip memory in accordance with aspects of the present disclosure.

[0022] Figure 7 Depicted is a flow diagram of an example process for installing a packet for an on-chip cache in accordance with aspects of the present disclosure.

[0023] Figure 8 Depicted is a flow diagram of an example process for deallocating packets for an on-chip cache in accordance with aspects of the present disclosure.

[0024] Fig. 9 Depicted is a flow diagram of an example process for updating a priority of a packet in accordance with aspects of the present disclosure. DETAILED DESCRIPTION

[0025] The present technology generally relates to a highly flexible and area-saving packet cache eviction engine for a reliable transport protocol for a network. A reliable transport protocol may refer to a connection-oriented architecture that provides reliable packet delivery over a lossy and unordered network. Implementing a reliable transport protocol may involve storing received packets until the upper layers of the protocol acknowledge the packets. Having an on-chip cache backed by an off-chip memory can achieve higher performance even in the case of network congestion and disconnection. According to one aspect of the present disclosure, a packet cache eviction engine for a reliable transport protocol for a network is provided. The packet cache eviction engine is configured to manage on-chip cache occupancy by evicting lower priority packets to off-chip memory and ensuring that higher priority packets remain on-chip, so as to achieve higher performance and lower latency when processing packets in the network. The eviction engine manages on-chip cache occupancy based on packet priority using various factors such as resource usage time and / or host occupancy. The eviction engine evicts lower priority packets to off-chip memory and ensures that higher priority packets remain on-chip, thereby achieving higher performance and lower latency when processing packets.

[0026] An on-chip cache (such as a receive (RX) packet cache) can buffer packets received via a reliable transport protocol in both the ingress and egress directions. The on-chip cache can be an SRAM, and can manage on-chip data memory by writing, reading, evicting, and / or deallocating packets. In order to minimize silicon area while still providing high performance, the on-chip cache can be supported by off-chip memory (e.g., DRAM). Since the size of the on-chip cache is relatively small and optimized to reduce latency and area, the eviction engine can monitor on-chip cache buffer usage and evict packets from the on-chip cache to off-chip memory when buffer usage exceeds a threshold. The eviction engine can determine when to evict packets and which packets to evict based on the relative priority of the packets in the on-chip cache, such as evict packets that are earlier in time or packets that are known to be served at a later time.

[0027] The eviction engine may include one or more interfaces to communicate with the on-chip cache, thereby allowing a modular and programmable eviction engine. The on-chip cache may inform the eviction engine via the installation interface that the packet has been installed in the cache. The eviction engine may use the completion interface to inform the on-chip cache that the eviction engine has updated its queue with the installed packet. When the buffer corresponding to the packet is released in the on-chip cache, the on-chip cache may use the deallocation interface to inform the eviction engine to delete the context of the packet from its queue. The eviction engine may use the completion interface or a separate deallocation completion interface to inform the on-chip cache that the eviction engine has updated its queue to delete the packet. The on-chip cache may use the priority update interface to inform the eviction engine to update the priority of each packet or all queues. The eviction engine may use the completion interface or a separate priority update completion interface to inform the on-chip cache that the eviction engine has updated its queue to update the priority of each packet or all queues. The eviction engine may use the eviction interface to inform the on-chip cache which packets are to be evicted from the on-chip cache.

[0028] The eviction engine may include one or more interfaces to communicate with a configurable number of queues. The eviction engine may use a push interface to tell a queue to push a packet to the tail of the corresponding queue based on the input queue number. The eviction engine may use a pop interface to tell a queue to pop a packet from the head of a queue. The eviction engine may use a delete interface to tell a queue to delete a packet from a queue.

[0029] The queues may be distributed to hosts connected to a network interface card (NIC) containing an on-chip cache. The hosts may correspond to processing devices (e.g., CPUs) that share the NIC. Distributing the queues to different hosts maintains isolation between packets belonging to different hosts, thereby allowing packets belonging to any particular host to be prioritized or de-prioritized. The queues may be implemented as a doubly linked list, with a next pointer, a previous pointer, and packet information stored in an on-chip memory separate from the on-chip cache to achieve area and power efficiency.

[0030] A fixed number of queues may be generated for each host, where each queue may represent a priority. Different priority levels may be assigned to queues in a host based on host and / or packet information (such as connection type and / or packet type). Packets may be installed into different queues based on host and / or packet information.

[0031] The queue allows determination of which packet in the queue is the oldest. The head of the queue may correspond to the oldest packet, and therefore, the eviction candidate within the queue that is the oldest packet corresponds to the least recently used packet. If a packet has been recently accessed, the eviction engine may move the packet from its position in the queue to the tail of the same queue or any other queue, thereby changing its priority. Since each queue is associated with a priority, the queue also allows determination of which queue has the lowest priority and is therefore a candidate to be selected for packet eviction.

[0032] The eviction engine may also include a configurable number of priorities and priority levels programmed via configurable registers. The configurable registers may include eviction registers and priority level registers to configure the eviction engine to implement a specific eviction scheme, providing flexibility in how to prioritize and / or evict packets.

[0033] A counter can track the availability of on-chip cache, such as the total number of bytes occupied in the on-chip cache. Once the occupancy reaches a configurable threshold, a strict priority arbiter can select a queue to evict packets, starting with the lowest priority queue. If multiple queues have the same lowest priority, a round-robin arbiter or a random selection arbiter can arbitrate among those queues. Packets can be popped from the selected queue and evicted from the on-chip cache to off-chip memory.

[0034] Each packet written to the on-chip cache can trigger an installation event to the eviction engine on the installation interface. Each installed packet can include a packet length (e.g., in bytes) and a unique index. As an example, the packet length can correspond to the size of the payload of the packet, the size of the payload of the packet plus the header, or the total size of the packet. The packet can be of variable length or uniform length. The packet can also include a portion of the payload in one packet and a second portion of the payload in another packet. The eviction engine can install the packet at a unique index in the queue. The packet length can be added to a counter that tracks cache occupancy. After installation, the eviction engine can generate a completion event to the on-chip cache.

[0035] Each packet that is deallocated from the on-chip cache may trigger a deallocation event on the deallocation interface to unload the packet at a unique index in the queue. The queue may correspond to a list of connected packets in both the forward and reverse directions, enabling the deletion of a packet at any position and then splicing the list using the back pointer and the forward pointer. This maintains the queue order. The deallocated packet length may be decremented from a counter. Once a packet is deleted, it is no longer a candidate for eviction.

[0036] The priority update interface can modify the priority of individual packets or all group queues. Individual packet priorities can be updated when the packet is at the head of the row and therefore needs to be located in the on-chip cache to minimize latency of the entire packet processing pipeline, or when the packet is in a receiver not ready (RNR) state that is known to have high inherent latency and is therefore preferably included in off-chip memory. The priority update interface can provide a unique index of the packet whose priority needs to be modified. The eviction engine can delete the packet from the queue and then use the adjusted priority level register to calculate the updated priority of the packet to increase or decrease the priority level of the corresponding host and / or packet by a configurable amount, map the priority to the new queue using the priority mapping table, and install the packet in the new queue.

[0037] To modify the priority of all queues associated with a host, the eviction engine can read the priority of each queue of the host from the priority map, update the priority of each queue using the configurable register, and write the updated priority of the queue back to the priority map. Using configurable registers to change the priority of individual packets or all queues provides an efficient and flexible mechanism to update the priority while the on-chip cache is still processing packets.

[0038] The eviction engine includes a counter to track the total packet length of all packets in all queues. When the total packet length is above a configurable threshold, the eviction engine can initiate an eviction scan. The eviction engine can use an arbitrator to select a group of queues with the same lowest priority. The eviction engine can then use the arbitrator to select each queue in a round-robin manner. The arbitrator can send an eject request to each selected queue. The queue can eject packets from the head of the queue. The eviction engine can present the ejected packets to the on-chip cache as eviction candidates. For each evicted packet, the occupancy counter decrements the length of the packet. The arbitrator continues to select queues until the counter reaches below another configurable threshold. The configurable threshold used to stop the eviction scan can be the same or different from the configurable threshold that starts the eviction scan.

[0039] Eviction can also be triggered via user request through the eviction interface. The on-chip cache can request eviction of a specific packet through the priority update interface. The eviction engine can immediately pop the packet from the queue and remove the evicted packet from the on-chip cache.

[0040] Figure 1A block diagram of an example cache eviction engine 100 for a network utilizing a reliable transport protocol is depicted. The cache eviction engine 100 can be configured to manage on-chip cache occupancy using configurable packet priorities. The cache eviction engine 100 can evict lower priority packets to off-chip memory while maintaining higher priority packets in the on-chip cache. Based on processing packets through the cache eviction engine 100, a network utilizing a reliable transport protocol can have higher performance and lower latency. The cache eviction engine 100 can include multiple queues 102, a priority mapping table 104, and an arbiter 106. The cache eviction engine 100 can use control logic to manage packet flows, such as eviction, installation, and / or maintenance of packets in on-chip memory.

[0041] The queues 102 may correspond to a configurable number of queues in an on-chip cache containing packets and / or packet information. The cache eviction engine 100 may include any number of queues. The queues 102 may also each have a configurable priority. The queues 102 may be distributed to different hosts of the on-chip cache to maintain isolation between packets belonging to different hosts. Maintaining isolation allows for the prioritization or de-prioritization of packets for any host of the on-chip cache. Since each queue is associated with a priority, the queues 102 also allow for determining which queue is a candidate for packet eviction based on which queue has the lowest priority.

[0042] Each host may include one or more queues 102, each of which may represent a priority. The queues 102 of each host may have different priority levels based on information associated with the host and / or the packet. The information associated with the host and / or the packet may include a connection type, such as an ordered or unordered connection, and / or a packet type, such as a packet that is immediately served, a packet that is delayed in scheduling, or a packet with different packet processing rates. Based on the information associated with the host and / or the packet, the cache eviction engine 100 may install the packet into different queues 102.

[0043] For example, each host may include a set of 4 queues, and each packet may contain information about which host the packet belongs to, so that the packet is enqueued to one of the 4 queues belonging to its host. Each packet may additionally have an indication that specifies whether the packet belongs to an ordered connection or an unordered connection. A configurable register (which may be included in the priority mapping table 104) may specify whether a packet belonging to an ordered connection has a higher priority. Based on this configurable register, ordered connection packets may be enqueued to one queue, while unordered connection packets may be enqueued to another queue.

[0044] As one example, queue 102 may be implemented as a doubly linked list. Figure 2 A block diagram of an example doubly linked list 200 is depicted. The doubly linked list may include a backward pointer 202, a forward pointer 204, and grouping information 206, which may be stored in an on-chip memory such as a RAM. The on-chip memory for storing the backward pointer 202, the forward pointer 204, and the grouping information 206 may be separated from the on-chip cache to achieve area and power efficiency. The cache eviction engine 100 may use a push interface to configure the doubly linked list 200 to push packets into the tail of a queue. The cache eviction engine 100 may use a pop-up interface to configure the doubly linked list 200 to pop up packets from the head of the queue. Popping a packet may correspond to ejecting a packet from the on-chip cache to an off-chip memory. The cache eviction engine 100 may use a delete interface to configure the doubly linked list 200 to remove a packet from any position in the queue. Deleting a packet may correspond to completely removing a packet, rather than ejecting it to an off-chip memory, because the packet is no longer needed. The cache eviction engine 100 may configure the doubly linked list 200 using an update interface to move a packet from one location in a queue to another location in the same queue or to any location in another queue.

[0045] The queue 102 may allow the cache eviction engine 100 to determine the age of packets in the queue. The age of a packet may correspond to its most recent use or access. For example, the oldest packet in a queue may correspond to the least recently used or accessed packet in the queue. The oldest packet in a queue may be located at the head of the queue and may be a candidate for eviction. If a packet has been recently accessed or used, the cache eviction engine 100 may move the packet from its position in the queue to the end of the same queue or to the end of another queue.

[0046] Return to reference Figure 1 , the priority mapping table 104 is configured to maintain a mapping between the queues 102 and their corresponding priority levels. The cache priority mapping table 104 may include any number of queues and corresponding priorities. The priority mapping table 104 may include configurable registers to allow flexibility in how the cache eviction engine 100 prioritizes and / or evicts packets. For example, the client computing device may configure the registers in the priority mapping table 104 to implement a specific eviction scheme to include a specific number of priority levels for the corresponding queues 102.

[0047] Figure 3A table of example configurable registers 300 is depicted. The configurable registers may include an eviction register 302 and a priority level register 304. The eviction register 302 may include a field for enabling eviction, such as evict_en. The eviction register 302 may also include a field for whether the number of packets in the cache is above or equal to a start eviction threshold, such as evict_thld_hi, and / or a field for whether the number of packets in the cache is below or equal to a stop eviction threshold, such as evict_thld_lo. When the start eviction threshold is met, the eviction engine 100 may begin to evict packets from the on-chip cache and may continue to evict packets until the number of packets drops below the stop eviction threshold. The start eviction threshold and the stop eviction threshold may be equal values ​​or different values.

[0048] The priority level registers 304 may include an initial priority level register (such as pri_level_init_cfg) and / or an adjusted priority level register (such as pri_level_adj_cfg). The initial priority level register may program the initial priority level of each queue. The initial priority level may include a field for the initial value of the lowest priority level queue for the host, such as host_level_init_cfg. <n>_low, and / or a field for a fixed increment or gap in the host's subsequent priority level queue, such as host_ <n>For example, assuming host_0_low=2 and host_0_gap=3, the initial priority assignments for host 0's queues will be Q0=2, Q1=5, Q2=8, Q3=11. This provides flexibility in the assignment of priority levels without the need for additional configurable registers.

[0049] The adjusted priority level register may program the priority adjustment based on cache occupancy. The cache occupancy may include the total number of packets enqueued and dequeued or the total number of data bytes enqueued and dequeued. The adjusted priority level register may include fields for increasing or decreasing the priority level of the corresponding host and / or packet by a configurable amount, such as case_ <n>_i nc and case_ <n>_adj_value. For example, if case_ <n>_i nc=1, the cache eviction engine 100 may increase the priority level of the host and / or packet by case_ <n>_adj_value amount, and if case_ <n>_i nc=0, then the cache eviction engine 100 can reduce the priority level by case_ <n>_adj_val ue.

[0050] Return to reference Figure 1 , the arbiter 106 is configured to determine which queues should evict packets when the cache occupancy reaches a threshold (such as a start eviction threshold). The cache eviction engine 100 may include a counter 108 to track the cache occupancy to determine when the cache occupancy reaches the threshold. The arbiter 106 may include a strict priority arbiter 110 and one or more round-robin arbiters 112. The strict priority arbiter 110 may be configured to select one or more queues 102 from which to evict packets. The strict priority arbiter 110 may start with the lowest priority queue and move to an increasing higher priority queue if the cache occupancy is still above the threshold. The strict priority arbiter 110 may use the priority mapping table 104 to determine the priority of the queue 102. If multiple queues have the same priority, such as the same lowest priority, one or more round-robin arbiters 112 may select in a round-robin manner among the multiple queues having the same priority. Alternatively or in addition, if multiple queues have the same priority, such as the same lowest priority, the arbiter may include one or more random selection arbiters (not shown) to randomly select among the multiple queues having the same priority. The arbiter 106 may pop one or more packets from the selected queue to evict them from the on-chip cache to the off-chip memory.

[0051] The cache eviction engine 100 may communicate with the on-chip cache via one or more interfaces, such as an installation interface, a completion interface, a deallocation interface, a priority update interface, and an eviction interface. The cache eviction engine 100 may include an installation interface for the on-chip cache to notify the cache eviction engine 100 that a packet has been added to the cache. The cache eviction engine 100 may include a completion interface for the cache eviction engine 100 to notify the on-chip cache that the queue 102 has been updated with the added packet. The cache eviction engine 100 may include a deallocation interface for the on-chip cache to notify the cache eviction engine 100 to delete the packet from its queue, such as when the buffer corresponding to the packet has been released in the on-chip cache. The cache eviction engine 100 may include a priority update interface for the on-chip cache to notify the cache eviction engine 100 to update the priority of a specific packet or all queues. The priority update interface may request a priority update based on each host or each packet. An update based on each host may change the priority of the host's queue, while a priority update based on each packet may move a packet from one queue to another queue. The cache eviction engine 100 may include an eviction interface to inform the on-chip cache which packets to evict from the on-chip cache.

[0052] Figure 4 A block diagram of an example network environment 400 is depicted with a network device 402. The network environment 400 includes a network 404 of interconnected network nodes 406, including a network device 402, which may be one of the network nodes 406. The network nodes 406 participate in the network 404 as a data source, a data destination or sink, and / or an intermediate node such as a switch, router, and / or gateway that propagates data from a source toward a destination through the network 404. The network 404 includes the network device 402 and links 408 to various other participating network nodes 406.

[0053] Network 404 can promote the interaction between participating devices. Example networks include the Internet, local networks, network structures, or any other local area network or wide area network. Network 404 can be composed of multiple connected subnetworks or autonomous networks. Network 404 can be a local area network (LAN) (such as a company intranet), a metropolitan area network (MAN), a wide area network (WAN), an interconnected network such as the Internet, or a peer-to-peer network. Any type and / or form of data network and / or communication network can be used for network 404. The network can be public, private, or a combination of public and private. In general, network 404 can be used to transmit information between computing devices, such as transmitting information between network node 406 and network device 402.

[0054] The network device 402 may be a host device or server that executes one or more virtual machines 410 on one or more processors of the network device 402. The network device 402 may include a network interface driver 412, an off-chip memory 414, a network interface card 416, a real operating system (OS) 418, a hypervisor 420, and a virtual machine 410. The network device 402 may communicate with other network devices through a network 404. The network device 402 and other network devices may create or participate in one or more virtual networks to facilitate communication between virtual machines 410. One or more virtual networks may overlay the physical network that constitutes the network 404. A data packet encapsulation system may be used to route data packets within one of the virtual networks overlaying the physical network. The network device 402 and the network node 406 may route data packets based on a virtual Internet Protocol address (VIP) and a media access control (MAC) address. In some examples, the network device 402 may host one or more virtual machines 410 that transmit data packets or receive data packets. In other examples, the network device 402 may be an intermediate node of a path between two virtual machines executed on two separate network devices. Thus, network device 402 may act as a source, destination, or intermediate switch in one or more virtual networks.

[0055] The hypervisor 420 may manage the operation of the virtual machine 410 , including starting and terminating the virtual machine 410 , managing the allocation of the off-chip memory 414 to the virtual machine 410 , and live migrating the virtual machine 410 to other network devices.

[0056] The off-chip memory 414 may store data and / or computer executable instructions related to the operation and use of the network interface driver 412. For example, the memory 414 may store lower priority packets evicted from an on-chip cache. As examples, the memory 414 may include high bandwidth memory (HBM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), and / or flash memory.

[0057] The off-chip memory 414 may store computer executable instructions of a transport protocol module 422, such as a transport control protocol (TCP) module or a TCP layer of a network stack, for execution on a processor of the network device 402. Functions occurring within the TCP layer of the network stack may additionally or alternatively be performed in another network protocol module within the transport layer, the network layer, or a combined transport / network layer of the network stack. For example, the functions may be implemented in a user datagram protocol (UDP) module, a reliable datagram protocol (RDP) module, a reliable user datagram protocol (RUDP) module, or a datagram congestion control protocol (DCCP) module. The network layer, the transport layer, or a combined transport layer / network layer may generally be referred to as the packet layer of the network stack.

[0058] The network interface driver 412 may include a network interface driver software module running on the real OS 418. As an example, the network interface driver 412 may be a collection of computer executable instructions stored in the off-chip memory 414 that help facilitate network communications when executed by the processor. As another example, the network interface driver 412 may be implemented as logic in a hardware processor or other integrated circuit, or as a combination of hardware and software logic. The network interface driver 412 may communicate with one of the software virtual machines 410 directly or via the guest OS of the virtual machine 410 and / or the hypervisor 420.

[0059] The network interface driver 412 may be included in the layer of the network stack of the real OS 418 of the network device 402. In one example, the network interface driver 412 may be included in the data link layer of the network stack and may communicate with a software module or application included in the application layer of the network stack. In another example, the network interface driver 412 may be included in the link layer of the network stack and may communicate with a TCP / IP module included in the Internet / transport layer of the network stack. In other examples, the function occurring in the layer of the network stack may be configured to receive packets from another network or transport layer protocol module (such as a UDP module, an RDP module, an RUDP module, or a DCCP module) in addition or alternatively. Although not shown, the network interface driver 412 may be included as a part of the network interface card 416.

[0060] The network interface card 416 may be configured to transmit data packets via a network protocol, such as a layer 2 protocol. The network interface card 416 may include a packet forwarder 424, a cache eviction engine 426, and an on-chip cache 428. The cache eviction engine 426 may correspond to Figure 1 A cache eviction engine 100 is depicted.

[0061] The on-chip memory 428 may store data and / or computer executable instructions related to the operation and use of the network interface card 416. For example, the cache 428 may store a subset of higher priority packets from the off-chip memory 414. These packets may be taken out of or evicted from the off-chip memory 414 to the memory 414. The on-chip cache 428 may buffer packets received by the transport protocol module 422 in both the ingress and egress directions. The on-chip cache 428 may manage the on-chip memory by writing, reading, evicting, and / or deallocating packets. As an example, the on-chip cache 428 may include HBM, DRAM, SRAM, and / or flash memory.

[0062] The packet forwarder 424 can be implemented in an application specific integrated circuit (ASIC) or as software executed on a microprocessor. The packet forwarder 424 can receive packets for transmission from the virtual machine 410 and route them to one or more destination virtual machines executed on the remote network device. The packet forwarder 424 can also perform a function that allows the network interface card 416 to act as a destination for packets received through the link 408. For example, the packet forwarder 424 can receive data packets through one of the links 408, decapsulate the data packets, and forward the data packets to one or more destination virtual machines 410 executed on the network device 402. The packet forwarder 424 can transmit packets from the on-chip cache 428 and receive packets to be stored in the on-chip cache 428.

[0063] The cache eviction engine 426 can be implemented in an AS IC or as software executed on a microprocessor. The cache eviction engine 426 can manage the installation, deallocation, priority update and / or eviction of packets from the on-chip cache 428. Since the size of the on-chip cache 428 can be smaller than the off-chip memory 414 to optimize and reduce latency and area, the eviction engine 426 can monitor the occupancy of the on-chip cache 428 and evict the packet from the on-chip cache 428 to the off-chip memory 414 when the occupancy exceeds a threshold. The eviction engine 426 can determine when to evict the packet and which packets to evict based on the relative priority of the packets in the on-chip cache 428, such as evict packets that are earlier in time or packets that are known to be served at a later time.

[0064] Figure 5 A block diagram of an example network device 500 of a network is depicted. The network device 500 may correspond to Figure 4 The depicted network device 402. The network device 500 may include one or more processors 510 in communication with a memory 520, at least one network interface controller 550 having a network interface port 560 for connecting to a network, input / output (I / O) components 570, and other components 580. The processor 510 may communicate or connect to the memory 520, the network interface controller 550, the I / O components 570, and / or other components 580 via a bus 540. The processor 510 may incorporate or connect to a cache memory 530. In some examples, instructions are read from the memory 520 into the cache memory 530 and executed by the processor 510 from the cache memory 530.

[0065] Processor 510 may be any logic circuitry that executes instructions taken from memory 520 or cache 530. Processor 510 may be a microprocessor unit or a dedicated processor. Network device 500 may be based on any processor or processor group that can operate as described herein. Processor 510 may be a single-core processor or a multi-core processor. Processor 510 may be a plurality of different processors. Processor 510 may be implemented as circuitry on one or more chips.

[0066] Memory 520 may be any device suitable for storing computer-readable data. Memory 520 may be a device with fixed storage or a device for reading removable storage media. Examples include all forms of nonvolatile memory, media, and memory devices, semiconductor memory devices (e.g., erasable programmable read-only memory (EPROM), electrically EPROM (EEPROM), SDRAM, and flash memory devices), magnetic disks, magneto-optical disks, and optical disks (e.g., CD ROM, DVD-ROM, or The network device 500 may have any number of memory devices 520 .

[0067] Cache memory 530 may be a form of computer memory placed in close proximity to processor 510 to achieve fast access times. Cache memory 530 may be part of processor 510 or located on the same chip as the processor. There may be multiple levels of cache 530, for example, a level 2 cache layer and a level 3 cache layer.

[0068] The network interface controller 550 may manage data exchanges via the network interface 560. The network interface controller 550 may handle the physical layer and the data link layer of the open system interconnection (OSI) model for network communications. Some tasks of the network interface controller 550 may be handled by one or more of the processors 510. The network interface controller 550 may be incorporated into the processor 510 (such as circuitry on the same chip).

[0069] The network device 500 may have multiple network interfaces 560 controlled by a single controller 550 or multiple network interface controllers 550. Each network interface 560 may be a connection point for a physical network link (e.g., a cat-5 or cat-6 Ethernet link). The network interface controller 550 may support wireless network connections, and the interface port 560 may be a wireless (e.g., radio) receiver / transmitter. The network interface controller 550 may implement one or more network protocols, such as Ethernet. The network device 500 may exchange data with other network devices via a physical or wireless link through the network interface 560. The network interface 560 may be linked directly to another device or linked to another device via an intermediate device (e.g., a hub, a bridge, a switch, or a router), thereby connecting the network device 500 to a data network (such as the Internet).

[0070] The network device 500 may include one or more input or output (I / O) components 570, or provide an interface for the one or more I / O components. As examples, input devices may include keyboards, microphones, touch screens, sensors, and pointing devices (such as mice or trackballs). As examples, output devices may include video displays, speakers, and printers.

[0071] Other components 580 may include I / O interfaces, external serial device ports, and any additional coprocessors. For example, network device 500 may include interfaces, such as a universal serial bus (USB) interface, for connecting input devices, output devices, or additional memory devices (e.g., a portable flash drive or external media drive). Network device 500 may include additional coprocessors, such as a math coprocessor, to assist processor 510 in performing high-precision or complex calculations.

[0072] Figure 6 A flow chart of an example process 600 for evicting a packet from an on-chip cache to an off-chip memory is depicted. The example process 600 may be performed in a system of one or more processors at one or more locations, such as Figure 1 The depicted example cache eviction engine 100 is executed on a processor.

[0073] As shown in block 610, the cache eviction engine 100 may determine that a cache occupancy counter of the on-chip cache is above a first threshold. The cache eviction engine 100 may track a total packet length of all packets on all queues, the total packet length representing packets stored in the on-chip cache. As an example, the packet length may correspond to the size of a payload of a packet, the size of a payload plus a header of a packet, or the total size of a packet. When the total packet length is above a first threshold, the cache eviction engine 100 may begin evicting packets. The first threshold may be configurable based on the size of the on-chip cache.

[0074] As shown in block 620, the cache eviction engine 100 may select one or more queues with the lowest priority to evict one or more packets from the on-chip cache. The cache eviction engine 100 may use an arbitrator to select a group of queues with the same lowest priority. As shown in block 630, for multiple queues in the one or more queues with the lowest priority, the cache eviction engine 100 may select queues in the multiple queues based on a round-robin basis. The cache eviction engine 100 may use an arbitrator to select queues in the group of queues with the same lowest priority in a round-robin manner.

[0075] As shown in block 640, the cache eviction engine 100 may remove one or more packets from the head of the queue having the lowest priority and / or selected on a round-robin basis to evict one or more packets. The cache eviction engine 100 may send a pop request to each selected queue to pop the packet from the head of the queue. The cache eviction engine 100 may report the ejected packets to the on-chip cache so that the ejected packets may be evicted.

[0076] The cache eviction engine 100 may decrement the cache occupancy counter based on the evicted packet, as shown in block 650. The cache eviction engine 100 may decrement the cache occupancy counter by the length of the packet for each evicted packet.

[0077] The cache eviction engine 100 may continue to select queues and eject packets until the cache eviction engine 100 may determine that the cache occupancy counter is below a second threshold, as shown in block 660. The second threshold may also be configurable based on the size of the on-chip cache. The first threshold and the second threshold may be equal values, or different values.

[0078] The cache eviction engine 100 may stop evicting packets, as shown in block 670. The cache eviction engine 100 may stop selecting a queue and popping packets to be evicted based on the selected queue.

[0079] The cache eviction engine 100 may also evict packets based on a request received from a client computing device. The request may be to select a queue with the lowest priority, as described above, or may include a specific packet to be evicted. If the request includes a specific packet, the cache eviction engine 100 may immediately remove the packet from its queue and evict the packet from the on-chip cache.

[0080] Figure 7 A flow chart of an example process 700 for installing a group for an on-chip cache is depicted. The example process 700 may be performed in a system of one or more processors at one or more locations, such as Figure 1 The depicted example cache eviction engine 100 is executed on a processor.

[0081] As shown in block 710, the cache eviction engine 100 may install the packet to a queue. The packet may include a packet length (such as in bytes) and a unique index for identifying the packet. The cache eviction engine 100 may determine a priority for the packet and install the packet in a queue based on the determined priority to add the packet to its appropriate priority level queue using the unique index.

[0082] The cache eviction engine 100 may increment a cache occupancy counter based on the installed packet, as shown in block 720. The cache eviction engine 100 may add the packet length to the cache occupancy counter to monitor the on-chip cache.

[0083] As indicated at block 730, the cache eviction engine 100 may generate a completion notification to send to the on-chip cache to inform the on-chip cache that the packet has been installed in its queue and therefore the packet may be added to the on-chip cache.

[0084] Figure 8 A flow chart of an example process 800 for deallocating packets for an on-chip cache is depicted. The example process 800 may be performed in a system of one or more processors at one or more locations, such as Figure 1 The depicted example cache eviction engine 100 is executed on a processor.

[0085] As shown in block 810, the cache eviction engine 100 may deallocate the packet from the queue. The cache eviction engine 100 may unload the packet at the unique index of the packet in the queue. The packet may be connected in both the forward and reverse directions to enable the packet to be deallocated from any position in the queue. The cache eviction engine 100 may splice the queue using the back pointer and the forward pointer to maintain the queue order.

[0086] The cache eviction engine 100 may decrement the cache occupancy counter based on the deallocated packet, as shown in block 820. When the cache eviction engine 100 monitors the on-chip cache, the cache eviction engine 100 may remove the packet length from the cache occupancy counter.

[0087] As indicated at block 830, the cache eviction engine 100 may generate a completion notification to send to the on-chip cache to inform the on-chip cache that the packet has been deallocated from its queue and therefore is no longer a candidate for eviction.

[0088] Fig. 9 A flow chart of an example process 900 for updating the priority of a packet is depicted. The example process 900 may be performed in a system of one or more processors at one or more locations, such as Figure 1 As an example, the packet priority may be updated when the packet is at the head of a line but needs to be in the on-chip cache to minimize latency of the overall packet processing pipeline, or when the packet has a high intrinsic latency and should be included in off-chip memory.

[0089] As indicated at block 910, the cache eviction engine 100 may delete the packet from the first queue based on the received unique index of the packet whose priority is being modified.

[0090] The cache eviction engine 100 may determine an updated priority for the packet, as shown in block 920. The cache eviction engine 100 may calculate the updated priority using the adjusted priority level register to increase or decrease the priority level of the packet by a configurable amount.

[0091] The cache eviction engine 100 may map the updated priority to a second queue associated with the updated priority, as shown in block 930. The cache eviction engine 100 may use the priority mapping table to determine the second queue associated with the updated priority.

[0092] As represented by block 940, the cache eviction engine 100 may install the packet in the second queue.

[0093] The cache eviction engine 100 can also update the priority of all queues associated with the host, rather than the priority of a single packet. The cache eviction engine 100 can read the priority of each queue of the host from the priority mapping table. The cache eviction engine 100 can use configurable registers (such as adjusted priority level registers) to update the priority of each queue. The cache eviction engine 100 can write the updated priority of the queue back to the priority mapping table. Using configurable registers to change the priority of individual packets or all queues provides an efficient and flexible mechanism to update the priority while the on-chip cache is still processing packets.

[0094] Aspects of the present disclosure may be implemented in digital circuits, computer-readable storage media, as one or more computer programs, or a combination of one or more of the foregoing. The computer-readable storage medium may be non-transitory, for example, as one or more instructions executable by a cloud computing platform and stored on a tangible storage device.

[0095] The phrase "configured to" is used in different contexts in relation to a portion of a computer system, hardware, or computer program. When a system is said to be configured to perform one or more operations, it means that the system has appropriate software, firmware, and / or hardware installed on the system that, when operated, causes the system to perform the one or more operations. When a piece of hardware is said to be configured to perform one or more operations, it means that the hardware includes one or more circuits that, when operated, receive inputs and generate outputs based on the inputs and corresponding to the one or more operations. When a computer program is said to be configured to perform one or more operations, it means that the computer program includes one or more program instructions that, when executed by one or more computers, cause the one or more computers to perform the one or more operations.

[0096] Unless otherwise stated, the foregoing alternative examples are not mutually exclusive, but may be implemented in various combinations to achieve unique advantages. Since these and other variations and combinations of the features discussed above may be utilized without departing from the subject matter defined by the claims, the foregoing description of the embodiments should be made in an illustrative manner rather than in a manner that limits the subject matter defined by the claims. In addition, the examples described herein and the provision of clauses expressed as "such as", "including", etc. should not be interpreted as limiting the subject matter of the claims to specific examples; rather, the examples are intended to illustrate only one of many possible embodiments. In addition, the same reference numerals in different figures may identify the same or similar elements.< / n> < / n> < / n> < / n> < / n> < / n> < / n> < / n>

Claims

1. A method for evicting a packet from an on-chip cache to an off-chip memory, characterized in that The method comprises: Determining, by one or more processors, that an occupancy counter of the on-chip cache is above a first threshold; selecting, by the one or more processors, a queue having a lowest priority from a plurality of queues for storing packets in the on-chip cache, each queue in the plurality of queues having a configurable priority; removing, by the one or more processors, a packet from a head of the selected queue to evict the packet from the on-chip cache; and The occupancy counter is decremented by the one or more processors based on a length of the removed packet.

2. The method according to claim 1, characterized in that Also includes: determining, by the one or more processors, that the occupancy counter is below a second threshold; as well as Evicting packets from the on-chip cache is stopped by the one or more processors.

3. The method according to claim 2, characterized in that The first threshold and the second threshold are equal.

4. The method according to claim 1, characterized in that The occupancy counter counts the total packet length of packets stored in the on-chip cache.

5. The method according to claim 1, characterized in that Also includes: selecting, by the one or more processors, a plurality of queues having the lowest priority; as well as A queue among the plurality of queues is selected by the one or more processors on a round robin basis.

6. The method according to claim 1, characterized in that Also includes: installing, by the one or more processors, the packet into a queue; as well as The occupancy counter is incremented by the one or more processors based on a length of the installed packet.

7. The method according to claim 1, characterized in that Also includes: de-allocating, by the one or more processors, the packet from the queue; as well as The occupancy counter is decremented by the one or more processors based on a length of the deallocated packet.

8. The method according to claim 1, characterized in that The method also includes updating, by the one or more processors, a priority of the packet by: deleting the packet from the first queue; using a configurable register to determine an updated priority of the packet; mapping the updated priority to a second queue associated with the updated priority; as well as The packet is installed in the second queue.

9. The method according to claim 1, characterized in that The method also includes updating the priorities of the plurality of queues by the one or more processors in the following manner: Reading the priority from a priority mapping table; Use configurable registers to determine updated priorities; and The updated priority is written into the priority mapping table.

10. The method according to claim 1, characterized in that The queue is implemented as a doubly linked list.

11. A system, characterized in that: include: one or more processors; as well as one or more memory devices coupled to the one or more processors and storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations for evicting packets from an on-chip cache to an off-chip memory, the operations comprising: determining that an occupancy counter of the on-chip cache is above a first threshold; selecting a queue having a lowest priority from a plurality of queues for storing packets in the on-chip cache, each queue in the plurality of queues having a configurable priority; removing a packet from a selected head of the queue to evict the packet from the on-chip cache; and The occupancy counter is decremented based on the length of the packet that was removed.

12. The system according to claim 11, characterized in that The operations also include: determining that the occupancy counter is below a second threshold; and Evicting packets from the on-chip cache is stopped.

13. The system according to claim 11, characterized in that The operations also include: selecting a plurality of queues having the lowest priority; and A queue among the plurality of queues is selected on a round robin basis.

14. The system according to claim 11, characterized in that The operations also include: installing the packet into the queue; and The occupancy counter is incremented based on the length of the packet being installed.

15. The system of claim 11, wherein: The operations also include: deallocating the packet from the queue; and The occupancy counter is decremented based on the length of the packet being deallocated.

16. The system of claim 11, wherein: The operation further includes updating the priority of the group by: deleting the packet from the first queue; using a configurable register to determine an updated priority of the packet; mapping the updated priority to a second queue associated with the updated priority; as well as The packet is installed in the second queue.

17. The system of claim 11, wherein: The operation also includes updating the priorities of the plurality of queues in the following manner: Reading the priority from a priority mapping table; Use configurable registers to determine updated priorities; and The updated priority is written into the priority mapping table.

18. A non-transitory computer-readable medium for storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations for evicting a packet from an on-chip cache to an off-chip memory, characterized in that: The operations include: determining that an occupancy counter of the on-chip cache is above a first threshold; selecting a queue having a lowest priority from a plurality of queues for storing packets in the on-chip cache, each queue in the plurality of queues having a configurable priority; removing a packet from a selected head of the queue to evict the packet from the on-chip cache; and The occupancy counter is decremented based on the length of the packet that was removed.

19. The non-transitory computer readable medium of claim 18, wherein: The operations also include: determining that the occupancy counter is below a second threshold; and Evicting packets from the on-chip cache is stopped.

20. The non-transitory computer readable medium of claim 18, wherein: The operations also include: selecting a plurality of queues having the lowest priority; and A queue among the plurality of queues is selected on a round robin basis.

Images