Fine-grained traffic shaping and offloading for network interface cards

Abstract

The present application discloses fine-grained traffic shaping and diversion for a network interface card, and provides a network interface card (140) with traffic shaping capability and a method for performing network traffic shaping using the network interface card (140). The network interface card (140) and the method can shape traffic originating from one or more applications (150a-c) executed on a host network device. The application (150a-c) can be executed in a virtual machine or a containerized computing environment. The network interface card (140) and the method can execute or include several traffic shaping mechanisms, including, for example but not limited to: a delayed completion mechanism, a time index data structure (130), a packet builder (142), and a memory manager (147).

Description

[0001] Description of the case

[0002] This application is a divisional application of Chinese invention patent application No. 201980026359.7, with a filing date of September 24, 2019.

[0003] CROSS-REFERENCE TO RELATED APPLICATIONS

[0004] This application is a continuation of U.S. patent application No. 16 / 146,373 filed on September 28, 2018, the disclosure of which is incorporated herein by reference. Technical Field

[0005] The present disclosure relates to fine-grained traffic shaping and offloading for network interface cards. Background Art

[0006] Traffic shaping is a technique that uses various mechanisms to regulate network data traffic to shape, rate limit, pace, prioritize, or delay traffic flows that are determined to be less important or less desired than prioritized traffic flows, or to implement distribution of network resources across equally prioritized packet flows. The mechanisms used to shape traffic include: classifiers that match and move packets between different queues based on policies; queue-specific shaping algorithms that delay, drop, or mark packets; and scheduling algorithms that reasonably prioritize packet assignments across different queues. Traffic shaping systems that employ these mechanisms are difficult to scale when considering the requirements of maintaining the desired network performance for a large number of traffic classes, or when deploying traffic shaping systems in a variety of network host architectures. Summary of the invention

[0007] At least one aspect relates to a network interface card. The network interface card includes a network interface card memory, which is configured to store a time index data structure, the time index data structure storing identifiers associated with corresponding communications to be transmitted by the network interface card and a pointer set. Each pointer set corresponds to one of the identifiers stored in the time index data structure. Each pointer in each pointer set points to a location in the network interface card memory or a location in the host memory of a host computing device to which the network interface device is coupled. The network interface card includes scheduling logic. The scheduling logic is configured to receive a notification of a new communication to be transmitted by the network interface card, the new communication originating from a source virtualized computing environment in a plurality of virtualized computing environments implemented on the host computing device. The notification includes header data and a pointer set pointing to data to be included in the communication. The scheduling logic is configured to determine whether to drop a packet or schedule the transmission of a new communication based on the available capacity in the network interface card memory allocated to the source virtualized computing environment. The scheduling logic is configured to store the identifier associated with the communication in the time index data structure at the scheduled transmission time in response to determining that the transmission of the communication is to be scheduled, and store the pointer set associated with the communication in the network interface card memory. The network interface card includes packet generation logic configured to generate a data packet using a set of pointers stored in a network interface card memory associated with a communication after a scheduled transmission time of the communication whose identifier is stored in a time index data structure. The network interface card includes a transceiver configured to transmit the generated data packet. The network interface card includes transmission completion logic configured to generate a transmission completion message to be transmitted to an application executed in a source virtualized computing environment that initiated the communication after completing the transmission of the data packet generated for the communication.

[0008] At least one aspect relates to a method for network traffic shaping. The method includes receiving, at a network interface card of a host computing device, a notification of a new communication to be transmitted by the network interface card, the new communication originating from a source virtualized computing environment in a plurality of virtualized computing environments implemented on the host computing device. The notification includes header data and a set of pointers to data to be included in the communication. The method includes determining whether to drop a packet or schedule the transmission of the new communication based on available capacity in a network interface card memory space allocated to the source virtualized computing environment. The method includes: in response to determining that the transmission of the communication is to be scheduled, storing an identifier associated with the communication in a time index data structure at a scheduled transmission time; and storing a set of pointers associated with the communication in a network interface card memory. The set of pointers corresponds to the identifier stored in the time index data structure, and each pointer in the set of pointers points to a location in the network interface card memory or a location in the host memory of the host computing device. The method includes: after the scheduled transmission time of the communication whose identifier is stored in the time index data structure arrives, using the set of pointers stored in the network interface card memory associated with the communication to generate a data packet. The method includes transmitting the generated data packet. The method includes generating, upon completion of transmission of a data packet generated for a communication, a transmission completion message to be communicated to an application executed in a source virtualized computing environment that initiated the communication.

[0009] At least one aspect relates to a non-transitory computer-readable medium having instructions stored thereon, the instructions being configured to cause one or more processors of a network interface card of a host computing device to perform a method of network traffic shaping. The method includes receiving a notification of a new communication to be transmitted by a network interface card, the new communication originating from a source virtualized computing environment in a plurality of virtualized computing environments implemented on the host computing device. The notification includes header data and a set of pointers pointing to data to be included in the communication. The method includes determining whether to drop a packet or schedule the transmission of the new communication based on available capacity in a network interface card memory space allocated to the source virtualized computing environment. The method includes, in response to determining that the transmission of the communication is to be scheduled, storing an identifier associated with the communication in a time index data structure at a scheduled transmission time, and storing a set of pointers associated with the communication in a network interface card memory. The set of pointers corresponds to the identifier stored in the time index data structure, and each pointer in the set of pointers points to a location in the network interface card memory or a location in the host memory of the host computing device. The method includes generating a data packet using the set of pointers stored in the network interface card memory associated with the communication after the scheduled transmission time of the communication whose identifier is stored in the time index data structure arrives. The method includes generating, upon completion of transmission of a data packet generated for a communication, a transmission completion message to be communicated to an application executed in a source virtualized computing environment that initiated the communication.

[0010] These and other aspects and embodiments are discussed in detail below. The foregoing information and the following detailed description include illustrative examples of the various aspects and embodiments, and provide an overview or framework for understanding the nature and character of the claimed aspects and embodiments. The accompanying drawings provide illustration and further understanding of the various aspects and embodiments, and are incorporated into and constitute a part of this specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The drawings are not intended to be drawn to scale. Like reference numbers and names refer to like elements in the various drawings. For clarity, not every component is labeled in every drawing. In the drawings:

[0012] Figure 1 is a block diagram of a network environment having network devices according to some embodiments;

[0013] Figure 2A is a block diagram of an exemplary virtual machine environment;

[0014] Figure 2B is a block diagram of an exemplary containerized environment;

[0015] Figure 3 is a flow chart illustrating the operation of a network interface card according to some embodiments;

[0016] FIG. 4A to FIG. 4C is a block diagram illustrating the operation of a network interface card according to some embodiments; and

[0017] Figure 5 is a block diagram of an exemplary computing system that may be employed to implement elements of the systems and methods described and illustrated herein, in accordance with an illustrative embodiment. DETAILED DESCRIPTION

[0018] The present disclosure generally relates to systems and methods for fine-grained traffic shaping for network interface cards. Traffic shaping is a technique that utilizes various mechanisms to regulate network data traffic to shape, rate limit, pace, prioritize, or delay traffic flows that are determined to be less important or less desired than prioritized traffic flows, or to implement distribution of network resources across equally prioritized packet flows. The mechanisms used to shape traffic include: a classifier that is used to match and move packets between different queues based on policies; a queue-specific shaping algorithm that is used to delay, drop, or mark packets; and a scheduling algorithm that is used to reasonably prioritize packet assignments across different queues. Traffic shaping systems that employ these mechanisms are difficult to scale when considering the requirements of maintaining the desired network performance for a large number of traffic classes, or when deploying traffic shaping systems in various network host architectures.

[0019] Traffic shaping systems should be designed to achieve efficient memory usage and host processor power consumption while managing higher-level congestion control, such as that used in the Transmission Control Protocol (TCP). Traffic shaping systems may include several different mechanisms and / or techniques for efficiently and reliably controlling the flow of applications from multiple to many different virtual machines and / or instances sharing bandwidth through network interface cards. The various traffic shaping mechanisms and techniques described herein include delayed completion mechanisms, time-indexed data structures, packet builders, and memory managers.

[0020] Typically, traffic shaping mechanisms are already implemented in the OS or network interface driver of a network device. Embodiments of the present disclosure may implement delayed completion mechanisms, time index data structures, packet builders, and memory managers in logic in the network interface card itself. Implementing these mechanisms in the network interface card may increase the throughput of the network interface card while reducing the computational load on the network device. In some embodiments, the traffic shaping mechanism may be implemented in logic in a microprocessor, field programmable gate array (FPGA), or application specific integrated circuit (ASIC) incorporated into the network interface card.

[0021] The delayed completion mechanism can delay, pace or rate limit the packet to avoid bursts or unnecessary transmission delays. Using delayed completion can achieve higher utilization of network resources and host processing resources. The packet delay mechanism can reduce the need for large memory buffers. For example, when a packet is delayed, the feedback mechanism can apply "back pressure", that is, feedback is sent to a sending module (e.g., a device or software component, such as a software application) to cause the sending module to reduce its rate of sending packets. The delayed completion mechanism may prevent the application from sending additional packets for transmission until the application receives a message confirming that the previously forwarded packet has been successfully transmitted. For example, the delayed completion mechanism conveys a packet transmission completion message to a sending module, such as a software application or a guest operating system, which can suppress the network interface card from requesting additional packet transmission until it receives a transmission completion message indicating that the previously requested packet transmission has been completed. As described herein, in some embodiments, the network interface card processes the received packets based on the traffic shaping policy stored on the network interface card to determine the transmission time for each packet. For example, the rate limiting policy may include a rate pacing policy or a target rate limit. Additionally or alternatively, the rate limiting policy may include a specific policy associated with a particular packet class or an overall rate for a particular packet class. Without a packet delay mechanism, the send module may continue to generate packets that may be buffered or discarded, wasting additional memory and host processor power to queue or regenerate packets.

[0022] The network interface card can store the identifier associated with the corresponding packet in the time index data structure at a location associated with the transmission time determined for the corresponding packet. The time index data structure can include a single time-based queue, such as a timing wheel or calendar queue data structure, to receive the identifier associated with the packet from multiple queues or TCP sockets. The packet identifier can be inserted and extracted based on the determined transmission time. In some embodiments, the network interface card can determine that the time indexed in the single time index queue has been reached, and in response, the packet associated with the identifier stored in the time index data structure is transmitted at a location associated with the reached time. For example, the network interface card or the network interface card can determine that the time t0 has been reached, and therefore, the network interface card and / or the network interface driver can cause the packet associated with the identifier of the specified t0 transmission time to be transmitted by the network interface card of the network device. In some embodiments, after the network interface card transmits the packet, the network interface card can convey the transmission completion notification back to the application that initiated the transmitted packet. The time index data structure can be used in combination with a delayed completion mechanism. In some embodiments, the network device is configured to receive packets from multiple applications at the TCP layer of the network device. The received packets originate from an application executed on a host computing device (eg, on one or more virtual machines or containerized execution environments (collectively referred to herein as "virtual computing environments") hosted by the host computing device).

[0023] The packet builder can build a packet for transmission based on the packet identifier in the time index data structure and the associated corresponding pointer set stored in the network interface card memory. Each identifier can be, for example, a pointer to an area in the memory containing the pointer set. Each pointer set includes multiple pointers, each of which points to a memory location in the host memory, which stores a portion of the data that will constitute the packet associated with the identifier. When the time index data structure reaches the time corresponding to the transmission time of the identifier in the time index data structure, the packet builder can use the identifier to retrieve the pointer set from the network interface card memory and use the pointer set to build a packet for transmission. In some embodiments, the pointer set can take the form of a scatter gather list. Each pointer in the pointer set can point to an area in the memory containing the appropriate field, component or payload of the packet. The packet builder can therefore build a packet header and payload based on the pointer set. In some embodiments, the packet builder can use, for example, direct memory access (DMA) to directly access the host memory. By maintaining only a single identifier in the time index data structure, the size of the time index data structure can be minimized. In addition, by storing only the pointer set in the network interface card memory, the network interface card does not have to store the entirety of all queued packets for transmission in its own local memory; therefore, the memory size of the network interface card can be minimized, and the transmission between the host network device and the network card can be optimized. In addition, by using the packet identifier and the pointer set for scheduling and other traffic shaping operations in the network interface card, the network interface card can receive notifications and schedule (or not schedule) packet transmission without having to receive a large amount of packet data. Therefore, the traffic burst (in bytes per second) from the application to the network interface card may be much smaller and is less likely to overwhelm the available bandwidth of the interface between the host device and the network interface card. When the packet is ready to be transmitted, the network interface card only needs to receive most of the data of the packet. In addition, the network interface card never needs to receive most of the data of the packet that is not scheduled for transmission (for example, due to the lack of available capacity in the network interface card memory allocated to the virtual machine that initiated the notification, causing the notification to be discarded). Therefore, the interface bandwidth from the host to the network interface card is reserved for the packets that are actually to be transmitted, and the bandwidth is used essentially at the transmission rate, which is controlled by the network interface card itself rather than by the application, virtual machine, or container that generates the packet. This can be particularly valuable when there is little coordination between group-generating applications, VMs, or containers.

[0024] The network interface card may implement a memory manager that can control the amount of network interface card resources available for each virtual machine or container implemented on the network device. In some cases, each virtual machine may implement many applications (tens or more). Other methods of traffic shaping (such as delayed completion) can control traffic initiated at individual application instances or on a flow-by-flow basis. However, in a cloud computing environment, it is possible for a single virtual machine or container to implement many applications simultaneously. For example, it is possible for a single virtual machine or container to host a hundred or more applications, all of which attempt to transmit packets at the same time. Other traffic shaping mechanisms may use different transmission queues for different traffic classes; however, these mechanisms also lack the ability to distinguish traffic by source. Therefore, for a host device or containerized environment that hosts virtual machines, it may be beneficial to have separate traffic shaping control capabilities for each virtual machine or container. The memory manager can provide separate traffic shaping control for each virtual machine or container by allocating portions of the network interface card memory that can be used for each virtual machine or container. Therefore, the memory manager can ensure that no single virtual machine or container can overload the resources of the network interface card and cause damage to other virtual machines or containers. For example, based on the lack of available capacity in the network interface card memory allocated to the source virtualized computing environment by the memory manager, the scheduling logic of the network interface card may cause the packet to be dropped. However, if there is capacity available for the source virtualized computing environment, the scheduling logic may schedule the transmission of the packet.

[0025] In the above-described embodiment, packet sources such as software applications running on the real OS of a host network device, on the guest OS of a virtual machine, or on the upper layer of a TCP stack in a guest OS managed by a hypervisor do not need to know the traffic shaping strategy or algorithm implemented on the network interface card. Therefore, the cost of implementing a network interface driver and a guest operating system in a virtual machine environment can be reduced. In addition, the packet source does not need to know other configuration parameters, such as packet classification rules and other rate limiting strategies. Therefore, compared to methods in which an application or user configures such detailed algorithms and strategies, traffic shaping can be performed in a more reliable manner.

[0026] Figure 1 1 is a block diagram of an exemplary network environment 100 having a network device 110. In general terms, the illustrated network environment 100 includes a network 700 of interconnected network nodes 750. The network nodes 750 participate in the network 700 as data sources, data destinations (or data sinks), and / or intermediate nodes (such as switches, routers, and / or gateways) that propagate data from a source to a destination through the network 700. The network 700 includes a network device 110 having links 600 to various other participating network nodes 750. See hereinafter for more details. Figure 1, network 700 is a network that facilitates interaction between participant devices. The illustrative example network 700 is the Internet; however, in other embodiments, network 700 may be another network, such as a local area network within a data center, a network structure, or any other local area network or wide area network. Network 700 may consist of multiple connected subnetworks or autonomous networks. Network 700 may be a local area network (LAN) (such as a company intranet), a metropolitan area network (MAN), a wide area network (WAN), an interconnected network such as the Internet, or a peer-to-peer network (e.g., a self-organizing (ad hoc) WiFi peer-to-peer network). Any type and / or form of data network and / or communication network may be used for network 700. It may be a public network, a private network, or a combination of a public network and a private network. In general, network 700 is used to transmit information between computing devices (e.g., network nodes 750), and the network device 110 of the data traffic shaping system facilitates the communication according to its configuration.

[0027] like Figure 1 As shown in , the network device 110 is a host or server that hosts one or more applications 150a to 150c (generally, application 150) that are executed on a real operating system (OS). As discussed further below, in other embodiments, the network device 110 can be a server that hosts a virtual machine or container that is executing application 150. The network device 110 includes a network interface driver 120, a memory 115, a network interface card 140, a real OS 220, and application 150. The network interface card 140 includes a scheduler 141, a packet builder 142, a transceiver 143, a transmission completion logic 144, a memory manager 147, and a memory 146. The memory 146 may include a time index data structure (such as a timing wheel 130), although for clarity, the timing wheel 130 is illustrated as being external to the memory 146. In some embodiments, the network device 110 has a plurality of components such as a time index data structure, such as a timing wheel 130, and a plurality of components such as a timing wheel 130. Figure 5 For example, the memory 115 may have a configuration similar to that of the computing system 1010 shown in FIG. Figure 5 The network interface card 140 may have a configuration similar to that of the memory 1070 shown in FIG. Figure 5 The network interface card 1022 and / or the network interface controller 1020 may be configured similarly to the configuration shown in FIG. Figure 5 The computing system 1010 is described in more detail. Figure 1 In some implementations of the network device 110 shown in FIG. Figure 5 Not all of the elements shown in the computing system 1010 illustrated in FIG. 1 need be present.

[0028] See again Figure 1In some implementations, the network device 110 hosts one or more applications 150 (e.g., applications 150a, 150b, and 150c). One or more of the applications 150a to 150c may be software applications that run on the actual operating system of the network device 110. Figure 2A and Figure 2B As further discussed, in some embodiments, one or more of the software applications 150a to 150c may be software applications running on a guest OS managed by a hypervisor in a virtual machine environment, or may be an upper layer of a protocol stack (e.g., a TCP stack) of a guest OS in a virtual machine environment. Figure 2A ,exist Figure 2A In the example, applications 150a to 150c may be software applications 230 running on real OS 220, software applications 265 running on guest OS 260 of virtual machine 1 managed by hypervisor 250, or upper layers of protocol stack 261 of guest OS 260 of virtual machine 1. Figure 2A The hypervisor 250 and its associated virtual machine environment are described in more detail.

[0029] Look back Figure 1 , the network device 110 includes a memory 115. In some embodiments, the memory 115 may store computer-executable instructions of a transport protocol module 145 (such as a TCP protocol module or a TCP layer of a network stack) to be executed on a processor of the network device 110. In some other embodiments, the memory 115 may store computer-executable instructions of a network interface driver 120. The memory 115 may store data and / or instructions related to the operation and use of the network interface driver 120. The memory 115 may include, for example, a random access memory (RAM), a dynamic random access memory (DRAM), a static random access memory (SRAM), a synchronous dynamic random access memory (SDRAM), a ferroelectric random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), and / or a flash memory.

[0030] The functionality occurring within the TCP layer of the network device described above may additionally or alternatively be implemented in another network protocol module within the transport layer, network layer, or combined transport / network layer of the network protocol stack. For example, the functionality may be implemented in a user datagram protocol (UDP) module, a reliable datagram protocol (RDP) module, a reliable user datagram protocol (RUDP) module, or a datagram congestion control protocol (DCCP) module. As used herein, the network layer, transport layer, or combined transport / network layer will generally be referred to as the packet layer of the network protocol stack.

[0031] The network interface driver 120 may include a network interface driver software module running on a real OS. A network interface driver (such as, network interface driver 120) may be a collection of computer executable instructions stored in memory 115 that, when executed by a processor, help facilitate network communications. In some other embodiments, the network interface driver 120 may be implemented as logic implemented in a hardware processor or other integrated circuit, or as a combination of hardware and software logic. The network interface driver 120 may communicate directly with one of the software applications 150a to 150c (e.g., via the guest OS of the virtual machine (or in some embodiments, through the hypervisor and the guest OS) if operating in a virtual machine environment) or via a container manager of a containerized environment. Figure 2A 265) in the real OS 220 of the network device 110 (if operating on the real OS 220 of the network device 110). In some embodiments, the network interface driver 120 is included in the first layer of the transmission control protocol (TCP) stack of the real OS of the network device 110, and communicates with the software modules or applications included in the upper layer of the TCP stack. In one example, the network interface driver 120 is included in the transport layer of the TCP stack, and communicates with the software modules or applications included in the application layer of the TCP stack. In another example, the network interface driver 120 is included in the link layer of the TCP stack, and communicates with the TCP / IP module included in the Internet / transport layer of the TCP stack. In some embodiments, the functionality is additionally or alternatively configured to receive packets from another network or transport layer protocol module (such as a user datagram protocol (UDP) module, a reliable datagram protocol (RDP) module, a reliable user datagram protocol (RUDP) module, or a datagram congestion control protocol (DCCP) module). In some other embodiments, the network interface driver 120 may be included as part of the network interface card 140.

[0032] The network interface card 140 includes a network interface card memory 146. In some other embodiments, the memory 146 may store computer-executable instructions for the network interface card 140. The memory 146 may store data and / or instructions related to the operation and use of the network interface card 140. The memory 146 may include, for example, a random access memory (RAM), a dynamic random access memory (DRAM), a static random access memory (SRAM), a synchronous dynamic random access memory (SDRAM), a ferroelectric random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), and / or a flash memory. Additionally or alternatively, the memory 146 may store a rate limiting algorithm, a rate limiting policy, or a computer-executable instruction utilized by the scheduler 141 and other modules and logic of the network interface card 140. In some embodiments, the memory 146 may store statistics or metrics associated with flows or categories of packets that have been transmitted by the network device 110 and / or have been scheduled for future transmission. For example, memory 146 may store statistical information or metrics, such as previous and upcoming transmission times and historical transmission rates of packets in each packet class to which rate limiting is to be applied. The statistical data may also include the number of packets currently in timing wheel 130 (discussed further below) associated with each class. In some embodiments, memory 146 stores computer-executable instructions that, when executed by network interface card 140, cause network interface card 140 to implement Figure 3 The processing stages shown in FIG. 1 and FIG. 2 are further described below.

[0033] As mentioned above, the network interface card 140 includes a scheduler 141. A scheduler, such as the scheduler 141, can be a collection of computer executable instructions stored, for example, in the memory 146, which, when executed by the processor, causes the functionality discussed below to be implemented. In some other embodiments, the scheduler 141 can be implemented as logic implemented in a hardware processor or other integrated circuit, or as a combination of hardware and software logic. In some embodiments, the scheduler 141 is utilized to manage the sequence of packet identifiers inserted into and extracted from the timing wheel data structure 130. In some embodiments, before scheduling or making any decisions, the scheduler 141 can: a) copy at least the packet header to the network interface card memory 146; b) prepare a descriptor pointing to the packet header and the packet payload (which may or may not be copied to the network interface card memory 146); and c) implement the scheduling decision and queue the identifier of the descriptor in the time index data structure 130.

[0034] Additionally or alternatively, the scheduler 141 may implement known, existing network scheduling algorithms that can be used for different operating system kernels. In some embodiments, the scheduler 141 may implement a customized, user-defined scheduling algorithm. For example, the scheduler 141 may include a rate limiting policy algorithm that can calculate the timestamp of the received packet. In some embodiments, the scheduler 141 may implement a weighted fair queuing algorithm to ensure that multiple packet flows share bandwidth in proportion to their weights in a minimum-maximum fair allocation scheme. Additionally or alternatively, the scheduler 141 may consolidate timestamps so that larger timestamps represent smaller target transmission rates. In some embodiments, the scheduler 141 may store and / or retrieve a rate limiting scheduling algorithm from the memory 146. Additionally or alternatively, the scheduler 141 may evaluate the packets received by the network interface driver 120 and store the packet identifiers in the timing wheel data structure 130. In some embodiments, the scheduler 141 may evaluate the received packet data to determine the transmission timestamp associated with the received packet. Additionally or alternatively, the scheduler 141 may determine an updated transmission timestamp for a received packet that already has a timestamp applied by an application, virtual machine, or container that originated the packet, and may apply the updated transmission timestamp to the packet identifier. In some embodiments, the scheduler 141 may direct the timing wheel data structure 130 to store the packet identifier with the transmission timestamp at an appropriate time slot in the timing wheel data structure 130. Additionally or alternatively, the scheduler 141 may direct the timing wheel 130 to extract a stored packet identifier, such as a packet identifier including a transmission timestamp, when a transmission time has been reached. FIG. 4A to FIG. 4C The scheduler 141 is described in more detail.

[0035] The network interface card 140 may also include memory allocation logic such as a memory manager 147. The memory manager 147 may enforce limits on the amount of memory space in the memory 146 that is available for each virtual machine or container executed on the network device 110. (See below Figure 2A and Figure 2B 146 ). In some cases, each virtual machine or container may execute many applications (tens or more). Memory manager 147 may allocate portions of memory 146 to each virtual machine or container. Thus, memory manager 147 may ensure that no single virtual machine or container is able to overload memory 146 and cause damage to applications executed on other virtual machines or containers.

[0036] The network interface card 140 has a configurable amount of memory 146 that can be allocated for pointers (i.e., packet descriptors). The configurable amount of memory 146 for descriptors must be balanced with the capacity requirements for routing tables and / or executable code. The configurable amount of memory 146 must then be shared among all producers of packets to be transmitted by the network interface card 140, which may include virtual machines, jobs on the host OS, etc. Sharing among producers may be oversubscribed (based on the assumption that not all producers will use their entire allocation at the same time). The memory manager 147 may have several configurable limits for each producer, such as different limits for packets with different quality of service (QoS) markings. For example, the memory 146 may store up to 100,000 packets, with capacity equivalent to 20,000 packets allocated to each of five producers. For each producer, the memory manager 147 may allow the producer to fill all 20,000 packet capacity with high priority packets, but cap low priority packets at 10,000 per producer. In this example, it is also possible to allocate 20,000 packets of capacity to each of the six producers, resulting in oversubscription of capacity equivalent to 20,000 packets. However, unless the combined producers each maximize their allocation using high priority packets, the producers will not maximize the total allocated memory 146.

[0037] In some embodiments, the memory manager 147 may allocate memory based on one or more of a variety of other factors or calculations. For example, the memory manager 147 may allocate space in the memory 146 based on the historical use of the memory 146 by certain virtual machines or containers. If a virtual machine or container typically uses at most, for example, 10MB of space in the memory 146, the memory manager 147 may allocate 10MB or 10MB plus an additional margin of 1MB, 2MB, 5MB, or 10MB, etc., for the virtual machine or container. In another example, the memory 147 may allocate space in the memory 146 based on a prediction of how much capacity a given virtual machine or container will need. Some virtual machines or containers may use more capacity during different times of the day or week; therefore, the memory manager 147 may employ a dynamic allocation scheme that changes the memory allocation based on predictable changes in usage. In some embodiments, the memory allocation by the memory manager 147 may be user-configurable.

[0038] In exemplary operation, when the network interface card 140 receives a notification of a new communication originating from a source virtualized computing environment (e.g., a virtual machine or container), the scheduler 141 may determine whether there is any capacity in the memory 146 available for the source based on the allocation made by the memory manager 147. If there is no capacity in the memory 146 available for the source, the network interface card 140 may drop the packet. The scheduler 141 may drop the packet by a variety of mechanisms. For example, the scheduler 141 may simply ignore the notification; however, this mechanism is typically only used if the scheduler 141 implements a data rate cap (such as a 20 Gbps cap) on the application 150. This mechanism will result in a more arbitrary form of traffic shaping that may be incompatible with the fine-grained traffic shaping disclosed herein. A second mechanism for dropping packets may include acknowledging the notification, dropping the packet, and sending a completion message back to the application 150 if the application 150 has no remaining quota for packet transmission. The completion message will be discussed further below with respect to the transmission completion logic 144. A third mechanism involves slowing down the packet sender (i.e., the application 150) rather than dropping the packet entirely. If scheduler 141 determines that there is capacity available in memory 146 for the source and application 150 has some remaining quota for transmission, scheduler 141 can respond to the notification, queue the packet to timing wheel 130, transmit the packet at the scheduled time and deliver a delayed completion message to application 150.

[0039] As mentioned above and as Figure 1As shown in , the network interface card 140 includes a timing wheel data structure 130 (also referred to as a timing wheel 130). The timing wheel data structure is a time index queue that can be implemented as a circular buffer that is used to queue objects at a given time in O(1) and obtain objects to be processed at a specific time in O(1). The time complexity of an algorithm can be estimated based on the number of elementary operations performed by the algorithm. The estimate can be expressed in the form of O(n). For example, if the value of the running time T(n) is bounded by a value that does not depend on the input size, the algorithm can have constant time (e.g., O(n), where n=1). As described above, accessing a single element (e.g., a packet identifier) ​​in the timing wheel data structure requires constant time (e.g., O(1)) because only one operation needs to be performed to locate the element. In some embodiments, the timing wheel data structure 130 can store the packet identifier provided by the scheduler 141 in a time slot associated with a timestamp specified by the application 150 that generated the packet, or store it according to an updated transmission timestamp determined by the scheduler 141. When the current time reaches or exceeds the timestamp, the timing wheel 130 can push the packet identifier to the packet builder 142 for packet generation. In some embodiments, another component such as the packet builder 142 or the scheduler 141 periodically polls the timing wheel 130 and extracts the packet identifier whose timestamp has reached or exceeded the current time. FIG. 4A to FIG. 4C The timing wheel data structure 130 and its operation are described in more detail. In some other implementations, instead of a timing wheel, a different time-indexed data structure such as a calendar queue may be used to schedule the transmission of packets.

[0040] The network interface card 140 includes a packet builder 142. The packet builder 142 includes packet generation logic that can generate data packets using a set of pointers associated with the communication stored in the memory 146. Each pointer can be a network interface card descriptor. The descriptor can describe the packet; for example, by specifying a memory location and a length of the packet data. In some embodiments, the packet data can be in the host memory 115. In some embodiments, the packet data can be in the network interface card memory 146. The packet builder 142 can build packets for transmission based on the identifiers in the timing wheel 130.

[0041] An exemplary operation of the packet builder 142 is as follows. When the current time reaches or exceeds the time slot associated with the timestamp of the packet identifier, the timing wheel 130 may push the packet identifier to the packet builder 142 for packet generation and transmission. In some embodiments, the packet builder 142 may be configured to query the timing wheel 130 to determine whether the time indexed in the timing wheel 130 has been reached, and based on the determination that the transmission time indexed in the timing wheel 130 has been reached, extract the appropriate packet identifier from the timing wheel 130. When the time index data structure 130 reaches the time corresponding to the transmission time of the identifier in the time index data structure, the timing wheel 130 may push the packet identifier to the packet builder 142, which may use the packet identifier to generate a packet for transmission. The identifier may be, for example, a pointer to an area in the memory 146 that contains a set of pointers associated with the packet identifier. The set of pointers may point to a memory location on the network interface card 140 or the network device 110, each of which stores a portion of the data constituting the packet associated with the identifier. In some embodiments, the pointer may indicate a memory address assigned by and accessible to the real OS of the network device 110. In some embodiments, the pointer may indicate a memory address assigned by and accessible to a virtual machine hypervisor or container manager. In some embodiments, the pointer set may take the form of a scatter list. Each pointer in the pointer set may point to an area in the memory 115 containing the appropriate field, component, or payload portion of the packet. The packet builder 142 may therefore build a packet header and payload based on the pointer set. In some embodiments, the packet builder 142 may directly access the host memory 115 using, for example, direct memory access (DMA). By maintaining only a single identifier for each packet or communication in the time index data structure 130, the size of the time index data structure may be minimized.

[0042] The network interface card 140 also includes a transceiver 143. The transceiver 143 includes hardware configured to send and receive communications to and from the network node 750. In some embodiments, the network interface card 140 may be capable of optionally supporting high-speed data reception and transmission in, for example, a fiber channel at which data frame rates may approach 100 gigabits per second. In some embodiments, the network interface card 140 may be configured to support lower speed communications, such as over copper (or other metal) wires, wireless channels, or other communications media.

[0043] The network interface card 140 includes a transmission completion logic 144. When the network interface card 140 transmits a data packet, the transmission completion logic 144 can generate a transmission completion message and send it to the source virtualized computing environment that initiated the communication. The application 150 requesting the transmission of the data packet can refrain from requesting additional packet transmissions until it receives notification of the completion of the previously requested packet transmission. The transmission completion message provides a feedback mechanism to the application 150 and restricts the application 150 from forwarding additional packets to the TCP layer. This mechanism can be used in conjunction with existing TCP functionality, such as a TCP small queue that is used to effectively limit the number of bytes that may be significant between the sender and the receiver.

[0044] Figure 2A A block diagram of an exemplary server 200a implementing a virtual machine environment is shown. In some embodiments, the server 200a includes hardware 210, a real operating system (OS) 220 running on the hardware 210, a hypervisor 250, and two virtual machines with guest operating systems (guest OS) 260 and 270. The hardware 210 may include a network interface card (NIC) 215 and other components. The hardware 210 may have Figure 5 The NIC 215 of the hardware 210 may have a configuration similar to that of the computing system 1010 shown in FIG. Figure 1 In some embodiments, the real OS 220 has a configuration similar to the configuration of the network interface card 140 shown in FIG. Figure 1 200a. In some embodiments, the real OS 220 includes software applications running on the real OS 220. In some embodiments, the guest OS 260 and 270 include protocol stacks 261 and 271, respectively. Each of the guest OS 260 and 270 can host a variety of applications, such as software applications 265, 266, 275, and 276. The server 200a can be a file server, an application server, a web server, a proxy server, an appliance, a network appliance, a gateway, a gateway server, a virtualization server, a deployment server, an SSL VPN server, or a firewall.

[0045] See again Figure 2A, server 200a implements a hypervisor 250 that instantiates and manages a first guest OS 260 on virtual machine 1 and a second guest OS 270 on virtual machine 2, respectively. The first guest OS 260 configured on virtual machine 1 hosts a first software application 265 and a second software application 266. The second guest OS 260 configured on virtual machine 2 hosts a third software application 275 and a fourth software application 276. For example, the applications may include database servers, data warehouse programs, stock market trading software, online banking applications, content publishing and management systems, hosted video games, hosted desktops, email servers, travel reservation systems, customer relationship management applications, inventory control management databases, and enterprise resource management systems. In some embodiments, the guest OS hosts other kinds of applications.

[0046] Figure 2B A block diagram of an exemplary server 200b implementing a containerized environment is shown. In some embodiments, the server 200b includes hardware 210, a real operating system (OS) 220 running on the hardware 210, a container manager 240, and two containerized environments (e.g., container 1 and container 2) that execute applications 241 and 242, respectively. The hardware 210 may include a network interface card (NIC) 215, among other components. The hardware 210 may have a Figure 5 The NIC 215 of the hardware 210 may have a configuration similar to the computing system 1010 shown. Figure 1 The configuration of the network interface card 140 shown in FIG. 1 is similar to that of the network interface card 140 shown in FIG. 1. In some embodiments, the real OS 220 has a protocol stack 225 (e.g., a TCP stack) and has software applications running on the real OS 220. Each of the containers (e.g., container 1 and container 2) can host various applications, such as software applications 241 and 242. The server 200b can be a file server, an application server, a web server, a proxy server, an appliance, a network appliance, a gateway, a gateway server, a virtualization server, a deployment server, an SSL VPN server, or a firewall.

[0047] See again Figure 2B, server 200b implements container manager 240, which instantiates and manages container 1 and container 2, respectively. Container 1 hosts software application 241. Container 2 hosts software application 242. For example, the application may include a database server, a data warehouse program, a stock market trading software, an online banking application, a content publishing and management system, a hosted video game, a hosted desktop, an email server, a travel reservation system, a customer relationship management application, an inventory control management database, and an enterprise resource management system. In some embodiments, a container (e.g., container 1 or container 2) may host other types of applications. In some embodiments, each container may host multiple applications simultaneously.

[0048] Figure 3 is the use of a network interface card (such as Figure 1 Flowchart of an exemplary method 300 for network traffic shaping performed by a network interface card 140 (shown in FIG. 1 ) for performing network traffic shaping. The method 300 includes receiving a notification of a new communication originating from a source virtualized computing environment (stage 310). The method 300 includes determining whether to drop a packet or schedule transmission of the new communication (stage 320). The method 300 includes storing an identifier associated with the communication in a time index data structure and storing a pointer set associated with the communication in a network interface card memory (stage 330). The method 300 includes generating a data packet using the pointer set stored in the network interface card memory (stage 340). The method 300 includes transmitting the generated packet (stage 350). The method 300 includes transmitting a transmission completion notification back to the application (stage 360).

[0049] The method 300 includes receiving a notification of a new communication originating from a source virtualized computing environment (stage 310). The notification of the new communication may include header data and a set of pointers to data to be included in the communication. The notification may include a requested transmission time. In some embodiments, the plurality of applications generating the packets may be applications hosted in one or more virtual machine environments, such as Figure 2A In addition or alternatively, the received packet may be received by an application (such as, Figure 2A In some embodiments, the Figure 2B As shown in , an application may be included in a containerized environment, such as application 241 or 242. Additionally or alternatively, the TCP layer receiving the packet may be an upper layer protocol stack of a guest OS in a virtual machine environment.

[0050] The method 300 includes determining whether to drop the packet or schedule the transmission of a new communication (stage 320). The determination may be made by a scheduling logic such as the previously described scheduler 141. The scheduler may determine whether to drop the packet or schedule the transmission of a new communication based on the available capacity in the memory space of the source virtualized computing environment in the memory (such as, memory 146) allocated to the network interface card. If the scheduler determines that there is no available capacity in the memory space allocated to the source, the scheduler may ignore the notification and drop the packet. If the scheduler determines that there is available capacity, the scheduler may schedule the packet for transmission by proceeding to stage 330.

[0051] Method 300 includes storing the identifier associated with communication in a time index data structure and storing the pointer set associated with communication in a network interface card memory (stage 330). The scheduler can store the identifier in a time index data structure, such as the timing wheel 130 described previously. The time index data structure can be configured to include multiple locations or time slots to store data or events. The time index data structure includes a time range, which is the maximum time period to the future that data or events can be stored. For example, the time index data structure can be configured to include 50 time slots, wherein each time slot represents the minimum time granularity between two events. If a time index data structure including 50 time slots is configured so that each time slot represents a granularity of 2 microseconds, the time range will be 100 microseconds. In this example, no data or event is scheduled beyond 100 milliseconds in the future. Suitable time range and timing wheel granularity (e.g., the number of time slots) can be configured based on the rate limiting strategy to be implemented. For example, in order to implement a rate of 1 megabit (Mb) per second, a suitable time range should be 12 milliseconds. A suitable number of time slots or positions for the time index data structure may be in the range of 10 to 1 million time slots or positions. A suitable time range for the time index data structure may be in the range of microseconds to seconds. In some embodiments, one or more timing wheels may be implemented hierarchically, and each of the one or more timing wheels may be configured to have a different number of time slots and a different timing wheel granularity. In this example, each of the one or more hierarchical timing wheels may have a different time range. In some embodiments, the packet identifier may correspond to a timestamp requested by an application generating the packet or an adjusted transmission timestamp determined by a scheduler. For example, a packet may include an identifier that may specify a requested transmission time that is 10 microseconds from the current time. The scheduler may process the packet to determine whether transmitting the packet immediately will exceed the rate limit based on a rate limiting policy associated with the particular class of packets. Assuming that the rate limit is not exceeded, the scheduler 11 may insert the packet identifier into the time index data structure at a position associated with a transmission time of 10 microseconds in the future. In some embodiments, if the timestamps of all packets are zero (e.g., the transmission time is now) or any value less than now, the time index data structure can act as a first-in-first-out (FIFO) queue. For example, the packet identifiers with a timestamp of zero will be transmitted immediately. Additionally or alternatively, all packet identifiers with timestamps earlier than now are inserted into the data structure position with the smallest time so that they can be transmitted immediately. Any packet identifiers with timestamps outside the time range of the time index data structure are inserted into the last position in the data structure (e.g., the position representing the largest time range).

[0052] Also in stage 330, the scheduler, upon determining that the source has available capacity in memory allocated to it, may store a set of pointers associated with the communication in the network interface card memory.

[0053] In some embodiments, the scheduler may process the received data packets based on a rate limiting algorithm or policy stored in a memory to determine the transmission time for each packet. For example, the scheduler may process the packets and apply a transmission timestamp according to a rate limiting algorithm or policy associated with a particular class of packets. In some embodiments, the scheduler is configured to determine the transmission time for each packet based on a rate pacing policy or a target rate limit. For example, the scheduler may determine the transmission time for each packet based on a rate pacing policy such as a packet class rate policy and / or an aggregate rate policy. In some embodiments, the scheduler may determine the transmission time based on a rate pacing policy (such as a weighted fair queuing policy) to process multiple packet flows. Additionally or alternatively, each packet may have a transmission timestamp requested by an application generating the packet. In some embodiments, the scheduler may receive a packet including a requested transmission time assigned to the packet by one of a plurality of applications before being received at the TCP layer and before being processed by the scheduler. The scheduler may process the packet substantially in real time to determine an updated transmission time based on exceeding at least one rate limiting policy and invoking a rate limiting algorithm associated with the packet. For example, if a received packet is processed and the scheduler determines that the transmission time of the packet will exceed the rate limit of the packet class, the scheduler can update the transmission time using an adjusted transmission timestamp that enables the packet to be transmitted at a later time to avoid exceeding the rate limit defined by the rate limiting policy for the particular packet class. The scheduler can be configured to locate the associated rate limiting algorithm via a hash table or a mapping that identifies the rate limiting algorithm associated with the received packet.

[0054] In some embodiments, the scheduler may determine that a specific transmission time associated with a packet identifier stored in the time index data structure 130 has been reached. The scheduler may query the time index data structure 130 with the current time to determine whether there are any packets to be transmitted. For example, the scheduler may query the data structure using the current CPU clock time (or some other reference time value, such as a regularly increasing integer value). Frequent polling may provide greater consistency with packet scheduling and rate limiting policies and reduce overhead compared to using a separate timer (which may cause a large amount of CPU overhead due to interrupts). In some embodiments, the time index data structure 130 may be implemented on a dedicated CPU core. Additionally or alternatively, the time index data structure 130 may be implemented on an interrupt-based system that may perform polling of the data structure at constant intervals to determine the packet transmission schedule. For example, the time index data structure 130 may be periodically polled with a period equal to the length of time associated with each time slot or a multiple thereof. In some embodiments, the polling of the timing round may be performed by logic different from the scheduler (such as a packet builder similar to the packet builder 142).

[0055] Method 300 includes generating a data packet using a pointer set stored in a network interface card memory (stage 340). The packet can be generated by a packet generation logic such as the previously described packet builder 142. After the scheduled transmission time of the communication whose identifier is stored in the time index data structure arrives, the packet builder can use the pointer set stored in the network interface card memory associated with the communication to generate the data packet. The packet builder can retrieve data from the network device memory location indicated by the pointer in the pointer set and generate the packet accordingly. For example, the pointer set can be stored in the network interface card memory 146, and the pointer set can indicate the location containing the packet data in the network interface card memory 146 and / or the network device memory 115. In some embodiments, the pointer set is stored as a scatter list. In some embodiments, the packet builder can use, for example, direct memory access (DMA) to directly access the host memory to retrieve the data stored in the memory location indicated by the pointer in the pointer set.

[0056] Method 300 includes transmitting the generated packets (stage 350). The network interface card can transmit the generated data packets via a network link such as one of the previously described links 600. In some embodiments, the network interface card can transmit the data packets based on reaching (or exceeding) the transmission time identified in the packet identifier stored in the time index data structure. For example, a scheduler or packet builder can poll the time index data structure and can determine that the transmission time identified in the packet identifier has been reached. In response, the scheduler can instruct the network interface card to transmit the generated data packets. In some embodiments, the scheduler can identify a transmission time earlier than now, and in response, transmit the packet immediately.

[0057] In some embodiments, method 300 may prevent one of the applications from sending additional packets for transmission until the application receives notification that the transmission is complete. In some embodiments, traffic shaping may be achieved in part by rate limiting the forwarding of additional data packets by the application to the TCP layer until a message indicating that the packet transmission is complete is received. For example, network interface card 140 (e.g., Figure 1 , and described in more detail later) can generate a completion notification back to the application 150, indicating that the packet has been transmitted through the network. The transmission completion notification provides a feedback mechanism to the application and limits the forwarding of additional packets to the TCP layer by the application 150. This mechanism can be used in conjunction with existing TCP functionality, such as TCP small queues that effectively limit the number of bytes that may be significant between the sender and the receiver.

[0058] The method 300 includes transmitting a transmission completion notification back to the application (stage 360). The network interface card may include transmission completion logic configured to transmit a completion notification back to the application that originated the packet after the transmission of the packet by the network interface card is completed. The completion notification allows the application 150 to send additional packets to the network interface card. The transmission completion notification mechanism will be described in more detail below.

[0059] The functionality described above that occurs within the TCP layer of a network device may additionally or alternatively be implemented in another network protocol module within the transport layer, network layer, or combined transport / network layer of a network protocol stack. For example, the functionality may be implemented in a user datagram protocol (UDP) module, a reliable datagram protocol (RDP) module, a reliable user datagram protocol (RUDP) module, or a datagram congestion control protocol (DCCP) module.

[0060] FIG. 4A to FIG. 4C1 is a block diagram representing exemplary operations of shaping network traffic using a scheduler, a time index data structure, a memory, a memory manager, and a packet builder as performed by a network interface card such as network interface card 140. In summary, as Figure 4A As shown in , the network interface card 140 receives notifications of new communications from applications 150 (e.g., applications 150a, 150b, and 150c) via the network driver 120. A notification, such as the notification labeled REQ:A1, may include a header (or portions of a header) and a set of pointers corresponding to memory locations in host memory storing additional data for the packet header and for the packet payload. The network interface card 140 includes one or more memory devices 146 for storing instructions and data, and allocation logic, such as a memory manager 147, for allocating portions of the memory 146 to different virtual machines, containers, or applications 150. In FIG. 4A to FIG. 4C In the exemplary operation shown in , for simplicity, each application 150 is considered to be implemented in a different virtual machine or container. The network interface card 140 includes a scheduler 141 to process packets according to a rate limiting algorithm or policy stored in a memory 146. The network interface card 140 also includes a time index data structure 130, also referred to as a timing wheel 130, to store packet identifiers according to their transmission time, such as a packet identifier marked as ID: A1. The network interface card 140 also includes one or more packet builders 142 to generate packets based on packet identifiers and associated pointer sets. If the network interface card 140 receives a notification of a new communication from an application, the scheduler 141 can determine whether the memory allocation in the memory 146 associated with a particular virtual machine or container has capacity for processing the pointer set associated with the packet. If the memory allocation is sufficient to store the pointer set, the scheduler 141 can store the packet identifier in the timing wheel 130 and store the associated pointer set in the memory 146. If the memory allocation lacks capacity for storing the pointer set, the network interface card 140 may ignore the notification, essentially discarding the packet requested for transmission.When the transmission time indexed for the packet identifier has been reached (e.g., at time now 420), the timing wheel 130 may push the packet identifier to the packet builder 142 to generate the packet. FIG. 4A to FIG. 4C These operations are illustrated.

[0061] See also Figure 4A, the network interface card 140 receives a notification REQ:A1 of a new communication from the application 150a at its scheduler 141. The notification REQ:A1 corresponds to a data packet for transmission from the application 150a via the network interface card 140. The notification REQ:A1 may include a header or header data and a set of pointers. The set of pointers may point to a memory area in the host memory where data for the packet header and / or payload is stored. At the network interface card 140, the set of pointers (in this case A1) is associated with an identifier (in this case ID:A1). Thus, the identifier is associated with: a notification of a new communication received; a packet for transmission; and a set of pointers to data, which will ultimately be used by the network interface card 140 to generate a packet.

[0062] Memory manager 147 has allocated space in memory 146 for each virtual machine or container. FIG. 4A to FIG. 4C In order to illustrate the operation of the network interface card 140 and its components, each application 150 is considered to be executed in a separate virtual machine or container. Figure 4A , memory manager 147 has allocated space in memory 146 for three pointer sets from each of the virtual machines hosting applications 150a and 150b, respectively. In practice, the memory allocation for the virtual machines may be a single pointer set or many pointer sets. In some implementations, memory space may be allocated in bytes, kilobytes, or megabytes rather than in terms of the number of pointer sets, and the size of each pointer set may be different so that the number of pointer sets that fit into a particular memory allocation depends on the total size of the pointer sets.

[0063] like Figure 4A As shown in FIG. 1 , the scheduler 141 of the network interface card 140 receives the notification REQ:A1 from the application 150a and checks the available capacity of the memory 146. Figure 4A In the example shown in , memory 146 has available capacity allocated to the virtual machine executing application 150a. Therefore, the scheduler adds the packet identifier associated with the notification to timing wheel 130 and adds the pointer set associated with the packet to memory 146.

[0064] like Figure 4AAs further shown in FIG. 1 , the scheduler 141 processes the received notification to determine the transmission time of each packet. The scheduler 141 may determine the transmission time of each packet by identifying the rate limiting algorithm or policy associated with the packet and assigning the initial or updated transmission time to the identifier associated with the packet. The scheduler 141 may retrieve the rate limiting algorithm or policy from the memory 146 to determine the initial or updated transmission time of each packet. The scheduler 141 may identify the received notification as a packet belonging to a specific category. A specific category of packets may require a specific rate limiting algorithm or policy associated with the packet category. The scheduler 141 may utilize a specific rate limiting algorithm or policy to determine the initial or updated transmission time of each packet of the category. In some embodiments, the scheduler 141 may evaluate the packet transmission time requested by the application 150 and determine whether the requested transmission time exceeds the rate limiting algorithm or policy associated with the packet category (e.g., whether the transmission at the requested time will cause the transmission rate of the packet category to be too high, taking into account the transmission history of other recently transmitted packets or packets in the category that have been scheduled for future transmission). Scheduler 141 may process the notification and determine that the transmission time requested by application 150 violates a rate limit or policy associated with the packet class. If the rate limit or policy is exceeded or otherwise violated, scheduler 141 may determine an updated transmission time that does not exceed or violate the rate limit or policy for each packet. Scheduler 141 may determine that the requested or updated transmission time is current and may implement instructions to immediately forward the identifier to packet builder 142 to generate a packet corresponding to the identifier.

[0065] like Figure 4AAs shown in , the scheduler 141 stores the identifier ID:A1 associated with the packet in the timing wheel 130 at a position associated with the transmission time determined for the packet. The timing wheel 130 can be a time index data structure or queue that can store and extract packet identifiers based on the determined transmission time of the associated packet. In some embodiments, each time slot in the timing wheel 130 stores a single data element or event (e.g., an identifier associated with the packet). In some embodiments, each time slot can store multiple data elements or events. The timing wheel 130 can include a preconfigured number of time slots or positions, and each time slot or position can represent a specific time increment. In some embodiments, the number of time slots or positions can be dynamically adjusted based on the changing level of data traffic and congestion at the network interface card 140. The timing wheel 130 can include any number of time slots or positions, wherein each time slot is defined as the amount of traffic that needs to be fully processed to be shaped. The sum of all time slots or positions in the timing wheel 130 represents the time range or forward queuing time frame that the timing wheel 130 can support. The appropriate time range and timing wheel granularity (e.g., number of time slots) may be configured based on the rate limiting policy to be implemented. For example, to implement a rate of 1 megabit (Mb) per second, an appropriate time range would be 12 milliseconds. An appropriate number of time slots or positions for the timing wheel 130 may be in the range of 10 to 1,000,000 time slots or positions. An appropriate time range for the timing wheel 130 may be in the range of 10 microseconds to 1 second. For example, if Figure 4A As shown in , the timing wheel 130 has 10 time slots, and each time slot can represent 2 microseconds. Therefore, Figure 4A The time range of the exemplary timing wheel 130 shown in is 20 microseconds, and the granularity of the timing wheel 130 is 2 microseconds. In some embodiments, the timing wheel 130 will have a maximum time range, and no packet identifier will be scheduled beyond this maximum time range. The timing wheel 130 may not need to store packet identifiers with timestamps earlier than now, because packets with transmission times earlier than now should be transmitted immediately. Once the time slot in the timing wheel 130 becomes earlier than now, the elements in the time slot can be dequeued and prepared for transmission.

[0066] For example, Figure 4AAs shown in , it is assumed that the scheduler 141 has processed the packet associated with the packet identifier ID: A1. The pointer set associated with the identifier A1 can be retained in the memory 146, and the scheduler 141 stores the identifier ID: A1 in the timing wheel 130 at a location associated with the transmission time determined for the packet associated with the identifier ID: A1. The packet identifier ID: A1 includes the transmission time t0 determined by the scheduler 141. The packet identifier ID: A1 is inserted into the timing wheel 130 at the time slot corresponding to the transmission time t0. The timing wheel 130 stores the packet identifier ID: A1 until it is determined that the transmission time determined for the packet associated with the identifier A1 has been reached. In some embodiments, when the time now becomes t0, the timing wheel 130 can push the packet identifier ID: A1 to the packet builder. In some embodiments, the scheduler 141 or the packet builder 142 can query the time index data structure 130 with the current time to determine whether there are any packets to be transmitted. For example, the packet builder 142 may poll the data structure with the CPU clock time (or some other value representing the current time, such as a regularly increasing integer.) The packet builder 142 may determine that the transmission time identified in the packet identifier ID: A1 has been reached, and therefore generate a packet for transmission using the pointer set A1.

[0067] like Figure 4B As shown in , scheduler 141 processes the next notification. In this case, the scheduler receives notification REQ:B4 from application 150b. The scheduler checks the available capacity in the space allocated in memory 146 for the virtual machine or container implementing application 150b. Figure 4A In the example shown in , the memory 146 lacks capacity for the pointer set associated with the notification REQ: B4. Therefore, the scheduler 141 discards the notification without queuing the packet identifier in the timing wheel 130 or storing the associated pointer set in the memory 146.

[0068] like Figure 4C As shown in , the current time has reached or exceeded t0, which is the time when packet A1 is scheduled for transmission. Therefore, the timing wheel 130 pushes the packet identifier ID: A1 to the packet builder 142 for packet generation and transmission. The packet builder 142 uses the packet identifier ID: A1 to retrieve the pointer set A1 from the memory 146. The packet builder 142 uses the pointer set A1 to retrieve the packet header data and the payload data from the host memory to generate the packet PKT: A1. The packet builder 142 can then send the packet PKT: A1 to the transceiver for transmission.

[0069] After transmission of the packet, the transmission completion logic 144 may determine whether the network interface card successfully transmitted the packet associated with the identifier stored in the time index data structure at the location associated with the arrival time. If the transmission completion logic 144 determines that the packet associated with the identifier stored in the time index data structure at the location associated with the arrival time was successfully transmitted, the transmission completion logic 144 may transmit a transmission completion notification to the application 150, which has been waiting to receive the transmission completion notification from the network interface card 140 before forwarding additional data packets to the network interface card. Exemplary operations of the transmission completion logic 144 are described in more detail below.

[0070] The transmission completion logic 144 can determine whether the network interface card 140 has successfully transmitted the packet associated with the identifier stored in the time index data structure 130 at the location associated with the arrival time. For example, in response to the network interface card 140 successfully completing the transmission of packet A1, the transmission completion logic 144 can notify the application 150 of the successful transmission of the packet by transmitting a message. In the event of multiple successful transmission completions, the transmission completion logic 144 can transmit a single message or multiple transmission completion notifications. Based on the notification of transmission completion from the network interface card 140, the application 150 determines that the network interface card 140 has successfully transmitted each packet associated with the identifier stored in the time index data structure at the location associated with the arrival time.

[0071] In response to the transmission completion logic 144 determining that the packet associated with the identifier stored at the location associated with the arrival time in the time index data structure has been successfully transmitted, the transmission completion logic 144 can transmit a transmission completion notification to the application 150, which has been waiting to receive the transmission completion notification from the network interface card before forwarding additional data packets to the network interface card. In some embodiments, each of the applications 150 can be configured to wait to receive the transmission completion notification from the network interface card 140 before forwarding additional packets to the network interface card. In some embodiments, each of the applications 150 can be configured to wait to receive a transmission completion message for a particular class of packets from the network interface card before forwarding additional packets of the same class to the network interface card.

[0072] Figure 5 is a block diagram illustrating the general architecture of a computer system 1000 that may be used to implement the elements of the systems and methods described and illustrated herein, according to an illustrative embodiment.

[0073] In summary, the computing system 1010 includes at least one processor 1050 for performing actions according to instructions; and one or more storage devices 1070 or 1075 for storing instructions and data. The illustrated example computing system 1010 includes one or more processors 1050, which communicate via a bus 1015 with at least one network interface driver controller 1020 having one or more network interface cards 1022 connected to one or more network devices 1024, a memory 1070, and any other devices 1080; for example, an I / O interface. The network interface card 1022 may have one or more network interface driver ports to communicate with the connected devices or components. Typically, the processor 1050 will execute instructions received from the memory. The illustrated processor 1050 incorporates or is directly connected to a cache memory 1075.

[0074] In more detail, processor 1050 may be any logic circuit that processes instructions retrieved, for example, from memory 1070 or cache 1075. In many embodiments, processor 1050 is a microprocessor unit or a special purpose processor. Computing device 1000 may be based on any processor or collection of processors capable of operating as described herein. Processor 1050 may be a single core or multi-core processor. Processor 1050 may be a plurality of processors. In some embodiments, processor 1050 may be configured to run multi-threaded operations. In some embodiments, processor 1050 may host one or more virtual machines or containers and a hypervisor or container manager for managing the operation of the virtual machines or containers. In such embodiments, the processing may be implemented within a virtualized or containerized environment provided on processor 1050. Figure 3 and Figure 5 The method shown in .

[0075] The memory 1070 may be any device suitable for storing computer-readable data. The memory 1070 may be a device having fixed storage or a device for reading removable storage media. Examples include all forms of non-volatile memory, media, and memory devices, semiconductor memory devices (e.g., EPROM, EEPROM, SDRAM, and flash memory devices), magnetic, magneto-optical, and optical disks (e.g., CD ROM, DVD-ROM, and The computing system 1000 may have any number of memory devices 1070. In some implementations, the memory 1070 supports virtualized or containerized memory accessible by a virtual machine or container execution environment provided by the computing system 1010.

[0076] Cache memory 1075 is typically a form of computer memory that is placed in close proximity to processor 1050 for fast reads. In some implementations, cache memory 1075 is part of processor 1050 or on the same chip as processor 1050. In some implementations, there are multiple levels of cache 1075, for example, L2 and L3 cache layers.

[0077] The network interface driver controller 1020 manages data exchange via the network interface driver 1022 (also referred to as the network interface driver port). The network interface driver controller 1020 processes the physical and data link layers of the OSI model for network communication. In some embodiments, some of the tasks of the network interface driver controller are handled by the processor 1050. In some embodiments, the network interface driver controller 1020 is a part of the processor 1050. In some embodiments, the computing system 1010 has multiple network interface driver controllers 1020. The network interface driver port configured in the network interface card 1022 is a connection point for a physical network link. In some embodiments, the network interface controller 1020 supports wireless network connections, and the interface port associated with the network interface card 1022 is a wireless receiver / transmitter. Typically, the computing device 1010 exchanges data with other network devices 1024 through a physical or wireless link docked with the network interface driver port configured in the network interface card 1022. In some embodiments, the network interface controller 1020 implements a network protocol such as Ethernet.

[0078] The other network devices 1024 are connected to the computing device 1010 via the network interface driver ports included in the network interface card 1022. The other network devices 1024 may be peer computing devices, network devices, or any other computing devices with network functionality. For example, the first network device 1024 may be a network device such as a network hub, a bridge, a switch, or a router that connects the computing device 1010 to a data network such as the Internet.

[0079] Other devices 1080 may include I / O interfaces, external serial device ports, and any additional coprocessors. For example, computing system 1010 may include interfaces (e.g., universal serial bus (USB) interfaces) for connecting input devices (e.g., keyboards, microphones, mice, or other pointing devices), output devices (e.g., video displays, speakers, or printers), or other storage devices (e.g., portable flash drives or external media drives). In some embodiments, computing device 1000 includes additional devices 1080 such as coprocessors, for example, math coprocessors that can assist processor 1050 in performing high-precision or complex calculations.

[0080] The implementation of the subject matter and operations described in this specification may be implemented in digital electronic circuits or in computer software embodied in tangible media, firmware or hardware (including the structures disclosed in this specification and their equivalents), or in a combination of one or more thereof. The implementation of the subject matter described in this specification may be implemented as one or more computer programs embodied in tangible media, that is, one or more modules of computer program instructions encoded on one or more computer storage media for execution by a data processing device or for controlling the operation of a data processing device. The computer storage medium may be a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination of one or more thereof, or included therein. The computer storage medium may also be one or more separate components or media (e.g., multiple CDs, disks, or other storage devices) or included therein. The computer storage medium may be tangible and non-transitory.

[0081] The operations described in this specification can be implemented as operations performed by a data processing device on data stored on one or more computer-readable storage devices or received from other sources. The operations can be performed within the local environment of the data processing device or within one or more virtual machines or containers hosted by the data processing device.

[0082] A computer program (also referred to as a program, software, software application, script, or code) may be written in any form of programming language (including compiled or interpreted languages, declarative or procedural languages) and may be deployed in any form, including deployment of standalone programs or modules, components, subroutines, objects, or other units suitable for use in a computing environment. A computer program may, but need not, correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), a single file dedicated to the program in question, or multiple coordinated files (e.g., files storing one or more modules, subroutines, or partial codes). A computer program may be deployed to execute on one computer or multiple computers or one or more virtual machines or containers located at one site or distributed at multiple sites and interconnected by a communication network. Examples of communication networks include local area networks ("LANs") and wide area networks ("WANs"), interconnected networks (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks).

[0083] The processes and logic flows described in this specification can be performed by one or more programmable processors executing one or more computer programs to perform actions by operating on input data and generating output. The processes and logic flows can also be performed by special purpose logic circuits, and the apparatus can also be implemented as special purpose logic circuits, such as FPGAs (field programmable gate arrays) or ASICs (application specific integrated circuits).

[0084] Although this specification contains many specific implementation details, these details should not be interpreted as limitations on the scope of any invention or content that may be claimed, but should be interpreted as descriptions of features of specific embodiments dedicated to specific inventions. Certain features described in this specification in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, the various features described in the context of a single embodiment may also be implemented separately or in any suitable sub-combination in multiple embodiments. Moreover, although the features may be described above as working in certain combinations and even initially claimed as such, one or more features in the claimed combination may be deleted from the combination in some cases, and the claimed combination may be directed to a sub-combination or a variant of the sub-combination.

[0085] Similarly, although operations are depicted in a particular order in the accompanying drawings, this should not be understood as requiring that such operations be performed in the particular order shown or in a continuous order, or that all of the operations shown be performed, to achieve the desired results. In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of various system components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0086] References to "or" may be interpreted as inclusive, such that any term described using "or" may indicate any of a single, more than one, and all of the described terms. Labels "first," "second," "third," etc. do not necessarily indicate an order and are generally used only to distinguish similar items or elements.

[0087] Various modifications to the embodiments described in this disclosure will be apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments without departing from the spirit or scope of the disclosure. Therefore, the claims are not intended to be limited to the embodiments shown herein, but should be given the broadest scope consistent with the disclosure, principles, and novel features disclosed herein.

Claims

1. A network interface card, comprising: a network interface card memory, the network interface card memory being configured to store: storing a time-indexed data structure of identifiers associated with respective communications to be transmitted by the network interface card; as well as a set of pointers, wherein each set of pointers corresponds to one of the identifiers stored in the time index data structure; A scheduling logic, wherein the scheduling logic is configured to: receiving a notification of a new communication to be transmitted by the network interface card, the new communication originating from a source virtualized computing environment, wherein the notification includes a set of pointers to data to be included in the new communication; determining whether to drop a data packet or schedule transmission of the new communication based on available capacity in the network interface card memory allocated to the source virtualized computing environment; and responsive to determining that the new communication is to be scheduled for transmission, storing an identifier associated with the new communication in the time index data structure and storing a set of pointers associated with the new communication in the network interface card memory at the scheduled transmission time; packet generation logic configured to generate a data packet at the scheduled transmission time using the identifier associated with the new communication stored in a memory of the network interface card; and A transceiver is configured to transmit the generated data packet.

2. The network interface card of claim 1, further comprising network interface card memory allocation logic configured to allocate portions of the network interface card memory among the source virtualized computing environments.

3. The network interface card according to claim 1, wherein: The notification further includes header data.

4. The network interface card according to claim 1, wherein: The identifier associated with the new communication is stored at the scheduled transmission time.

5. The network interface card according to claim 4, wherein: The scheduling logic is further configured to calculate the scheduled transmission time for communication based on a traffic shaping policy stored in the network interface card memory.

6. The network interface card according to claim 5, wherein: The scheduling logic is configured to calculate the scheduled transmission time based on header data associated with the communication.

7. The network interface card according to claim 5, wherein: The service shaping strategy includes a packet class rate strategy or an aggregation rate strategy.

8. The network interface card according to claim 1, wherein: The source virtualized computing environment refrains from requesting additional packet transmissions until notification of completion of a previously requested packet transmission is received.

9. The network interface card according to claim 1, wherein: The set of pointers is archived as a scatter-gather list.

10. A method for network service shaping, comprising: receiving, at a network interface card of a host computing device, a notification of a new communication to be transmitted by the network interface card, the new communication originating from a source virtualized computing environment, wherein the notification includes a set of pointers to data to be included in the new communication; determining whether to drop the data packet or schedule transmission of the new communication based on available capacity in a network interface card memory allocated to the source virtualized computing environment; responsive to determining that the new communication is to be scheduled for transmission, storing an identifier associated with the new communication in a time index data structure at the scheduled transmission time and storing a set of pointers associated with the new communication in the network interface card memory, wherein each set of pointers corresponds to one of the identifiers stored in the time index data structure; generating a data packet at the scheduled transmission time using the set of pointers associated with the new communication stored in a memory of the network interface card; as well as The generated data packets are transmitted.

11. The method of claim 10, further comprising allocating portions of the network interface card memory among the source virtualized computing environments.

12. The method according to claim 10, wherein: The notification further includes header data.

13. The method according to claim 10, wherein: The identifier associated with the new communication is stored at the scheduled transmission time.

14. The method of claim 13, further comprising calculating the scheduled transmission time for communication based on a traffic shaping policy stored in a memory of the network interface card.

15. The method of claim 14, further comprising calculating the scheduled transmission time based on header data associated with the communication.

16. The method according to claim 14, wherein: The service shaping strategy includes a packet class rate strategy or an aggregation rate strategy.

17. A non-transitory computer readable medium having instructions stored thereon, the non-transitory computer readable medium being configured to cause one or more processors of a network interface card of a host computing device to perform a method of network traffic shaping, the method comprising: receiving a notification of a new communication to be transmitted by the network interface card, the new communication originating from a source virtualized computing environment, wherein the notification includes a set of pointers to data to be included in the new communication; determining whether to drop the data packet or schedule transmission of the new communication based on available capacity in a network interface card memory allocated to the source virtualized computing environment; responsive to determining that the new communication is to be scheduled for transmission, storing an identifier associated with the new communication in a time index data structure at the scheduled transmission time and storing a set of pointers associated with the new communication in the network interface card memory, wherein each set of pointers corresponds to one of the identifiers stored in the time index data structure; generating a data packet at the scheduled transmission time using the set of pointers associated with the new communication stored in a memory of the network interface card; as well as The generated data packets are transmitted.

18. The computer-readable medium of claim 17, wherein: The method further includes allocating portions of the network interface card memory among the source virtualized computing environments.

19. The computer-readable medium of claim 17, wherein: The notification includes header data.

20. The computer-readable medium of claim 17, wherein: The identifier associated with the new communication is stored at the scheduled transmission time.

21. The computer readable medium of claim 20, wherein: The method further includes calculating the scheduled transmission time for communication based on a traffic shaping policy stored in a memory of the network interface card.

Images