Methods for intelligent network interface controllers and related machine-readable media

Abstract

Some embodiments provide a method for a first smart NIC among a plurality of smart NICs for a host computer. Each of the smart NICs executes a smart NIC operating system that performs virtual network operations for a set of data computing machines running on the host computer. The method receives a data message sent by one of the data computing machines running on the host computer. The method performs virtual network operations on the data message to determine that the data message will be transmitted from a port of a second smart NIC among the plurality of smart NICs. The method transmits the data message to the second smart NIC via a dedicated communication channel connecting the plurality of smart NICs.

Description

Technical Field

[0001] This disclosure relates to methods for intelligent network interface controllers and related machine-readable media. Background Technology

[0002] Many operations typically associated with a host computer are pushed to a programmable intelligent network interface controller (NIC). Some of the operations pushed to these intelligent NICs involve virtual network processing of data messages for computing machines. In some cases, the host computer will have multiple such intelligent NICs performing network processing or other operations. While performing operations previously executed on the host computer (e.g., by the host computer's hypervisor), it is desirable for these intelligent NICs to work together. Summary of the Invention

[0003] Some embodiments provide methods for enabling multiple smart NICs on the same host computer to operate as a single entity (e.g., as a group of smart NICs). In some embodiments, each smart NIC executes a smart NIC operating system that performs virtual network operations (and / or other operations, such as virtual storage operations) for a group of data compute nodes (e.g., virtual machines (VMs), containers, etc.) running on the host computer. In some embodiments, the smart NICs are connected via dedicated communication channels to share dynamic state information, share configuration data (so that one of the smart NICs can act as a single point of contact for network management and control systems), and / or transmit data messages to and from data compute nodes (DCNs) that require virtual network processing.

[0004] By running a smart NIC operating system, a smart NIC can perform various tasks that would otherwise be performed by the host computer software (e.g., the host computer's hypervisor). These tasks may include virtual network processing of data messages (i.e., performing virtual switching and / or routing, firewall operations, etc.), virtual storage operations, etc. To enable multiple smart NICs to perform these operations that would otherwise be performed entirely by a single entity (e.g., the hypervisor), communication between the smart NICs may be necessary.

[0005] As mentioned, in some embodiments, a dedicated communication channel is established between smart NICs to enable communication between them. In some embodiments, the dedicated communication channel is a physically separate channel. For example, in some embodiments, smart NICs are connected via a set of physical cables that carry only communication between smart NICs. In different such embodiments, smart NICs may be connected serially (so that each smart NIC is directly connected to two other smart NICs, except for a smart NIC at one end of the connection that is connected to only one other smart NIC), connected in a loop (similar to a serial connection, but in which all smart NICs are connected to two other smart NICs), or connected via a separate physical switch, so that each smart NIC can communicate directly with any other smart NIC through the physical switch. If there are enough available ports, the cable may be connected to the Ethernet ports of the smart NICs (thereby occupying these ports so that the host computer's network traffic does not use these ports) or to the management ports of the smart NICs (typically lower bandwidth ports). In some embodiments, smart NICs use a separate, dedicated channel designed to connect smart NICs to each other, rather than occupying ports that can be used for other purposes.

[0006] In other embodiments, the smart NICs communicate via a logically dedicated communication channel using existing physical connections. For example, if all smart NICs are connected to the same data link layer (Layer 2) network, a dedicated virtual local area network (VLAN) can be used as a dedicated communication channel for the smart NICs. However, if this existing Layer 2 network has numerous other host computers (each with its own group of smart NICs requiring separate VLANs) and also carries data messages for DCNs on the host computers, then a maximum number of VLANs can be reached. Some embodiments alternatively use overlay networks based on encapsulation (e.g., Virtual Extensible LAN (VXLAN) or Geneve Network Virtualization Encapsulation) as logically dedicated communication channels. Such overlay networks are not as number-constrained as VLANs and also have the advantage of allowing smart NICs to communicate across multiple Layer 2 networks when necessary (i.e., as long as all smart NICs are on the same Layer 3 network).

[0007] In other embodiments, the host computer's smart NIC communicates via a dedicated communication channel traversing the host computer. For example, the smart NIC is typically connected to the host computer's Fast Peripheral Component Interconnect (PCIe) subsystem, which can be used for the dedicated communication channel. In various embodiments, the smart NIC uses the standard point-to-point transmission features of PCIe, utilizes PCIe switching infrastructure, or uses other enhancements on top of PCIe (e.g., Compute Fast Link (CXL)).

[0008] As mentioned, one use of the dedicated communication channel is to enable a first smart NIC to pass data messages (e.g., data messages sent to or from a host computer or executed by a DCN on a host computer) to a second smart NIC. The smart NICs operate as a single entity because their respective smart NIC operating systems collectively implement a set of virtual network operations (e.g., implementing logical switches and / or routers, firewalls, etc.). However, each smart NIC has its own interface (e.g., physical and virtual functions) and its own physical network port, to which the host computer's DCN is bound.

[0009] Thus, the first smart NIC will receive data messages from the DCN bound to its port. If the smart NICs jointly implement virtual network operation, then this first smart NIC processes these data messages. However, based on this processing, the data messages may need to be transmitted to the second smart NIC via a dedicated communication channel so that the second smart NIC can output the data messages. For example, if the destination is another DCN bound to the second smart NIC on the host computer, then the first smart NIC will need to pass the data messages to the second smart NIC so that the data messages can be output via the correct interface. Additionally, if all ports of the smart NIC are grouped in a Link Aggregation Group (LAG), then connections for a single DCN are load-balanced across these ports; therefore, some data messages sent from a specific DCN bound to the first smart NIC's interface to the first smart NIC will be output to the physical network via other smart NICs. Conversely, data messages received at the first smart NIC's physical network port will be processed by the first smart NIC but may need to be sent to the second smart NIC for delivery to the destination DCN bound to the second smart NIC. In another scenario, if all physical network ports of the first smart NIC have failed, but the smart NIC itself is still operational, then the smart NIC can still perform virtual network operations on data messages, but will need to send those data messages to other smart NICs for output to the physical network, regardless of whether the ports are operating in LAG.

[0010] In many cases, smart NICs receive configuration data for virtual network operation from network management and control systems. Each smart NIC has its own set of ports (potentially including management ports) with its own network address, but many network management and control systems treat each host computer as a single entity (e.g., communicating with an agent in the hypervisor of a host computer that is not using a smart NIC for network virtualization). The network management and control system uses a single management network address for each host computer and therefore should not communicate directly with all the multiple smart NICs on the host computer.

[0011] In some embodiments, smart NICs use clustering technology to appear as a single entity of a host computer to the network management and control system. For example, in some embodiments, the host computer's smart NICs perform leader election to determine the individual smart NICs that communicate with the network management and control system. In some such embodiments, each of the smart NICs' operating systems runs a deterministic algorithm that selects one of the smart NICs as the contact point. Any messages required for this leader election are transmitted via a dedicated communication channel.

[0012] The selected smart NIC receives configuration data (e.g., configuration data for logical switches and logical routers) from the network management and control system and distributes this data to other smart NICs via a dedicated communication channel, enabling all smart NICs to perform virtual network operations on data messages sent to and from the DCN running on the host computer. In some embodiments, the network management and control system includes a management plane (MP) and a central control plane (CCP), which perform different functions and provide different configuration data to the host computer (in addition to receiving different data from the host computer). In some cases, the smart NIC selects two different leaders, one for communicating with the MP and one for communicating with the CCP.

[0013] In addition to disseminating configuration data from the network management and control system, the leader smart NIC also receives information from other smart NICs via a dedicated communication channel. Some of this information is reported to the network management and control system. This information may include runtime statistics (e.g., data message processing statistics), status information, etc., and can be used by the network management and control system and / or the leader smart NIC to monitor the host computer and / or smart NICs. The network management and control system can also use this information to modify the virtual network configuration for the smart NICs.

[0014] For various purposes, in some embodiments, the smart NIC also uses a dedicated communication channel to synchronize dynamic status information. For example, in the event of a leader smart NIC failure, monitoring data retrieved by the selected leader smart NIC can be synchronized with at least one backup smart NIC. Additionally, when performing virtual network processing, smart NICs may need to store dynamic status information and share this data with each other. In many cases, the smart NIC operating system stores connection tracking information indicating open connections and congestion windows for each open connection. This connection tracking information is used by firewalls to determine whether to allow or drop / block data messages. If a smart NIC becomes inoperable and has not yet shared any state with other smart NICs, then all connections managed by that smart NIC will be transferred to other smart NICs, which will not have any records of those connections. Thus, smart NICs sharing this connection tracking status information with each other enables seamless failover between smart NICs.

[0015] This state sharing can also be used by intelligent NICs performing operations other than virtual networking (or intelligent NICs performing multiple types of operations that utilize state sharing). If storage virtualization operations are handled by intelligent NICs, then in some embodiments, the storage virtualization functionality includes running a network stack to manage Layer 4 connections to the storage device. In this case, in the event of a failover, connection information should be shared again among the intelligent NICs so that if one of the intelligent NICs fails, these connections will not be reset.

[0016] The foregoing summary is intended as a brief introduction to some embodiments of the present invention. It is not intended to be a description or overview of all inventive objects disclosed in this document. The following detailed description and the drawings referenced in the detailed description will further describe the embodiments described in the summary and other embodiments. Therefore, a comprehensive review of the summary, detailed description, and drawings is necessary to understand all the embodiments described in this document. Furthermore, the claimed object is not limited to the illustrative details in the summary, detailed description, and drawings, but will be defined by the appended claims, as the claimed object may be embodied in other specific forms without departing from the spirit of the object. Attached Figure Description

[0017] The novel features of the invention are set forth in the appended claims. However, for illustrative purposes, several embodiments of the invention are shown in the following figures.

[0018] Figure 1 A conceptual diagram illustrating a host computer with multiple physical intelligent NICs performing network virtualization operations.

[0019] Figure 2 The conceptual diagram illustrates the joint operation of a group of smart NICs on a single host computer in some embodiments.

[0020] Figure 3 The diagram illustrates some embodiments in which the host computer's smart NICs are connected in series.

[0021] Figure 4 The conceptual diagram illustrates some embodiments in which each smart NIC of the host computer is directly connected to each other smart NIC of the host computer.

[0022] Figure 5 This provides a conceptual illustration of an example of a smart NIC connected to a host computer via a separate physical switch.

[0023] Figure 6 The concept illustrates two host computers, each with three smart NICs connected to a data center network and using overlays on the data center network as their respective dedicated communication channels.

[0024] Figure 7 The conceptual diagram illustrates a host computer with three intelligent NICs that communicate via the host computer's PCIe bus.

[0025] Figure 8 The present invention provides a conceptual illustration of some embodiments of the process for processing data messages at a smart NIC, which is one of a plurality of smart NICs of a host computer.

[0026] Figure 9 The diagram illustrates the path of two data messages received at the smart NIC via the smart NIC's physical port.

[0027] Figure 10 The diagram illustrates the path of two data messages received from the VM via the VF to which the VM is bound at the smart NIC.

[0028] Figure 11 The diagram illustrates the path of data messages sent from a first VM operating on a host computer to a second VM.

[0029] Figure 12 The diagram illustrates the path of multicast data messages received at the first smart NIC of the host computer via the physical port of the smart NIC.

[0030] Figure 13 The conceptual diagram illustrates what happens after the physical port of the smart NIC has failed. Figure 10The path of the data messages of one of the connections shown in the image.

[0031] Figure 14 This document provides a conceptual illustration of the process for configuring multiple smart NICs to perform network virtualization operations in several embodiments.

[0032] Figure 15 The diagram illustrates that each of a pair of smart NICs executes a smart NIC operating system with a leader selection module.

[0033] Figure 16 The diagram illustrates how configuration data is distributed to multiple smart NICs.

[0034] Figure 17 The concept is illustrated by a selected leader smart NIC that collects statistics from itself and another smart NIC and reports the statistics to the network management and control system.

[0035] Figure 18 The concept is illustrated by three intelligent NICs of a host computer, each operating an intelligent NIC operating system with a leader selection module.

[0036] Figure 19 This concept illustrates two smart NICs sharing a connection state.

[0037] Figure 20 Diagram illustrating how to keep the connection open. Figure 19 The first smart NIC has become inoperable and the VM that was previously bound to the first smart NIC is now bound to the interface of the second smart NIC.

[0038] Figure 21 The electronic systems used in implementing some embodiments of the invention are illustrated conceptually. Detailed Implementation

[0039] In the following detailed description of the invention, numerous details, examples, and embodiments of the invention are set forth and described. However, it will be apparent to those skilled in the art that the invention is not limited to the stated embodiments and that the invention may be practiced without some of the specific details and examples discussed.

[0040] Some embodiments provide methods for enabling multiple smart NICs on the same host computer to operate as a single entity (e.g., as a group of smart NICs). In some embodiments, each smart NIC executes a smart NIC operating system that performs virtual network operations (and / or other operations, such as virtual storage operations) for a group of data compute nodes (e.g., virtual machines (VMs), containers, etc.) running on the host computer. In some embodiments, the smart NICs are connected via dedicated communication channels to share dynamic state information, share configuration data (so that one of the smart NICs can act as a single point of contact for network management and control systems), and / or transmit data messages to and from data compute nodes (DCNs) that require virtual network processing.

[0041] By running a smart NIC operating system, a smart NIC can perform various tasks that would otherwise be performed by the host computer software (e.g., the host computer's hypervisor). These tasks may include virtual network processing of data messages (i.e., performing virtual switching and / or routing, firewall operations, etc.), virtual storage operations, etc. To enable multiple smart NICs to perform these operations that would otherwise be performed entirely by a single entity (e.g., the hypervisor), communication between the smart NICs may be necessary.

[0042] Figure 1 A host computer 100 is conceptually illustrated with multiple physical intelligent NICs 105 and 110 performing network virtualization operations. As shown, the host computer 100 includes multiple DCNs (in this case, virtual machines) 115 to 125 connected to the intelligent NICs 105 and 110 in a pass-through mode (i.e., without applying any kind of network virtualization processing within the virtualization software 130 of the host computer 100). Each of the VMs 115 to 125 has an associated virtual NIC (vNIC) 135 to 145, which is connected to different virtual functions (VFs) 161 to 164 of one of the intelligent NICs 105 and 110 via a Fast Peripheral Component Interconnect (PCIe) structure 165 (a motherboard-level interconnect that connects the physical processor of the host computer 100 to the physical interface of the intelligent NICs 105 and 110).

[0043] Each vNIC 135 to 145 and therefore each VM 115 to 125 is bound to a different VF of one of the smart NICs 105 or 110. In some embodiments, VFs 161 to 164 are virtualized PCIe functions exposed as interfaces of the smart NIC. Each VF is associated with a Physical Function (PF), which is the physical interface of the smart NIC considered as a unique PCIe resource. In this case, smart NIC 105 has one PF 170 and smart NIC 110 has one PF 175, but in many cases, each smart NIC will have more than one PF. PF 170 is virtualized to provide at least VFs 161 to 162, while PF 175 is virtualized to provide at least VFs 163 to 164.

[0044] In some embodiments, a Virtual Function (VF) is provided to provide different virtual interfaces to different VMs, each of which can connect to a smart NIC. In some embodiments, VF drivers 150 to 160 execute in each of the VMs 115 to 125 to manage the respective VM-to-VF connections. As shown, in some embodiments, each VM 115 to 125 is associated with vNICs 135 to 145 provided by virtualization software 130 as a software emulation of the NIC. In different embodiments, VMs 115 to 125 access the VF either through their respective vNICs 135 to 145 or directly in pass-through mode (where virtualization software 130 does not participate in most network communications). In other embodiments, VMs 115 to 125 may switch between this pass-through mode and accessing the VF via their respective vNICs 135 to 145. In either case, virtualization software 130 is involved in assigning VFs 161 to 164 to VMs 115 to 125 and enabling the VFs to be accessed from VF drivers 150 to 160.

[0045] It should also be noted that although in this scenario all network virtualization operations have been moved from the virtualization software 130 on the host computer to the intelligent NICs 105 and 110, in other embodiments, the virtual switch provided by the virtualization software 130 can be directly connected to the PFs 170 and 175. In some such embodiments, data traffic is sent from the VM to the virtual switch via the vNIC, which then provides the traffic to the PF. In this scenario, the virtual switch performs basic switching operations, but leaves the network virtualization operations to the intelligent NIC.

[0046] Smart NICs 105 and 110 also include physical network ports 181 to 184. In different embodiments, each smart NIC may each include only a single physical network port or multiple physical network ports (e.g., 2, 3, 4, etc.). These physical network ports 181 to 184 provide physical communication between the host computer 100 and the data center network. Additionally, a dedicated communication channel 180 is presented between the two smart NICs 105 and 110, which allows these smart NICs to communicate. As further described below, this communication channel 180 may take various forms (e.g., a direct physical connection, a logical connection via an existing network, or a connection via PCIe messages).

[0047] at last, Figure 1 The diagram illustrates smart NICs 105 and 110 performing network virtualization operations 185. In some embodiments, these operations may include logical switching and / or routing operations, distributed firewall operations, encapsulation, and other network operations typically performed in the virtualization software of the host computer. In some embodiments, all smart NICs on a given host computer have the same virtual network configuration.

[0048] Although not shown in the figures, in some embodiments, each smart NIC is a NIC comprising (i) packet processing circuitry, such as an application-specific integrated circuit (ASIC), (ii) a general-purpose central processing unit (CPU), and (iii) memory. In some embodiments, the packet processing circuitry is an I / O ASIC that handles the processing of data messages forwarded to and from a DCN in a host computer and is at least partially controlled by the CPU. In other embodiments, the packet processing circuitry is a field-programmable gate array (FPGA) configured to perform packet processing operations or a firmware-programmable processing core specifically designed for network processing (which differs from a general-purpose CPU in that the processing core is specialized and therefore more efficient in packet processing). In some embodiments, the CPU executes a NIC operating system that controls the packet processing circuitry and may run other programs. In some embodiments, the CPU configures the packet processing circuitry to implement network virtualization operations by configuring the packet processing circuitry to process stream entries of data messages.

[0049] When a data message is sent by one of VMs 115 to 125, the data message (in the software of host computer 100) is transmitted via the corresponding vNIC 135 to 145. The data message is passed via PCIe bus 165 to the corresponding VF 161 to 164 of the appropriate smart NIC. The smart NIC ASIC processes the data message to apply the configured network virtualization operation 185, and then (provided the data message does not need to be sent to another smart NIC of the host computer and the destination of the data message is outside the host computer) sends the data message out from one of its physical ports 181 to 184.

[0050] It should be noted that, although Figure 1 The diagram illustrates a host computer with various virtualization software running on it, but the discussion of smart NICs in this article also applies to host computers that control other types of virtualization DCNs (e.g., containers) and bare-metal computing devices (i.e., computers that do not run virtualization software on said bare-metal computing devices). In the latter case, the bare-metal computing device will typically directly access the power fields (PFs) of multiple smart NICs, rather than any virtualization fields (VFs). That is, the smart NIC is used to provide network virtualization (or other operations, such as storage virtualization), and the software on the computing device is unaware of these operations.

[0051] Figure 2 The conceptual diagram illustrates the common operation of a set of smart NICs 205 to 210 on a single host computer in some embodiments. Each of these smart NICs includes multiple VMs for communicating with the host computer's VMs and multiple physical ports for communicating with a data center network (e.g., other host computers, which may or may not use smart NICs, are also connected to said multiple physical ports).

[0052] Each of the smart NICs runs (i.e., on the CPU of the respective smart NIC) a smart NIC operating system 215 to 220. Each smart NIC operating system 215 to 220 controls the smart NIC's ASIC and performs additional operations, such as network virtualization operation 225 and storage virtualization operation 230. These operations 225 and 230 (and other types of operations in other embodiments) are distributed across the various smart NICs 215 to 220 of the host computer, such that the smart NIC appears to operate as a single entity (i.e., in the same way that the virtualization software of the host computer is a single entity). As indicated above, network virtualization operation 225 includes performing logical switching and / or routing of data messages for one or more logical forwarding elements, applying distributed firewall rules, performing network address translation, and other network features. If each of the smart NICs 205 to 210 is configured to perform the same network virtualization operation, then any of the smart NICs can receive and correctly process data messages that are directed to or sent from one of the DCNs running on the host computer.

[0053] Similarly, if storage virtualization operation 230 is configured across all smart NICs, then the VM can be bound to any of the smart NICs and can handle I / O requests from the VM to the virtual storage network. The VM is bound to the smart NIC network adapter VF for network operations, and the VF to which the VM is bound for storage virtualization purposes is a storage VF (e.g., a Fast Non-Volatile Memory (NVMe) device or a Small Computer System Interface (SCSI) device).

[0054] In order for multiple smart NICs to perform these operations as if they were a single entity (similar to a hypervisor on a host computer), communication may be required between the smart NICs. Therefore, in some embodiments, a dedicated communication channel is established between the smart NICs to enable communication between them.

[0055] In some embodiments, a dedicated communication channel is a physically separate channel. For example, in some embodiments, smart NICs are connected via a set of physical cables that carry only communication between smart NICs. Figure 3The following are conceptual illustrations of some embodiments in which the smart NICs of the host computer 300 are connected in series. As shown, each of the smart NICs 305 to 320 is connected to the other two smart NICs. That is, smart NIC 305 is connected to smart NICs 320 and 310, smart NIC 310 is connected to smart NICs 305 and 315, smart NIC 315 is connected to smart NICs 310 and 320, and therefore smart NIC 320 is connected to smart NICs 315 and 305. Depending on the physical arrangement of the smart NICs, some embodiments do not connect the smart NICs directly at the end (i.e., in the example described, smart NICs 305 and 320 will not be connected).

[0056] With full ring connection (e.g.) Figure 3 (As shown in the diagram) This allows any of the smart NICs 305 to 320 to communicate with any of the other smart NICs even if one of the communication links (or one of the smart NICs themselves) fails. For example, if the link between smart NIC 310 and smart NIC 315 fails, smart NIC 310 can still reach smart NIC 315 via the other two smart NICs 305 and 320. Similarly, if smart NIC 310 itself fails, smart NIC 305 can still reach smart NIC 315 via smart NIC 320.

[0057] To achieve more robust fault protection, some embodiments include a dedicated communication channel link (i.e., full mesh connectivity) between each pair of smart NICs. Figure 4 The following is a conceptual illustration of some embodiments in which each smart NIC of host computer 400 is directly connected to each of the other smart NICs of host computer 405. As shown, each of smart NICs 405 to 420 is directly connected to each of the other three smart NICs 405 to 420. In this setup, if the host computer has N smart NICs, then each smart NIC needs to make N-1 direct connections to the other smart NICs. This setup is reasonable for host computers with a reasonably small number of smart NICs (e.g., 3 to 5 smart NICs), but becomes more difficult for larger numbers of smart NICs.

[0058] In some embodiments, these connections may use a separate, dedicated channel for inter-NIC communication. In other embodiments, if the smart NIC has sufficient physical ports, the connection may repurpose the NIC's physical network ports (e.g., using Ethernet cable—but if there are more than two smart NICs, this may require two network ports). Other embodiments utilize the smart NIC's management ports, if these ports are available and if the bandwidth of the management ports is high enough to handle the intended communication between the smart NICs. In some embodiments, the smart NIC component with the dedicated communication channel enabled is isolated from other components of the smart NIC. In this case, even if other smart NIC components are inoperable (e.g., due to firmware or software errors, hardware failures, etc.), the smart NIC can still relay traffic at least between smart NICs.

[0059] In other embodiments, these smart NICs are connected via separate physical switches, allowing each smart NIC to communicate directly with any other smart NIC through the physical switch, rather than having the smart NICs directly connected to each other (whether in serial or mesh connection). Figure 5 This conceptual illustration illustrates an example of a host computer 500 connected to intelligent NICs via a separate physical switch 505. As shown, each of the intelligent NICs 510 to 520 of the host computer 500 is connected to the isolated physical switch 505. In some embodiments, this physical switch 505 handles only communication between intelligent NICs (i.e., the physical switch is not part of a data center network that handles data messaging and / or management and control services between DCNs). In fact, this physical switch may not even use the same switching technology (e.g., Ethernet or Infiniband) used to carry network services within the data center. As with previous examples, these connections may use separate, specially constructed channels, physical network ports, or management ports. Additionally, for redundancy, some embodiments use two (or more) separate isolated switches, with each intelligent NIC 510 to 520 connected to each of these isolated switches.

[0060] In some embodiments, smart NICs communicate via logically dedicated communication channels that utilize existing physical connections, rather than using separate physical channels for dedicated communication between smart NICs (e.g., if no separate, specially constructed channel exists and network ports are unavailable for this purpose). For example, all smart NICs of a host computer are typically connected to the same physical data center network, thus allowing dedicated communication channels to be overlaid on that network.

[0061] Figure 6The diagram illustrates two host computers 605 and 610, each with three smart NICs connected to a data center network 600 and using overlays on the data center network as their respective dedicated communication channels. As shown, the first host computer 605 includes three smart NICs 615 to 625 connected to the data center network 600, while the second host computer 610 also includes three smart NICs 630 to 640 connected to the data center network 600.

[0062] Each of these corresponding groups of smart NICs uses a different overlay network (e.g., encapsulation) as a dedicated communication channel. The first group of smart NICs 615 to 625 uses a first overlay network 645, and the second group of smart NICs 630 to 640 uses a second overlay network 650. These overlay networks used as dedicated communication channels can be VXLAN networks, Geneve networks, etc. In some embodiments, the encapsulation network addresses used are those associated with the physical network ports of the smart NICs (i.e., the same network addresses used to encapsulate data traffic between DCNs on their respective host computers), while the base overlay network addresses are logical addresses associated with the smart NIC operating system (in fact, the first group of smart NICs 615 to 625 can use the same set of overlay network addresses as the second group of smart NICs 630 to 640).

[0063] The use of an overlay network only requires that all smart NICs of the host computer be attached to the same Layer 3 network (but not necessarily the same subnet). Therefore, if one smart NIC is connected only to a physically separate management network, but other smart NICs are connected to the data network within the data center (rather than to the management network), then the smart NICs cannot communicate via this overlay network. If all smart NICs of the host computer are connected to the same data link layer (Layer 2) network, some other embodiments use dedicated VLANs as private communication channels. However, if this existing physical Layer 2 network has numerous other host computers (each with its own group of smart NICs requiring separate VLANs) and also carries data messages for the DCN on these host computers, then the maximum number of VLANs available on a single network (4094) can be reached.

[0064] In other embodiments, the host computer's smart NIC communicates via a dedicated communication channel traversing the host computer. As described above, the smart NIC is typically connected to the host computer's PCIe subsystem, which can be used for the dedicated communication channel. Figure 7The conceptual diagram illustrates a host computer 700 with three intelligent NICs 705 to 715 that communicate via the host computer's PCIe architecture 720. Communication via the PCIe subsystem typically allows any intelligent NIC to speak directly with any of the other intelligent NICs. In various embodiments, the intelligent NICs utilize the standard point-to-point transmission features of PCIe, leverage the PCIe switching architecture, or use other enhancements on top of PCIe (e.g., compute fast link (CXL)).

[0065] As mentioned, one use of the dedicated communication channel is to enable a first smart NIC to pass data messages (e.g., data messages sent to or from a host computer or executed by a DCN on a host computer) to a second smart NIC. The smart NICs operate as a single entity because their respective smart NIC operating systems collectively implement a set of virtual network operations (e.g., implementing logical switches and / or routers, firewalls, etc.). However, each smart NIC has its own interface (e.g., physical and virtual functions) and its own physical network port, to which the host computer's DCN is bound.

[0066] Figure 8 The following is a conceptual illustration of process 800 for processing data messages at a smart NIC, which is one of multiple smart NICs on a host computer. As described above, each of the smart NICs has one or more interfaces, and each DCN operating on the host computer is bound to a different interface. Furthermore, the virtual network operations performed on the data messages are pushed to the smart NIC operating system (rather than being performed by a forwarding element running in the host computer's hypervisor). Reference will be made to... Figures 9 to 11 Partially describe process 800, Figures 9 to 11 This diagram illustrates an example of a data message being processed by a smart NIC.

[0067] As shown, process 800 begins with receiving a data message at the smart NIC (at 805). This data message can be received from the data center network via the smart NIC's physical port (e.g., as shown). Figure 9 (in the middle) or through the interface of a smart NIC bound to a host computer or one or more DCNs on the host computer (e.g., from a DCN running on the host computer) receives (e.g., as...) from the host computer. Figure 10 and 11(In Chinese). It should be understood that the terms data message, packet, data packet, or message are used herein to refer to various formatted sets of bits that can be sent between network endpoints (e.g., between DCNs in a host and / or across a physical network), such as Ethernet frames, IP packets, TCP segments, UDP datagrams, etc. Although the examples herein refer to data messages, packets, data packets, or messages, it should be understood that the invention is not limited to any particular format or type of data message.

[0068] Then, process 800 applies network virtualization operations (at 810) to the received data message based on the data message header. As described above, these operations may include logical switching (e.g., based on the logical destination MAC address of the data message), logical routing (e.g., based on the logical destination IP address of the data message), distributed firewall operations (e.g., based on the connection 5-tuple of the data message, including source and destination IP addresses, transport layer protocols, and source and destination transport layer ports), network address translation, encapsulation (if necessary), and other operations typically performed by the host computer's hypervisor. If the smart NICs jointly implement virtual network operations, the smart NIC that first receives the data message performs this processing. When the first smart NIC receives the data message from the second smart NIC via a dedicated communication channel, the second smart NIC will typically have already performed the necessary network virtualization operations (or most of these operations), and the first smart NIC can determine the destination of the data message with minimal additional processing.

[0069] Based on these network virtualization operations, the smart NIC is able to determine the destination of data messages. It should be understood that process 800 is a conceptual process and does not necessarily reflect specific operations performed by the smart NIC. For example, the smart NIC will typically only identify matching records of data messages (e.g., stream records) and perform the actions specified by said matching record, rather than performing a series of determinations regarding whether the destination is of a particular type (i.e., those shown in operations 815, 825, and 840). It should also be noted that this process does not cover the full range of data message processing options. For example, in some embodiments, the smart NIC may block and / or drop data messages due to firewall rules, congestion, etc.

[0070] Process 800 determines (at 815) whether the destination of the data message is a DCN bound to the current smart NIC (i.e., the smart NIC executing process 800). This could be for data messages received from an external network or from other DCNs on the host computer (which can be bound to any of the smart NICs). When the destination is this DCN bound to the current smart NIC, the process outputs from the smart NIC via the interface to which the destination DCN is bound (at 820). In some embodiments, the data message is then processed by the host computer (e.g., sent to the DCN via a vNIC or directly to a VF driver executing on the DCN, without additional network virtualization processing in the host computer's hypervisor).

[0071] When the destination of a data message is not the DCN bound to the current smart NIC, process 800 determines (at 825) whether the destination is the DCN of a different smart NIC bound to the host computer. This could be for data messages received from an external network or from other DCNs bound to the current smart NIC on the host computer. Furthermore, if the dedicated communication channel does not have direct communication between each pair of smart NICs, then the first smart NIC may receive data messages from the second smart NIC and need to send those data messages to a third smart NIC (e.g., in...). Figure 3 (As shown in the example). When the destination is this DCN bound to another smart NIC, process 800 sends a data message (at 830) to the other smart NIC (or an intermediate smart NIC, if the NICs are connected in series) via a dedicated communication channel between smart NICs.

[0072] Figure 9The diagram illustrates the paths of two data messages 910 and 915 received at a first smart NIC 900 via physical port 905 of the smart NIC. In this example, each of the smart NICs 900 and 920 on the host computer 925 has a single physical port. At least two VMs 930 and 935 run on the host computer 925, and for simplicity, only the VFs bound to these VMs are shown. The first smart NIC 900 provides the VF 940 to which the first VM 930 is bound (e.g., via its vNIC, not shown), while the second smart NIC 920 provides the VF 945 to which the second VM 935 is bound (also via its vNIC). Each of the smart NICs 900 and 920 performs network virtualization operation 950, and a dedicated communication channel 955 connects the two smart NICs. This dedicated communication channel can be any of the types described above (e.g., a separate physical channel, a VLAN or overlay network on the physical network to which the physical ports 905 and 960 of the smart NIC are connected, or a connection via the PCIe subsystem).

[0073] The intelligent NIC 900 performs a network virtualization operation 950 on each of data messages 910 and 915. Since the destination address of the first data message 910 is the address of VM1 930 bound to the intelligent NIC 900, the intelligent NIC 900 outputs data message 910 to VM 930 via VF 940. On the other hand, the network virtualization operation 950 applied to the second data message 915 identifies that the destination address of this data message 915 is the address of VM2 935 bound to the second intelligent NIC 920. Thus, the first intelligent NIC 900 transmits this data message 915 to the second intelligent NIC 920 via a dedicated communication channel 955. In some embodiments, the first intelligent NIC 900 also provides the second intelligent NIC 920 with contextual information about the processing of the data message by the network virtualization operation 950, so that this processing does not need to be completely repeated at the second intelligent NIC 920. In some embodiments, the second smart NIC 920 applies network virtualization operation 950 to evaluate this context and determine that data message 915 should be sent to VM2 935. Thus, the smart NIC 920 outputs data message 915 to VM 935 via VF 945.

[0074] Return to Figure 8If the destination of the data message is not a DCN on the host computer, then (assuming the data message will not be dropped or blocked) the destination is outside the host computer. Thus, process 800 identifies (at 835) the physical network output port used for the data message. In some cases, all ports of all smart NICs are grouped in a Link Aggregation Group (LAG) or other grouping mechanism. In this case, the connection used to bind to a single DCN for a particular smart NIC is load-balanced across all physical output ports of all smart NICs, and not just output by the smart NIC receiving the data message. In other cases, different smart NIC ports may have different connectivity, such that data messages for a specific destination need to be output from one smart NIC, and data messages for other destinations need to be output from another smart NIC (without considering any load balancing operations).

[0075] Thus, process 800 determines (at 840) whether the identified physical network output port is on another smart NIC or on the current smart NIC. If the output port for the data message is a port of another smart NIC, then process 800 sends the data message (at 830) to the other smart NIC (or an intermediate smart NIC, if the NICs are cascaded) via a dedicated communication channel between smart NICs. On the other hand, if the identified output port is a port of the current smart NIC, then process 800 outputs the data message (at 845) to the physical network via the identified output port. After outputting the data message to the DCN (via the interface of the current smart NIC), the physical network, or another smart NIC via the dedicated communication channel, process 800 ends.

[0076] Figure 10 The diagram conceptually illustrates the paths of two data messages 1005 and 1010 received from the first VM via the VF 940 to which the first intelligent NIC 900 is bound. As shown, the first data message 1005 is routed to a first destination (Dest1), while the second data message 1010 is routed to a second destination (Dest2). The intelligent NIC 900 processes these two data messages according to a configured network virtualization operation 950, which in this case (i) determines that both data messages should be output to the physical network and (ii) includes load balancing across multiple output ports (e.g., in a LAG). In some embodiments, only the first data message in the connection is subject to load balancing, while for subsequent operations, the cached result routes data messages to the same physical output port.

[0077] Based on these operations, the smart NIC 900 outputs the first data message 1005 to the physical network via its own physical port 905. However, the second data message 1010 is sent to the second smart NIC 920 via a dedicated communication channel 955. In some embodiments, the first smart NIC 900 also provides context information indicating that network virtualization operations have been performed on the data message 1010 and that the data message should be output via the physical port 960 of the second smart NIC 920. The second smart NIC 920 receives the second data message 1010 via the dedicated communication channel 955 and outputs this data message 1010 to the physical network via its physical port 960.

[0078] As referenced above Figure 8 As described, a dedicated communication channel is also used when a DCN bound to an interface of one smart NIC sends data messages to a DCN bound to an interface of another smart NIC. This dedicated communication channel allows data messages to be sent directly between smart NICs, rather than the first smart NIC outputting data messages onto the physical network for exchange and / or routing back to the host computer via a second smart NIC.

[0079] Figure 11 The conceptual diagram illustrates the path of data message 1100 sent from a first VM 930 operating on a host computer 925 to a second VM 935. Here, the first smart NIC 900 receives data message 1100 via a VF 940 to which the source VM 930 is bound. The smart NIC 900 applies a network virtualization operation 950 to data message 1100 to determine that the destination of the data message is a VM bound to the second smart NIC 920. Based on this determination, the smart NIC 900 sends data message 1100 to the second smart NIC 920 via a dedicated communication channel 955 between smart NICs. As in other instances, in some embodiments, the first smart NIC 900 also provides context information indicating that a network virtualization operation has been performed on data message 1100 and that the data message is directed to a DCN bound to the smart NIC 920 (although it is not necessary to indicate which interface the DCN is bound to). The second intelligent NIC 920 receives data message 1100 via dedicated communication channel 955 and outputs the data message to VM 935 via VF interface 945.

[0080] The process described above 800 and Figures 9 to 11 The examples shown relate to unicast data messages (i.e., data messages with a single destination). In some embodiments, a single data message (e.g., a broadcast or multicast data message) can be sent along multiple paths via network virtualization operations performed within a single smart NIC.

[0081] Figure 12 The path of a multicast data message 1200 received at a first smart NIC 900 via its physical port 905 is illustrated conceptually. In this example, the first smart NIC 900 applies a network virtualization operation 950 to the multicast data message 1200 and determines that both a first VM 930 and a second VM 935 are in the multicast group to which the data message 1200 is sent. Based on this, the smart NIC 900 (i) outputs a first copy of the multicast data message 1200 to the first VM 930 via a VF 940, and (ii) transmits a second copy of the data message 1200 to the second smart NIC 920 via a dedicated communication channel 955. In some embodiments, the first smart NIC 900 also provides the second smart NIC 920 with contextual information about the processing of the data message by the network virtualization operation 950, so that this processing does not need to be completely repeated at the second smart NIC 920. In some embodiments, the second smart NIC 920 applies network virtualization operation 950 to evaluate the context and determine that the multicast data message 1200 should be sent to the second VM 935. Thus, the smart NIC 920 outputs the data message 1200 to the VM 935 via VF 945. In some embodiments, if multiple destinations of the multicast data message are bound to the second smart NIC 920, then only one copy of the data message is transmitted via communication channel 955, thereby allowing the second smart NIC 920 to generate and output the necessary copies of the data message. Similarly, if one of the VMs attached to the first smart NIC sends a broadcast or multicast data message, then the receiving smart NIC can process the data message and generate any copies required to send the data message to other VMs attached to the first smart NIC, output the data message via its physical port, and / or transmit the data message to other smart NICs (sending the data message to VMs bound to those smart NICs, outputting the data message via its physical port, or a combination thereof).

[0082] If all physical network ports of a smart NIC become inoperable, but the smart NIC itself remains operational, another situation arises where a dedicated communication channel may be needed to transfer data messages between smart NICs. In this case, the smart NIC can still perform virtual network operations on data messages sent from the DCN bound to it, but those data messages will need to be sent to other smart NICs for output to the physical network, regardless of whether the port is operational in the LAG. When a port is operational in the LAG or the smart NIC is configured using another grouping mechanism in a NIC group, connections previously assigned to inoperable physical ports will be moved to another physical port (e.g., on another smart NIC).

[0083] Figure 13 The conceptual diagram illustrates the situation after the physical port 905 of the first intelligent NIC 900 has failed. Figure 10 The path of the data message in one of the connections shown is illustrated. This could be due to a problem with the smart NIC itself, a disconnection of the physical cable connecting port 905 to the data center network, etc. As shown, another data message 1300, directed to Dest1, is sent from VM 930 to smart NIC 900 via VF 940. Smart NIC 900 processes the data message according to the configured network virtualization operation 950, which determines that the data message should be output to the physical network, but the physical port 905 previously used for this connection is no longer available. Thus, the connection is now rebalanced to use another physical port 960 of the second smart NIC 920. Therefore, data message 1300 is sent to the second smart NIC 920 via dedicated communication channel 955. In some embodiments, the first smart NIC 900 also provides context information indicating that network virtualization operation has been performed on data message 1300 and that the data message should be output via physical port 960 of the second smart NIC 920. The second intelligent NIC 920 receives the second data message 1300 via a dedicated communication channel 955 and outputs the data message 1300 to the physical network via its physical port 960.

[0084] In many cases, the intelligent NIC receives configuration data for virtual network operations from a network management and control system. In some embodiments, this network management and control system receives data from a user (e.g., a network and / or security administrator) defining network operations (e.g., defining logical networks), security operations, etc., and uses this definition data to generate configuration data for various network elements (e.g., forwarding elements such as virtual switches and routers, middleware elements such as distributed firewalls, etc.), and provides the configuration data to the network elements so that the network elements can perform various network and security operations. Such network elements include intelligent NICs that perform network virtualization operations.

[0085] Each of the ports on different smart NICs (potentially including management ports) has its own network address, but many network management and control systems treat each host computer as a single entity. For example, for host computers that do not use smart NICs for network virtualization, some embodiments of the network management and control system communicate with an agent in the host computer's hypervisor. The network management and control system uses a single management network address for each host computer and therefore should not communicate directly with all the multiple smart NICs on the host computer.

[0086] In some embodiments, smart NICs use clustering technology to appear as a single entity of a host computer to the network management and control system. For example, in some embodiments, the host computer's smart NICs perform leader selection to determine the individual smart NICs that communicate with the network management and control system. In some such embodiments, each of the smart NICs' operating systems runs a deterministic algorithm that selects one of the smart NICs as the contact point. Any messages required for this leader selection are transmitted via a dedicated communication channel.

[0087] Figure 14 This conceptually illustrates process 1400 for configuring multiple smart NICs to perform network virtualization operations in some embodiments. In some embodiments, process 1400 is executed independently by each of the group of smart NICs on the host computer when a smart NIC is online (e.g., when the host computer starts up). Process 1400 may also be executed in response to a change in smart NIC group membership (e.g., a smart NIC being added to or removed from a NIC group, whether due to an external action or a NIC failure) (again, executed independently by each smart NIC on the host computer). In some embodiments, smart NICs use beacons or keep-alive messages (e.g., sent via a dedicated communication channel) to monitor group membership. References Figure 15 and 16 Partially describe process 1400, Figure 15 and 16 The diagram illustrates the operation of a pair of intelligent NICs 1500 and 1505 on a host computer (not shown).

[0088] Figure 15The diagram illustrates that each of the smart NICs 1500 and 1505 executes a smart NIC operating system 1510 and 1515, respectively. The smart NIC operating system includes multiple modules, such as network virtualization operations 1520 and 1525, control agents 1530 and 1535, and leader selection modules 1540 and 1545. A dedicated communication channel 1550 connects the two smart NICs and allows communication between them (e.g., for sending data messages, configuration data, etc.).

[0089] In some embodiments, control agents 1530 and 1535 communicate with a network management and control system that configures network virtualization operations on numerous host computers in a data center (e.g., by configuring these host computers to perform switching and / or routing to implement a logical network). Control agents 1530 and 1535 receive configuration data from this network management and control system and use the configuration data to appropriately configure their respective network virtualization operations 1520 and 1525. Control agents 1530 and 1535 are capable of communicating with each other via a dedicated communication channel 1550.

[0090] Leader selection modules 1540 and 1545 perform leader selection to assign one of the smart NICs as the leader for a specific task (e.g., communicating with a network management and control system). Leader selection modules 1540 and 1545 can communicate via a dedicated communication channel 1550 to confirm the leader selection for the task, share identification information, and ensure that each leader selection module is aware of all smart NICs in the host computer that can be selected as the task leader.

[0091] As shown, process 1400 begins by using (at 1405) a leader selection algorithm to determine which smart NIC is the single point of communication for the network management and control system. In some embodiments, this leader selection algorithm is a deterministic algorithm executed individually on each individual smart NIC in the host computer's smart NIC group. That is, if there are five smart NICs, then each of the five smart NICs runs the leader selection algorithm to arrive at the same selected leader. An example of this algorithm is a hash-based decision that hashes the identifiers of the five smart NICs and calculates the hash modulo 5 (the number of smart NICs) to determine the leader. In other embodiments, the leader selection algorithm involves communication and / or negotiation between smart NICs to determine the selected leader smart NIC designated to communicate with the network management and control system.

[0092] Once this selection is complete, process 1400 determines (at 1410) whether the currently selected intelligent NIC (i.e., the intelligent NIC executing this process) is chosen as the contact point. It should be understood that process 1400 is a conceptual process and each intelligent NIC may not necessarily make this specific determination. Instead, the intelligent NIC selected as the leader performs the first set of operations, while the other intelligent NICs perform a different set of operations after the leader is selected. Figure 16 In an example (which illustrates the distribution of configuration data to multiple smart NICs), the leader selection module 1540 is highlighted to indicate that smart NIC 1500 has been selected as the leader to communicate with the network management and control system 1600.

[0093] For smart NICs that are not selected contact points with the network management and control system, process 1400 ultimately receives configuration data from the selected smart NIC (at 1415) via a dedicated communication channel. It should be noted that this will not occur until the selected smart NIC receives this configuration data from the network management and control system and distributes the data to other smart NICs.

[0094] At the smart NIC selected as the contact point with the network management and control system, the process uses the management IP address assigned to the host computer to establish (at 1420) communication with the network management and control system. In some embodiments, the network management and control system treats each host computer as a single entity, which may not involve the internal network implementation on each host computer. To establish communication, in some embodiments, the selected smart NIC sends a message or a set of messages from the management IP address to the network management and control system. In some embodiments, the network management and control system will automatically use the assigned IP address, but the selected smart NIC needs to announce to the data center network that messages sent to the IP address should be routed to the specific port in its ports that uses the IP address.

[0095] Once communication is established, the process receives configuration data (at 1425) from the network management and control system. In some embodiments, this configuration data specifies how the smart NIC should handle data messages. The configuration data may include routing tables, virtual switch configurations, firewall rules, network address translation rules, load balancing rules, etc. In some embodiments, the configuration data is in a specific format for a particular type of network virtualization software running on the smart NIC's operating system. In other embodiments, the configuration data is in a generic format, and the controller agent on each smart NIC is responsible for converting the data into a format specific to the network virtualization software. Figure 16The diagram illustrates how the network management and control system 1600 provides configuration data 1605 to the control agent 1530 of the intelligent NIC 1500, which has been selected as the contact point for the network management and control system 1600.

[0096] Next, the process shares the received configuration data (at 1430) with other smart NICs (i.e., those smart NICs that do not communicate directly with the network management and control system). This data is provided to the other smart NICs via a dedicated communication channel between smart NICs. At the same time, the other smart NICs reach operation 1415 in their own processes because they are now able to receive configuration data.

[0097] Process 1400 (whether executed on the selected smart NIC or on one of the other smart NICs) then configures network virtualization operations on the smart NIC based on configuration data (at 1435). As mentioned, in some embodiments, the control agent uses configuration data received from the network management and control system (e.g., as a first set of data tuples) to generate configuration data (e.g., as a second set of data tuples) for the network virtualization operations. In some embodiments, the network virtualization operations in the smart NIC operating system and / or the control agent also program the smart NIC's data message processing ASIC based on this configuration data. Process 1400 then ends, but in practice, the selected smart NIC will periodically receive updates from the network management and control system when configuration changes are provided.

[0098] Figure 16 The control agent 1530 on the selected smart NIC 1500 provides configuration data 1605 to the control agent 1535 on the second smart NIC 1505 (e.g., via a dedicated communication channel 1550). The control agent 1530 also uses this configuration data 1605 to configure network virtualization operation 1520 on its smart NIC 1500, while the control agent 1535 on the second smart NIC 1505 uses the configuration data 1605 to configure its corresponding network virtualization operation 1525.

[0099] In addition to disseminating configuration data from the network management and control system, in some embodiments, the leader smart NIC also receives information from other smart NICs via a dedicated communication channel. In some embodiments, this information includes statistics (e.g., data message processing statistics), status / monitoring information, and other data. In some embodiments, the selected leader smart NIC performs various monitoring tasks based on this information (e.g., ensuring that various smart NICs are currently operational and sending messages to other smart NICs if one of the smart NICs fails).

[0100] In some embodiments, shared information is reported to the network management and control system. Figure 17 The diagram illustrates a selected leader smart NIC 1500 that collects statistical data from itself and another smart NIC 1505 and reports the statistical data to a network management and control system 1600. As shown, control agents 1530 and 1535 collect statistical data from their respective group's network virtualization operations 1520 and 1525. Control agent 1535 provides this statistical data to the leader smart NIC 1500 via a dedicated communication channel 1550. At least some of the statistical data from the two smart NICs is sent from control agent 1530 to the network management and control system 1600. In some embodiments, control agent 1530 or another module on the selected leader smart NIC 1500 aggregates the statistical data, such that the network management and control system 1600 is provided with information that appears to originate from a single entity.

[0101] The information collected can be used by the network management and control system 1600 to monitor host computers and / or individual smart NICs. The network management and control system can also use this information to modify the virtual network configuration of the smart NICs. In this scenario, the network management and control system provides configuration updates to a leader smart NIC, which then distributes these updates to other smart NICs via a dedicated communication channel.

[0102] In some embodiments, a network management and control system includes multiple components that perform different functions and provide different configuration data to host computers (in addition to receiving different data from host computers). For example, some embodiments of a network management and control system include both a management plane (MP) and a central control plane (CCP). The MP receives configuration data from an administrator, stores this data, and provides specific configuration information to host computers. Additionally, in some embodiments, host computers provide the MP with statistics, status, and other real-time data. In some embodiments, the CCP receives network configuration data from the MP, determines the host computers (and other forwarding elements, such as gateways) that require each part of the network configuration data, and provides this data to agents on those host computers.

[0103] In some embodiments, the smart NIC selects multiple different leaders for multiple different tasks. For example, some embodiments select one leader for receiving configuration data, another leader for collecting traffic statistics, a third leader for collecting monitoring data, etc. In some embodiments, one leader is selected to communicate with the MP and a second leader is selected to communicate with the CCP. These leader selections can use different hash functions or different inputs to the same hash function to derive different smart NICs as the selected leaders. In some embodiments, if a smart NIC is selected to communicate with the MP, then the smart NIC is not considered for communicating with the CCP to ensure that the load is shared.

[0104] Figure 18 The diagram conceptually illustrates three intelligent NICs 1805 to 1815 of a host computer (not shown), each operating an intelligent NIC operating system 1820 to 1830. As shown in the previous diagrams, each of the intelligent NIC operating systems 1820 to 1830 includes a corresponding control agent 1835 to 1845 and a leader selection module 1850 to 1860. Furthermore, each of the intelligent NICs 1805 to 1815 is connected to the other intelligent NICs via a dedicated communication channel 1865.

[0105] Additionally, the network management and control system 1800, which includes both MP 1870 and CCP 1875, communicates with smart NICs 1805 to 1815. Here, leader selection modules 1850 to 1860 have designated the first smart NIC 1805 as the contact point for MP 1870 and the third smart NIC 1815 as the contact point for CCP 1875. Thus, the control agent 1835 on the first smart NIC 1805 communicates with MP 1870, and the control agent 1840 on the third smart NIC 1815 communicates with CCP 1875. In some embodiments, each of the smart NIC operating systems actually runs separate MP agents and CP agents, wherein the selected MP agent communicates with MP 1870 and the selected CP agent communicates with CCP 1875.

[0106] For various purposes, in some embodiments, the smart NIC also uses a dedicated communication channel to synchronize dynamic state information. That is, when a first smart NIC receives or creates a set of dynamic state information, it uses the dedicated communication channel to provide the same set of dynamic state information to one or more other smart NICs. Different types of states can be shared with a single other smart NIC or multiple (or all) other smart NICs on a given host computer. If one of the smart NICs fails, the synchronization of the dynamic state information allows the information to be retained rather than lost. Smart NICs may fail due to electrical short circuits, disconnections, overheating, etc.

[0107] As mentioned, the selected leader smart NIC in the host computer's smart NIC group can collect monitoring data from all other smart NICs. This collected data, or data generated from the collected data, may contain dynamic status information synchronized with at least one backup smart NIC. Therefore, if the leader smart NIC fails, the next leader can retrieve the monitoring status information.

[0108] Additionally, when performing virtual network processing, smart NICs may need to store dynamic state information and share the data with each other. Figure 19 The diagram illustrates two smart NICs 1905 and 1910 sharing a connection state. As shown in the previous diagram, each of the smart NICs (e.g., within a smart NIC operating system) performs network virtualization operations 1915 and 1920. These operations include switching and routing 1925 and 1930, and firewall engines 1935 and 1940 performing firewall operations (i.e., determining whether to allow, block, or drop data messages based on their headers). In some embodiments, the firewall operations are stateful and therefore utilize information from the respective connection trackers 1945 and 1950.

[0109] Connection trackers 1945 and 1950 store information about open connections processed by the smart NIC. As shown, some embodiments store at least 5 tuples (source and destination IP addresses, source and destination transport layer ports, transport layer protocol), the current state of the connection, and the congestion window for each open connection. This connection information is a dynamic state synchronized by connection trackers 1945 and 1950 via a dedicated communication channel 1955 between smart NICs.

[0110] As shown, the connection tracker 1945 on the first smart NIC 1905 stores information for two open connections (cxn1 and cxn2), and congestion windows for these open connections. Other embodiments may also store additional data (e.g., receiver windows). Firewall engines 1935 and 1940 use this dynamic connection state information from their respective connection trackers to process data messages sent to and from the DCN on their host computers. Information about whether a particular connection has been opened (e.g., whether a three-way handshake has been completed) allows firewall engines 1935 and 1940 to determine whether data messages should be allowed. The congestion window is a dynamic state variable determined by the connection endpoints (and known to the smart NIC), which limits the amount of data that can be sent over the network (i.e., from the physical port of one of the smart NICs) for a particular connection, and typically starts small and increases to a maximum value (which can be set by the receiver window).

[0111] If the connection state of an ongoing connection is lost (e.g., due to the smart NIC failing to store the connection state in its connection tracker), then depending on the firewall engine settings, all traffic on that connection will be blocked by the firewall engine of the smart NIC that picked up the connection, or the firewall engine on that smart NIC will need to retrieve the connection state from the endpoint. In the first option, not only does the connection need to be re-established, but the congestion window will again start small, thus limiting the amount of data that can be transmitted. The latter option avoids disconnection, but at the cost of a looser window for security.

[0112] Thus, connection trackers 1945 and 1950 share their dynamic state information with each other to avoid needing either of these options. At this time, the state information of cxn1 and cxn2 has been shared; these connections can be handled by either smart NIC 1905 or 1910. Meanwhile, VM 1900 is in the process of opening a new connection (cxn3) and sending data messages 1960 for this connection to the first smart NIC 1905 (i.e., the smart NIC to which VM 1900 is bound) in a network virtualization operation 1915. Therefore, connection tracker 1945 also synchronizes this connection state data 1965 to connection tracker 1950. In some embodiments, each smart NIC synchronizes its connection state data (or other state data) only to one other smart NIC, while in other embodiments, each smart NIC synchronizes its connection state data (or other state data) to all other smart NICs.

[0113] Different embodiments synchronize dynamic state information at different intervals. Some embodiments synchronize each change via a dedicated communication channel, while others synchronize state data at regular time intervals (e.g., every 1 ms, every 100 ms, every second, every 5 seconds, etc.). If the dedicated communication channel is a specially constructed channel, this can achieve very fast synchronization (e.g., around every 1 ms). Additionally, some embodiments use mechanisms in the smart NIC to write connection state (or other synchronized data) to a specific storage area in the smart NIC, where this write is automatically mirrored to a peer storage area on another smart NIC, thereby achieving faster synchronization (e.g., latency less than 10 μs). If the synchronization interval is long (high latency) so that the congestion window cannot be accurately synchronized, some embodiments only synchronize basic connection state (i.e., whether the connection is open and allowed). In the event that the first smart NIC handling a particular connection fails, a new smart NIC that begins handling the connection is allowed to be used for the service of that connection until the new smart NIC has become aware of the congestion window of the connection.

[0114] When VM 1900 is bound to the first smart NIC 1905 (and assuming this connection is sent to and from the physical port of the first smart NIC 1905), the second smart NIC 1910 actually has no use for this information. However, Figure 20The diagram illustrates that while these connections remain open, the first smart NIC 1905 becomes inoperable, and therefore VM 1900 is now bound to the interface of the second smart NIC 1910. This does not mean that the VM needs to restart all its connections, as this information has been synchronized from the first smart NIC 1905. Depending on the configuration, if there are more than two smart NICs, in different embodiments, all VMs bound to the now inoperable smart NIC are either moved to the same smart NIC or balanced across all remaining smart NICs.

[0115] As shown, VM 1900 continues to send data messages 2000 for cxn3 (now sent to the second smart NIC 1910). Since the current state of this connection is that it is now open with congestion window 3 (before the failure of the first smart NIC 1905), firewall engine 1940 is able to process these data messages without requiring the connection or its congestion window to be restarted.

[0116] This state sharing can also be used by intelligent NICs performing operations other than virtual networking (or intelligent NICs performing multiple types of operations that utilize state sharing). If storage virtualization operations are handled by intelligent NICs, then in some embodiments, the storage virtualization functionality includes running a network stack to manage transport layer (e.g., TCP) connections to the storage device. In this case, in the event of a failover, connection information should be shared again among the intelligent NICs so that if one of the intelligent NICs fails, these connections will not be reset.

[0117] Figure 21 The electronic system 2100, conceptually illustrated, is used in some embodiments of the invention. The electronic system 2100 may be a computer (e.g., a desktop computer, personal computer, tablet computer, server computer, mainframe, blade computer, etc.), a telephone, a PDA, or any other type of electronic device. This electronic system includes various types of computer-readable media and interfaces for various other types of computer-readable media. The electronic system 2100 includes a bus 2105, a processing unit 2110, a system memory 2125, a read-only memory 2130, a permanent storage device 2135, an input device 2140, and an output device 2145.

[0118] Bus 2105 collectively represents all system, peripheral, and chipset buses, communicatively connecting numerous internal devices of electronic system 2100. For example, bus 2105 communicatively connects processing unit 2110 to read-only memory 2130, system memory 2125, and permanent storage device 2135.

[0119] From these various memory units, the processing unit 2110 retrieves instructions to be executed and data to be processed in order to perform the process of the present invention. In different embodiments, the processing unit may be a single-processor or a multi-core processor.

[0120] Read-only memory (ROM) 2130 stores static data and instructions required by processing unit 2110 and other modules of the electronic system. On the other hand, permanent storage device 2135 is a read-write memory device. This device is a non-volatile memory unit that stores instructions and data even when the electronic system 2100 is turned off. Some embodiments of the present invention use mass storage devices (e.g., magnetic disks or optical disks and their corresponding disk drives) as permanent storage device 2135.

[0121] Other embodiments use removable storage devices (e.g., floppy disks, flash drives, etc.) as permanent storage devices. Like permanent storage device 2135, system memory 2125 is a read-write memory device. However, unlike storage device 2135, system memory is volatile read-write memory, such as random access memory. System memory stores some instructions and data required by the processor during runtime. In some embodiments, the processes of the present invention are stored in system memory 2125, permanent storage device 2135, and / or read-only memory 2130. From these various memory units, processing unit 2110 retrieves instructions to be executed and data to be processed in order to perform the processes of some embodiments.

[0122] Bus 2105 is also connected to input device 2140 and output device 2145. The input device allows the user to transmit information and select commands to the electronic system. Input device 2140 includes an alphanumeric keypad and a pointing device (also called a "cursor control device"). Output device 2145 displays images generated by the electronic system. Output devices include printers and display devices, such as cathode ray tube (CRT) or liquid crystal displays (LCDs). Some embodiments include devices that function as both input and output devices, such as touchscreens.

[0123] Finally, as Figure 21 As shown, bus 2105 also connects electronic system 2100 to network 2165 via a network adapter (not shown). In this way, the computer can be part of a computer network (e.g., a local area network (“LAN”), a wide area network (“WAN”), or an intranet, or a network of several networks, such as the Internet). Any or all components of electronic system 2100 can be used in conjunction with this invention.

[0124] Some embodiments include electronic components, such as microprocessors, storage devices, and memories, that store computer program instructions in machine-readable or computer-readable media (alternatively referred to as computer-readable storage media, machine-readable media, or machine-readable storage media). Examples of such computer-readable media include RAM, ROM, read-only optical disc (CD-ROM), recordable optical disc (CD-R), rewritable optical disc (CD-RW), read-only digital versatile optical disc (e.g., DVD-ROM, dual-layer DVD-ROM), various recordable / rewritable DVDs (e.g., DVD-RAM, DVD-RW, DVD+RW, etc.), flash memory (e.g., SD card, mini SD card, micro SD card, etc.), magnetic and / or solid-state hard disk drives, read-only and recordable... Optical discs, high-density optical discs, any other optical or magnetic media, and floppy disks. Computer-readable media may store computer programs that can be executed by at least one processing unit and contain a set of instructions for performing various operations. Instances of computer programs or computer code include machine code (e.g., machine code generated by a compiler), and files containing higher-level code executed by a computer, electronic component, or microprocessor using an interpreter.

[0125] While the foregoing discussion primarily refers to microprocessors or multi-core processors that execute software, some embodiments are executed by one or more integrated circuits, such as application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions stored on the circuit itself.

[0126] As used in this specification, the terms "computer," "server," "processor," and "memory" refer to electronic or other technical devices. These terms do not include people or groups of people. For the purposes of this specification, the term "display" means display on an electronic device. As used in this specification, the terms "computer-readable media," "computer-readable media," and "machine-readable media" are entirely limited to tangible physical objects that store information in a computer-readable form. These terms do not include any wireless signals, wired download signals, or any other transient signals.

[0127] This specification refers throughout to computing and networking environments that include virtual machines (VMs). However, a virtual machine is only one instance of a Data Compute Node (DCN) or Data Compute End Node (also known as an addressable node). A DCN may contain non-virtualized physical hosts, virtual machines, containers running on top of a host operating system (without a hypervisor or separate operating system), and hypervisor kernel network interface modules.

[0128] In some embodiments, a VM operates on the host machine using its own guest operating system, virtualized by virtualization software (e.g., a hypervisor, virtual machine monitor, etc.). The tenant (i.e., the VM owner) can choose which applications run on the guest operating system. On the other hand, some containers are constructed to run on top of the host operating system without requiring a hypervisor or a separate guest operating system. In some embodiments, the host operating system uses namespaces to isolate containers from each other, thus providing operating system-level separation of different groups of applications operating within different containers. This separation is similar to the VM separation provided in a hypervisor virtualization environment of virtualization system hardware, and can therefore be considered a form of virtualization for isolating different groups of applications operating in different containers. Such containers are more lightweight than VMs.

[0129] In some embodiments, the hypervisor kernel network interface module is a non-VM DCN, which includes a network stack with a hypervisor kernel network interface and receive / transmit threads. An example of a hypervisor kernel network interface module is the vmknic module, which is used in VMware, Inc.'s ESXi. TM Part of the management process.

[0130] It should be understood that although this specification refers to VMs, the examples given can be any type of DCN, including physical hosts, VMs, non-VM containers, and hypervisor kernel network interface modules. In fact, in some embodiments, the instanced network may contain a combination of different types of DCNs.

[0131] Although the invention has been described with reference to numerous specific details, those skilled in the art will recognize that the invention may be embodied in other specific forms without departing from its spirit. Additionally, several figures (including...) Figure 8 and 14 The process is illustrated conceptually. Specific operations of these processes may not be performed in the exact order shown and described. Specific operations may not be performed within a series of consecutive operations, and different specific operations may be performed in different embodiments. Furthermore, the process may be implemented using several sub-processes or as part of a larger macro-process. Therefore, those skilled in the art will understand that the invention is not limited to the foregoing illustrative details but is defined by the appended claims.

Claims

1. A method for a smart network interface controller (NIC), comprising: at a first smart NIC of a plurality of smart NICs of a host computer, each of the smart NICs executing a smart NIC operating system that performs virtual network operations for a set of data computing machines executing on the host computer: receiving a data message sent by one of the data computing machines executing on the host computer, the data message comprising a first data message; performing a virtual network operation on the data message to determine that the data message is to be transmitted from a port of a second smart NIC of the plurality of smart NICs; passing the data message to the second smart NIC via a dedicated communication channel connecting the plurality of smart NICs; receiving a second data message from the second smart NIC via the dedicated communication channel, wherein the second smart NIC performed a virtual network operation on the second data message to determine that a destination of the second data message is the data computing machine; and sending the second data message to the data computing machine through a port of the first smart NIC to which the data computing machine is bound.

2. The method of claim 1, wherein performing a virtual network operation on the data message to determine that the data message is to be transmitted from a port of a second smart NIC comprises: determining, based on the virtual network operation, a destination of the data message that is external to the host computer and is reachable through a first physical port of the first smart NIC; and determining that the first physical port of the first smart NIC is currently inoperable and that the destination is reachable through a second physical port of the second smart NIC.

3. The method of claim 2, wherein all physical ports of the first smart NIC that connect the host computer to a physical network are inoperable.

4. The method of claim 2, wherein the data computing machine that sent the data message is bound to a port of the first smart NIC.

5. The method of claim 1, wherein performing a virtual network operation on the data message comprises: determining, based on the virtual network operation, a destination of the data message that is external to the host computer; and performing load balancing to assign the data message to the port of the second smart NIC.

6. The method of claim 5, wherein: a set of physical ports of the first smart NIC and a set of physical ports of the second smart NIC are in a same link aggregation group, load balancing data messages across the link aggregation group; and the data computing machine that sent the data message is bound to an interface of the first smart NIC.

7. The method of claim 1, wherein: the data computing machine is a first data computing machine bound to the first smart NIC; and the second smart NIC is a second data computing machine bound to the second smart NIC. performing a virtual network operation on the data message includes determining, based on the virtual network operation, that a destination of the data message is a second data computing machine executing on the host computer that is bound to the second smart NIC.

8. The method of claim 1, wherein the virtual network operation comprises a logical switching operation.

9. The method of claim 8, wherein the virtual network operation further comprises a logical routing operation.

10. The method of claim 1, the method further comprising: receiving a second data message sent by the data computing machine; performing a virtual network operation on the data message to determine that the data message is to be transmitted from a physical port of the first smart NIC; and transmitting the data message from the physical port.

11. A non-transitory machine-readable medium storing a program for execution by at least one processing unit of a first smart network interface controller (NIC) of a plurality of smart NICs of a host computer, each of the smart NICs executing a smart NIC operating system that performs virtual network operations for a set of data computing machines executing on the host computer, the program comprising a set of instructions for: receiving a data message sent by one of the data computing machines executing on the host computer; performing a virtual network operation on the data message to determine that the data message is to be transmitted from a port of a second smart NIC of the plurality of smart NICs; and communicating the data message to the second smart NIC via a dedicated communication channel connecting the plurality of smart NICs; wherein the set of instructions for performing a virtual network operation on the data message to determine that the data message is to be transmitted from a port of a second smart NIC comprises a set of instructions for: determining, based on the virtual network operation, a destination of the data message, the destination being external to the host computer and reachable through a first physical port of the first smart NIC; and determining that the first physical port of the first smart NIC is currently inoperable and that the destination is reachable through a second physical port of the second smart NIC.

12. The non-transitory machine-readable medium of claim 11, wherein all physical ports of the first smart NIC connecting the host computer to a physical network are inoperable.

13. The non-transitory machine-readable medium of claim 11, wherein the data computing machine sending the data message is bound to a port of the first smart NIC.

14. The non-transitory machine-readable medium of claim 11, wherein the set of instructions for performing a virtual network operation on the data message comprises a set of instructions for: determining, based on the virtual network operation, a destination of the data message that is external to the host computer; and performing load balancing to assign the data message to the port of the second smart NIC.

15. The non-transitory machine-readable medium of claim 14, wherein: a set of physical ports of the first smart NIC and a set of physical ports of the second smart NIC are in a same link aggregation group, load balancing data messages across the link aggregation group; and the data computing machine that sent the data message is bound to an interface of the first smart NIC.

16. The non-transitory machine-readable medium of claim 11, wherein: the data computing machine is a first data computing machine bound to the first smart NIC; and the set of instructions to perform a virtual network operation on the data message includes a set of instructions to determine, based on the virtual network operation, that a destination of the data message is a second data computing machine bound to the second smart NIC executing on the host computer.

17. The non-transitory machine-readable medium of claim 11, wherein the data message is a first data message, the program further comprising a set of instructions to: receive a second data message from the second smart NIC via the dedicated communication channel, wherein the second smart NIC performed a virtual network operation on the second data message to determine that a destination of the second data message is the data computing machine; and send the second data message to the data computing machine through a port of the first smart NIC to which the data computing machine is bound.

18. The non-transitory machine-readable medium of claim 17, wherein the data message is a first data message, the program further comprising a set of instructions to: receive a second data message sent by the data computing machine; perform a virtual network operation on the data message to determine that the data message is to be transmitted from a physical port of the first smart NIC; and transmit the data message from the physical port.

19. A method for a smart network interface controller (NIC), comprising: at a first smart NIC of a plurality of smart NICs of a host computer, each of the smart NICs to perform virtual network operations for a set of data computing machines executing on the host computer: determining to select the first smart NIC to communicate with a network management and control system that configures the virtual network operations, wherein the network management and control system includes a management plane and a control plane, wherein the determining to select the first smart NIC to communicate with the network management and control system includes determining to select the first smart NIC to communicate with the management plane; receiving a set of configuration data for the virtual network operations from the network management and control system, wherein the set of configuration data is received from the management plane; and providing the received set of configuration data to other smart NICs of the host computer.

20. The method of claim 19, wherein determining to select the first smart NIC to communicate with the network management and control system comprises: performing a deterministic selection algorithm to determine to select the first smart NIC.

21. The method of claim 20, wherein each of the other smart NICs of the plurality of smart NICs performs the same deterministic algorithm to determine to select the first smart NIC.

22. The method of claim 19, wherein determining to select the first smart NIC to communicate with the network management and control system comprises: exchanging messages with the other of the plurality of smart NICs via a dedicated communication channel connecting the smart NICs.

23. The method of claim 19, further comprising collecting run-time statistics from the other smart NICs.

24. The method of claim 19, further comprising determining to select a second smart NIC of the plurality of smart NICs to communicate with the control plane.

25. The method of claim 24, wherein: the set of configuration data is a first set of configuration data; and the second smart NIC receives a second set of configuration data from the control plane and provides the second set of configuration data to the first smart NIC and the other smart NICs of the host computer.

26. The method of claim 19, wherein the received set of configuration data is provided to the other smart NICs via a dedicated communication channel connecting the plurality of smart NICs.

27. The method of claim 19, further comprising monitoring the other smart NICs to determine whether the other smart NICs are operational.

28. The method of claim 19, wherein upon determining to select the first smart NIC to communicate with a network management and control system, a network address used by the network management and control system to communicate with the host computer is assigned to an interface of the first smart NIC.

29. A non-transitory machine-readable medium storing a program for execution by at least one processing unit of a first smart network interface controller (NIC) of a plurality of smart NICs of a host computer, each of the smart NICs to perform virtual network operations for a set of data computer machines executing on the host computer, the program comprising a set of instructions for: determining to select the first smart NIC to communicate with a network management and control system that configures the virtual network operations such that the first smart NIC is the only one of the plurality of smart NICs of the host computer that communicates with a particular component of the network management and control system, the particular component being a management plane of the network management and control system; receiving a set of configuration data for the virtual network operations from the network management and control system; and providing the received set of configuration data to other smart NICs of the host computer that do not communicate with the particular component of the network management and control system.

30. The non-transitory machine-readable medium of claim 29, wherein the program further comprises a set of instructions for collecting run-time statistics from the other smart NICs.

31. The non-transitory machine-readable medium of claim 29, wherein: the set of instructions for determining to select the first smart NIC to communicate with the network management and control system comprises a set of instructions for performing a deterministic selection algorithm to determine to select the first smart NIC; and the set of instructions for providing the received set of configuration data to the other smart NICs comprises a set of instructions for providing the received set of configuration data to the other smart NICs via a dedicated communication channel connecting the plurality of smart NICs. Each of the other smart NICs of the plurality of smart NICs performs the same deterministic algorithm to determine to select the first smart NIC and not to select any of the other smart NICs.

32. The non-transitory machine-readable medium of claim 29, wherein the set of instructions to determine to select the first smart NIC to communicate with the network management and control system comprises a set of instructions to exchange messages with the other smart NICs of the plurality of smart NICs via a dedicated communication channel connecting the smart NICs.

33. The non-transitory machine-readable medium of claim 29, wherein: the network management and control system comprises a control plane; the set of instructions to determine to select the first smart NIC to communicate with the network management and control system comprises a set of instructions to determine to select the first smart NIC to communicate with the management plane; and the set of configuration data is received from the management plane.

34. The non-transitory machine-readable medium of claim 33, wherein: the program further comprises a set of instructions to determine to select a second smart NIC of the plurality of smart NICs to communicate with the control plane such that the second smart NIC is the only one of the plurality of smart NICs of the host computer that communicates with the control plane; the set of configuration data is a first set of configuration data; and the second smart NIC receives a second set of configuration data from the control plane and provides the second set of configuration data to the first smart NIC and the other smart NICs of the host computer that do not communicate with the control plane.

35. The non-transitory machine-readable medium of claim 29, wherein the received set of configuration data is provided to the other smart NICs via a dedicated communication channel connecting the plurality of smart NICs.

36. The non-transitory machine-readable medium of claim 29, wherein the program further comprises a set of instructions to monitor the other smart NICs to determine whether the other smart NICs are operational.

37. The non-transitory machine-readable medium of claim 29, wherein upon determining to select the first smart NIC to communicate with the particular component of a network management and control system, a network address used by the network management and control system to communicate with the host computer is assigned to an interface of the first smart NIC.

Images