Packet flow in a cloud infrastructure based on cached and non-cached configuration information

Abstract

Techniques are described for managing distribution of configuration information supporting flow of packets in a cloud environment. In one example, a virtual network interface card (VNIC) hosted on a network virtualization device NVD receives a first packet from a compute instance associated with the VNIC. The VNIC determines that flow information for transmitting the first packet on a virtual network is unavailable from a memory of the NVD. The VNIC transmits the first packet to a network interface service via the NVD, the network interface service maintaining configuration information for transmitting the packet on a substrate network and configured to transmit the first packet on the substrate network based on the configuration information. The NVD receives the flow information from the network interface service, the flow information being a subset of the configuration information. The NVD stores the flow information in memory.

Description

[Technical Field]

[0001] REFERENCE TO RELATED APPLICATIONS This international patent application claims priority to U.S. patent application 17 / 237,745, entitled "PACKET FLOW IN A CLOUD INFRASTRUCTURE BASED ON CACHED AND NON-CACHED CONFIGURATION INFORMATION," filed on April 22, 2021, which claims the benefit of U.S. provisional application 63 / 149,276, filed on February 13, 2021, the entire contents of which are incorporated by reference for all purposes. [Background technology]

[0002] background Cloud infrastructures such as Oracle Cloud Infrastructure (OCI) can provide a set of cloud services that enable entities subscribing to the cloud services (e.g., enterprises) to build and run a wide range of applications and services in a highly available, cloud-hosted environment. The subscribing entities are called customers of the cloud service provider. The cloud infrastructure can provide high-performance compute, storage, and network capabilities in a flexible overlay virtual network that operates on a physical underlay network and is securely accessible from the enterprise's on-premises network. Cloud infrastructures such as OCI generally enable customers to manage their cloud-based workloads in the same way that customers manage their on-premises workloads. Thus, organizations can obtain all the benefits of the cloud with the same control, isolation, security, and predictable performance as the organization's on-premises network.

[0003] Virtual networking is the foundation of cloud infrastructure and applications because it enables the ability to access, connect to, secure, and modify cloud resources. Virtual networking enables communication between multiple computers, virtual machines (VMs), virtual servers, or other devices across different physical locations. While physical networking connects computer systems through cabling and other hardware, virtual networking uses software management to connect computers and servers in different physical locations over the Internet. Virtual networks use virtualized versions of traditional network components such as network switches, routers, and adapters, allowing for more efficient routing and easier network configuration and reconfiguration. Summary of the Invention [Means for solving the problem]

[0004] overview The present disclosure generally relates to managing the distribution of configuration information that supports the flow of packets between endpoints in a cloud environment. The configuration information generally includes policies, rules, mappings, routing, and other types of information for sending and / or receiving packets. The distribution can optimize the balance between memory usage and bandwidth. In particular, the configuration information may be stored by a network interface service that supports connectivity between compute instances in a virtual network in a centralized manner. A relevant subset of the configuration information (referred to herein as flow information) may also be stored locally on a network virtualization device that provides the compute instance's network interface service and interface to the virtual network. If flow information does not exist locally on the network virtualization device to support the packet flow to an endpoint, the packet may be sent to a network interface service that processes and sends it to the endpoint. Otherwise, the flow information is used in the network virtualization device to process and send the packet to the endpoint without sending the packet through the network interface service. In this way, rather than storing the entire set of configuration information on the network virtualization device (which would use a relatively large amount of memory and limit the interface density that the network virtualization device can support), only the relevant subset is stored and used locally for a packet flow, thereby saving memory space (which in turn allows for a relatively high interface density on the network virtualization device) and saving bandwidth (e.g., by not sending packets to endpoints through a network interface service).

[0005] The terms and expressions employed are used as terms of description rather than limitation, and there is no intention in the use of such terms and expressions to exclude any equivalents of the features shown and described or portions thereof. It is recognized, however, that various modifications are possible within the scope of the claimed system and method. Thus, while the present system and method have been specifically disclosed by example and optional features, it should be understood that modifications and variations of the concepts disclosed herein will be recognized by those skilled in the art, and that such modifications and variations are deemed to be within the scope of the system and method as defined by the claims.

[0006] This Summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to be used alone to determine the scope of the claimed subject matter, which subject matter should be understood by reference to the entire specification of this disclosure, any or all drawings, and appropriate portions of each claim.

[0007] The foregoing, together with other features and embodiments, will become more apparent with reference to the following specification, claims, and accompanying drawings.

[0008] BRIEF DESCRIPTION OF THE DRAWINGS Illustrative examples are described in detail below with reference to the following drawings: [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a high-level view of a distributed environment illustrating a virtual or overlay cloud network hosted by a cloud service provider infrastructure, according to an embodiment. [Figure 2] 1 illustrates a simplified architectural diagram of physical components in a physical network within CSPI, according to one embodiment. [Figure 3]1 illustrates an exemplary arrangement within CSPI in which a host machine is connected to multiple network virtualization devices (NVDs), according to one embodiment. [Figure 4] FIG. 1 illustrates a connection between a host machine and an NVD for providing I / O virtualization to support multi-tenancy, according to one embodiment. [Figure 5] 1 illustrates a simplified block diagram of a physical network provided by CSPI, according to one embodiment. [Figure 6] FIG. 1 illustrates an example network architecture that supports packet flows based on cached configuration information, according to an embodiment. [Figure 7] FIG. 1 illustrates an example network architecture that supports packet flows based on non-cached configuration information, according to an embodiment. [Figure 8] FIG. 1 illustrates an example network architecture that supports packet flows based on cached flow information and non-cached configuration information, according to an embodiment. [Figure 9] FIG. 1 illustrates an example of outbound packet flows that result in cached flow information, according to an embodiment. [Figure 10] FIG. 2 illustrates an example of an outbound packet flow based on cached flow information, according to an embodiment. [Figure 11] FIG. 1 illustrates an example of inbound packet flows that result in cached flow information, according to an embodiment. [Figure 12] FIG. 2 illustrates an example of an inbound packet flow based on cached flow information, according to an embodiment. [Figure 13] 10 illustrates an example of invalidating and updating cached flow information, according to an embodiment. [Figure 14] 10 illustrates another example of invalidating and updating cached flow information, according to an embodiment. [Figure 15]FIG. 1 illustrates an example method for transmitting packets that result in cached flow information, according to an embodiment. [Figure 16] FIG. 1 illustrates an example method for receiving packets that result in cached flow information, according to an embodiment. [Figure 17] FIG. 1 illustrates an example method for generating and transmitting flow information, according to an embodiment. [Figure 18] 1 illustrates an example method for invalidating and updating cached flow information, according to an embodiment. [Figure 19] 10 illustrates another example of a method for invalidating and updating cached flow information, according to an embodiment. [Figure 20] 10 illustrates another example of a method for invalidating and updating cached flow information, according to an embodiment. [Figure 21] FIG. 1 is a block diagram illustrating one pattern for realizing a cloud infrastructure as a service system, according to at least one embodiment. [Figure 22] FIG. 1 is a block diagram illustrating another pattern for realizing a cloud infrastructure as a service system, according to at least one embodiment. [Figure 23] FIG. 1 is a block diagram illustrating another pattern for realizing a cloud infrastructure as a service system, according to at least one embodiment. [Figure 24] FIG. 1 is a block diagram illustrating another pattern for realizing a cloud infrastructure as a service system, according to at least one embodiment. [Figure 25] FIG. 1 is a block diagram illustrating an exemplary computer system according to at least one embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Detailed Description Exemplary Virtual Networking Architecture The term cloud services generally refers to services that a cloud service provider (CSP) makes available to users or customers on demand (e.g., via a subscription model) using systems and infrastructure (cloud infrastructure). Typically, the servers and systems that make up the CSP's infrastructure are separate from the customer's own on-premise servers and systems. Therefore, customers can use cloud services provided by CSPs without separately purchasing hardware and software resources for the services. Cloud services are designed to provide subscribing customers with easy and scalable access to application and computing resources without requiring the customer to invest in procuring the infrastructure used to deliver the services.

[0011] There are several cloud service providers that offer different types of cloud services, including a variety of different types or models such as Software-as-a-Service (SaaS), Platform-as-a-Service (PaaS), and Infrastructure-as-a-Service (IaaS).

[0012] A customer can subscribe to one or more cloud services offered by a CSP. A customer can be any entity, such as an individual, an organization, or a business. When a customer subscribes or registers for a service offered by a CSP, a tenant or account is created for the customer. The customer can then access one or more subscribed cloud resources associated with the account through this account.

[0013] As mentioned above, Infrastructure as a Service (IaaS) is a specific type of cloud computing service. In the IaaS model, a CSP provides infrastructure (called Cloud Service Provider Infrastructure, or CSPI) that customers can use to build their own customizable networks and deploy customer resources. Therefore, customer resources and networks are hosted in a distributed environment by the infrastructure provided by the CSP. This differs from traditional computing, where the customer's infrastructure hosts the customer's resources and networks.

[0014] CSPI may comprise interconnected high-performance computing resources, including various host machines, memory resources, and network resources, that form a physical network, also known as a substrate network or underlay network. Resources in a CSPI may span one or more data centers, which may be geographically spread across one or more regions. Virtualization software may run on these physical resources to provide a virtualized, distributed environment. Virtualization creates an overlay network (also known as a software-based network, software-defined network, or virtual network) on the physical network. The CSPI physical network provides the foundation for creating one or more overlay or virtual networks on top of the physical network. The physical network (or substrate or underlay network) includes physical network devices such as physical switches, routers, computers, and host machines. An overlay network is a logical (or virtual) network that operates on the physical substrate network. A given physical network can support one or more overlay networks. Overlay networks typically use encapsulation techniques to distinguish between traffic belonging to different overlay networks. A virtual or overlay network is also called a virtual cloud network (VCN). Virtual networks are realized using software virtualization technologies (e.g., hypervisors, virtualization functions realized by network virtualization devices (NVDs) (e.g., smart NICs), top-of-rack (TOR) switches, smart TORs that realize one or more of the functions performed by NVDs, and other mechanisms) to create a layer of network abstraction that can run on top of a physical network. Virtual networks can take many forms, including peer-to-peer networks, IP networks, and others. Virtual networks are typically either Layer 3 IP networks or Layer 2 VLANs. This method of virtual or overlay networking is often referred to as virtual or overlay Layer 3 networking.Examples of protocols developed for virtual networks include IP-in-IP (or Generic Routing Encapsulation (GRE)), Virtual Extensible LAN (VXLAN - IETF RFC7348), Virtual Private Networks (VPNs) (e.g., MPLS Layer 3 Virtual Private Networks (RFC4364)), VMware's NSX, GENEVE (Generic Network Virtualization Encapsulation), etc.

[0015] In IaaS, the infrastructure provided by the CSP (CSPI) may be configured to provide virtualized computing resources over a public network (e.g., the Internet). In the IaaS model, a cloud computing service provider may host infrastructure elements (e.g., servers, storage devices, network nodes (e.g., hardware), deployment software, platform virtualization (e.g., hypervisor layer), etc.). In some cases, the IaaS provider may offer various services (e.g., billing, monitoring, logging, security, load balancing, clustering, etc.) associated with those infrastructure elements. Because these services are policy-driven, IaaS users can maintain application availability and performance by implementing policies to drive load balancing. The CSPI provides infrastructure and a set of complementary cloud services, enabling customers to build and run a wide range of applications and services in a highly available, hosted, distributed environment. The CSPI provides high-performance computing resources and power, as well as storage capacity, over a flexible virtual network that can be securely accessed from various network locations, such as customer on-premises networks. When a customer subscribes or registers for an IaaS service offered by a CSP, the tenancy created for that customer is a secure, isolated partition from the CSP where the customer can create, organize, and manage their cloud resources.

[0016] Customers can build their own virtual networks using the compute, memory, and networking resources provided by CSPI. They can deploy one or more customer resources or workloads, such as compute instances, on these virtual networks. For example, customers can build one or more customizable private virtual networks called virtual cloud networks (VCNs) using resources provided by CSPI. Customers can deploy one or more customer resources, such as compute instances, on the customer VCN. Compute instances may be virtual machines, bare metal instances, etc. Thus, CSPI provides infrastructure and a set of complementary cloud services that enable customers to build and run various applications and services in a highly available virtual host environment. Customers do not manage or control the underlying physical resources provided by CSPI, but they control the operating systems, storage, and deployed applications, and in some cases have limited control over some networking components (e.g., firewalls).

[0017] The CSP may provide a console that enables customers and network administrators to configure, access, and manage resources deployed in the cloud using CSPI resources. In certain embodiments, the console provides a web-based user interface that can be used to utilize and manage CSPI. In some embodiments, the console is a web-based application provided by the CSP.

[0018] CSPI can support single-tenancy or multi-tenancy architectures. In a single-tenancy architecture, software (e.g., applications, databases) or hardware elements (e.g., host machines or servers) serve a single customer or tenant. In a multi-tenancy architecture, software or hardware elements serve multiple customers or tenants. Thus, in a multi-tenancy architecture, CSPI resources are shared among multiple customers or tenants. In a multi-tenancy environment, CSPI employs precautions and safeguards to ensure that each tenant's data is isolated and not visible to other tenants.

[0019] In a physical network, a network endpoint (endpoint) refers to a computing device or system that is connected to the physical network and communicates bidirectionally with the connected network. A network endpoint of a physical network may be connected to a local area network (LAN), a wide area network (WAN), or other types of physical networks. Examples of traditional endpoints of a physical network include modems, hubs, bridges, switches, routers, and other networking devices, physical computers (or host machines), etc. Each physical device of a physical network has a fixed network address that can be used to communicate with the device. This fixed network address may be a Layer 2 address (e.g., a MAC address), a fixed Layer 3 address (e.g., an IP address), etc. In a virtualized environment or virtual network, endpoints may include various virtual endpoints, such as virtual machines hosted by elements of the physical network (e.g., hosted by a physical host machine). These endpoints of the virtual network are addressed by overlay addresses, such as overlay Layer 2 addresses (e.g., an overlay MAC address) and overlay Layer 3 addresses (e.g., an overlay IP address). Network overlays achieve flexibility by allowing network administrators to move overlay addresses associated with network endpoints using software management (e.g., via software implementing the virtual network's control plane). Thus, unlike physical networks, in virtual networks, overlay addresses (e.g., overlay IP addresses) can be moved from one endpoint to another using network management software. Because virtual networks are built on physical networks, both the virtual network and the underlying physical network are involved in communications between elements of the virtual network.To facilitate such communications, each element of the CSPI is configured to learn and store mappings that map overlay addresses in the virtual network to real physical addresses in the substrate network, or real physical addresses in the substrate network to overlay addresses in the virtual network. These mappings are used to facilitate communications. To facilitate virtual network routing, customer traffic is encapsulated.

[0020] Thus, a physical address (e.g., a physical IP address) is associated with a component in a physical network, and an overlay address (e.g., an overlay IP address) is associated with an entity in a virtual or overlay network. A physical IP address is an IP address associated with a physical device (e.g., a network device) in a substrate or physical network. For example, each NVD has an associated physical IP address. An overlay IP address is an overlay address associated with an entity in an overlay network, such as a compute instance in a customer's virtual cloud network (VCN). Two different customers or tenants, each with their own private VCN, can potentially use the same overlay IP address in their VCN without each other's knowledge. Both physical IP addresses and overlay IP addresses are types of real IP addresses. They are separate from virtual IP addresses. A virtual IP address is typically a single IP address that represents or maps to multiple real IP addresses. A virtual IP address provides a one-to-many mapping between a virtual IP address and multiple real IP addresses. For example, a load balancer may use a VIP to map to or represent multiple servers, each with its own actual IP address.

[0021] A cloud infrastructure or CSPI is physically hosted in one or more data centers in one or more regions around the world. The CSPI may include elements of a physical or substrate network and virtualized elements (e.g., virtual networks, compute instances, virtual machines) of a virtual network built on the physical network elements. In certain embodiments, the CSPI is organized and hosted in realms, regions, and availability domains. A region is typically a local geographic area that includes one or more data centers. Regions are generally independent of each other and may be separated by vast distances, for example, across countries or continents. For example, a first region may be in Australia, another region may be in Japan, and yet another region may be in India. CSPI resources are divided among these regions so that each region has an independent subset of CSPI resources. Each region can provide a set of core infrastructure services and resources, such as compute resources (e.g., bare metal servers, virtual machines, containers, and related infrastructure), storage resources (e.g., block volume storage, file storage, object storage, archive storage), networking resources (e.g., virtual cloud networks (VCNs), load balancing resources, connectivity to on-premises networks), database resources, edge networking resources (e.g., DNS), access management and monitoring resources, etc. Each region typically has multiple routes connecting it to other regions within the realm.

[0022] Typically, applications are deployed in the region where they will be used most (i.e., on infrastructure associated with that region) because using nearby resources is faster than using resources that are farther away. Applications may also be deployed in different regions for a variety of reasons, such as redundancy to mitigate the risk of region-wide events such as large weather systems or earthquakes, or to meet various requirements for legal jurisdictions, tax domains, and other business or societal criteria.

[0023] Datacenters within a region may be further organized and subdivided into availability domains (ADs). An availability domain may correspond to one or more datacenters located in a region. A region may consist of one or more availability domains. In such a distributed environment, CSPI resources may be region-specific, such as a virtual cloud network (VCN), or availability domain-specific, such as a compute instance.

[0024] ADs within a region are isolated from each other to be fault-tolerant, with the likelihood of simultaneous failures extremely low. This is achieved by configuring ADs so that they do not share critical infrastructure resources, such as networking, physical cables, cable routes, and cable entrances, so that a failure of one AD in a region rarely impacts the availability of other ADs in the same region. Connecting ADs within the same region to each other via low-latency, high-bandwidth networks provides highly available connections to other networks (e.g., the Internet, customer on-premises networks), allowing multiple ADs to be replicated for both high availability and disaster recovery. Crowdsense utilizes multiple ADs to ensure high availability and protect against resource failures. As the infrastructure provided by an IaaS provider grows, more regions and ADs may be added along with additional capacity. Traffic between available domains is typically encrypted.

[0025] In certain embodiments, regions are grouped into realms. A realm is a logical collection of regions. Realms are isolated from each other and do not share any data. Regions within the same realm can communicate with each other, but regions within different realms cannot. A CSP's customer tenancy or account exists in a single realm and can span one or more regions within that single realm. Typically, when a customer subscribes to an IaaS service, their tenancy or account is created in a customer-specified region (called their "home" region) within a realm. The customer can extend their tenancy to one or more other regions within the realm. The customer cannot access regions that do not exist within the realm in which the customer's tenancy resides.

[0026] An IaaS provider may offer multiple realms, each corresponding to a particular set of customers or users. For example, a commercial realm may be offered for commercial customers. As another example, a realm may be offered for a particular country or for customers in that country. As yet another example, a government realm may be offered, for example, for a government. For example, a government realm may be created for a particular government and may have a higher security level than a commercial realm. For example, Oracle® Cloud Infrastructure (OCI) currently offers a realm for the commercial domain and two realms for the government cloud domain (e.g., FedRAMP-authorized and IL5-authorized).

[0027] In certain embodiments, an AD can be subdivided into one or more fault domains. A fault domain is a grouping of infrastructure resources within an AD to provide anti-affinity. Fault domains can distribute compute instances so that compute instances are not located on the same physical hardware within an AD. This is known as anti-affinity. A fault domain refers to a collection of hardware elements (computers, switches, etc.) that share a single point of failure. A compute pool is logically divided into fault domains. Thus, a hardware failure or compute hardware maintenance event that affects one fault domain does not affect instances in other fault domains. Depending on the embodiment, the number of fault domains in each AD may vary. For example, in certain embodiments, each AD includes three fault domains. Fault domains function as logical data centers within an AD.

[0028] When a customer subscribes to an IaaS service, resources from CSPI are provisioned to the customer and associated with the customer's tenancy. Customers can use these provisioned resources to build private networks and deploy resources on these networks. A customer network hosted on the cloud by CSPI is called a virtual cloud network (VCN). Customers can configure one or more virtual cloud networks (VCNs) using the CSPI resources allocated for the customer. A VCN is a virtual or software-defined private network. Customer resources deployed in a customer's VCN can include compute instances (e.g., virtual machines, bare metal instances) and other resources. These compute instances may represent various customer workloads, such as applications, load balancers, and databases. Compute instances deployed on a VCN can communicate with publicly accessible endpoints (public endpoints) over a public network, such as the Internet; with other instances in the same VCN or other VCNs (e.g., other VCNs of the customer or VCNs not belonging to the customer); with customer on-premises data centers or networks; with sender endpoints; and with other types of endpoints.

[0029] CSPs can offer a variety of services using CSPI. In some cases, customers of a CSPI themselves can act as service providers and provide services using CSPI resources. Service providers can expose service endpoints characterized by identifying information (e.g., IP addresses, DNS names, and ports). Customer resources (e.g., compute instances) can consume a particular service by accessing the service endpoint for that particular service exposed by the service. These service endpoints are generally publicly accessible over a public communications network, such as the Internet, with users using the public IP address associated with the endpoint. Publicly accessible network endpoints are sometimes referred to as public endpoints.

[0030] In certain embodiments, a service provider may expose a service through a service endpoint (sometimes referred to as a service endpoint). Customers of the service may access the service using this service endpoint. In certain embodiments, a service endpoint provided for a service may be accessed by multiple customers wishing to consume the service. In other implementations, a dedicated service endpoint may be provided to a customer. Thus, only that customer may access the service using that dedicated sender endpoint.

[0031] In certain embodiments, when a VCN is created, it is associated with a private overlay Classless Inter-Domain Routing (CIDR) address space, which is a private overlay IP address range (e.g., 10.0 / 16) assigned to the VCN. A VCN includes associated subnets, route tables, and gateways. A VCN exists within a single region but can extend to one or more or all available domains in the region. A gateway is a virtual interface configured for a VCN that enables traffic communication between the VCN and one or more endpoints outside the VCN. One or more different types of gateways can be configured for a VCN to enable communication between different types of endpoints.

[0032] A VCN may be subdivided into one or more subnetworks, such as one or more subnets. A subnet is thus a building block or division that can be created within a VCN. A VCN can have one or more subnets. Each subnet within a VCN is associated with a contiguous range of overlay IP addresses (e.g., 10.0.0.0 / 24 and 10.0.1.0 / 24) that does not overlap with other subnets within the VCN and represents a subset of the VCN's address space.

[0033] Each compute instance is associated with a virtual network interface card (VNIC), which allows it to participate in a subnet of a VCN. A VNIC is a logical representation of a physical network interface card (NIC). Generally, a VNIC is an interface between an entity (e.g., a compute instance, a service) and a virtual network. A VNIC resides in a subnet and has one or more associated IP addresses and associated security rules or policies. A VNIC corresponds to a Layer 2 port on a switch. A VNIC connects a compute instance to a subnet within a VCN. A VNIC associated with a compute instance allows the compute instance to be part of a subnet of a VCN and enables the compute instance to communicate (e.g., send and receive packets) with endpoints on the same subnet as the compute instance, endpoints in a different subnet within the VCN, or endpoints outside the VCN. Thus, the VNIC associated with a compute instance determines how the compute instance connects with endpoints inside and outside the VCN. A VNIC for a compute instance is created and associated with the compute instance when the compute instance is created and added to a subnet within the VCN. If a subnet consists of a set of compute instances, it includes VNICs corresponding to the set of compute instances, each VNIC being connected to a compute instance in the set of compute instances.

[0034] Each compute instance is assigned a private overlay IP address via the VNIC associated with the compute instance. This private overlay IP address is assigned to the VNIC associated with the compute instance when the compute instance is created and is used to route traffic for the compute instance. All VNICs in a particular subnet use the same route table, security lists, and DHCP options. As described above, each subnet in a VCN is associated with a contiguous range of overlay IP addresses (e.g., 10.0.0.0 / 24 and 10.0.1.0 / 24) that do not overlap with other subnets in the VCN and represent an address space subset of the VCN's address space. For a VNIC on a particular subnet of a VCN, the overlay IP address assigned to the VNIC is an address from the contiguous range of overlay IP addresses assigned to the subnet.

[0035] In certain embodiments, if desired, a compute instance can be assigned additional overlay IP addresses in addition to the private overlay IP address, for example, one or more public IP addresses in the case of a public subnet. These multiple addresses are assigned to the same VNIC or multiple VNICs associated with the compute instance. However, each instance has a primary VNIC associated with the overlay private IP address that is created and assigned to the instance at instance launch. This primary VNIC cannot be deleted. Additional VNICs, called secondary VNICs, can be added to an existing instance within the same availability domain as the primary VNIC. All VNICs are in the same availability domain as the instance. Secondary VNICs can be in the same subnetwork in the same VCN as the primary VNIC, or in different subnetworks in the same or a different VCN.

[0036] Compute instances can optionally be assigned public IP addresses if they are in a public subnet. When creating a subnet, you can specify that the subnet is either a public or private subnet. A private subnet means that resources (e.g., compute instances) and associated VNICs within the subnet cannot have public overlay IP addresses. A public subnet means that resources and associated VNICs within the subnet can have public IP addresses. Customers can specify subnets that exist across a single available domain or multiple available domains within a region or realm.

[0037] As described above, a VCN may be subdivided into one or more subnets. In certain embodiments, a virtual router (referred to as a VCN VR or simply VR) configured for a VCN enables communication between subnets of the VCN. For a subnet within a VCN, the VR represents the logical gateway for that subnet, enabling communication between the subnet (i.e., the compute instances on that subnet) and endpoints on other subnets within the VCN and other endpoints outside the VCN. A VCN VR is a logical entity configured to route traffic between VNICs in a VCN and a virtual gateway (gateway) associated with the VCN. Gateways are further described below with respect to FIG. 1. A VCN VR is a Layer 3 / IP layer concept. In one embodiment, there is one VCN VR per VCN. This VCN VR potentially has an unlimited number of ports addressed by IP addresses, one port for each subnet of the VCN. In this way, a VCN VR has a different IP address for each subnet of the VCN to which the VCN VR is connected. VRs are also connected to various gateways configured for the VCN. In certain embodiments, a particular overlay IP address from a subnet's overlay IP address range is reserved in a port of the subnet's VCN VR. For example, consider a VCN with two subnets, each with associated address ranges 10.0 / 16 and 10.1 / 16. For a first subnet in the VCN with address range 10.0 / 16, addresses from this range are reserved in a port of the subnet's VCN VR. In some cases, a first IP address from this range may be reserved in a VCN VR. For example, for a subnet with overlay IP address range 10.0 / 16, IP address 10.0.0.1 may be reserved in a port of the subnet's VCN VR. For a second subnet in the same VCN with address range 10.1 / 16, the VCN VR may have a port of the second subnet with IP address 10.1.0.1.A VCN VR has a different IP address for each subnet in the VCN.

[0038] In some other embodiments, each subnet within a VCN may have its own associated VR that is addressable by the subnet using a reserved or default IP address associated with the VR. The reserved or default IP address may, for example, be the first IP address from a range of IP addresses associated with the subnet. VNICs within a subnet can use this default or reserved IP address to communicate (e.g., send and receive packets) with the VR associated with the subnet. In such embodiments, a VR is the ingress / egress point for that subnet. VRs associated with subnets within a VCN can communicate with other VRs associated with other subnets within the VCN. VRs can also communicate with gateways associated with the VCN. The VR functions for a subnet are performed on or by one or more NVDs that perform the VNIC functions for VNICs within the subnet.

[0039] Route tables, security rules, and DHCP options may be configured for a VCN. A route table is a virtual route table for a VCN and contains rules for routing traffic from subnets inside the VCN to destinations outside the VCN through gateways or specially configured instances. You can customize a VCN's route table to control the forwarding / routing of packets into and out of the VCN. DHCP options refer to configuration information automatically provided to instances when they are launched.

[0040] Security rules configured for a VCN represent the VCN's overlay firewall rules. Security rules can include inbound and outbound rules and can specify the type of traffic allowed in and out of instances in the VCN (e.g., based on protocol and port). Customers can choose whether certain rules are stateful or stateless. For example, a customer can allow incoming SSH traffic from anywhere to a set of instances by configuring a stateful inbound rule with source CIDR 0.0.0.0 / 0 and destination TCP port 22. Security rules may be implemented using network security groups or security lists. A network security group consists of a set of security rules that apply only to resources within that group. On the other hand, a security list contains rules that apply to all resources in subnets that use that security list. A VCN may also include default security rules and a default security list. DHCP options configured for a VCN provide configuration information that is automatically provided when instances in the VCN are launched.

[0041] In particular embodiments, configuration information for a VCN is determined and stored by a VCN control plane. The configuration information for a VCN may include, for example, address ranges associated with the VCN, subnets and associated information within the VCN, one or more VRs associated with the VCN, compute instances and associated VNICs within the VCN, NVDs that perform various virtualized network functions (e.g., VNICs, VRs, gateways) associated with the VCN, VCN state information, and other VCN-related information. In particular embodiments, a VCN distribution service publishes the configuration information stored by the VCN control plane, or portions thereof, to the NVD. The distributed information can be used to forward packets to and from compute instances within the VCN by updating information (e.g., forwarding tables, routing tables, etc.) stored and used by the NVD.

[0042] In certain embodiments, VCN and subnet creation is handled by a VCN control plane (CP), and compute instance launch is handled by the compute control plane. The compute control plane is configured to allocate physical resources for the compute instance and then invoke the VCN control plane to create and attach VNICs to the compute instance. The VCN CP also sends VCN data mappings to a VCN data plane configured to perform packet forwarding and routing functions. In certain embodiments, the VCN CP provides a distribution service configured to provide updates to the VCN data plane. Examples of VCN control planes are shown in Figures 21, 22, 23, and 24 (see reference numerals 2116, 2216, 2316, and 2416) and described below.

[0043] Customers can create one or more VCNs with resources hosted by CSPI. Compute instances deployed on a customer VCN can communicate with different endpoints. These endpoints can include endpoints hosted by CSPI and endpoints external to CSPL.

[0044] Various different architectures for implementing cloud-based services using CSPI are shown in Figures 1, 2, 3, 4, 5, 21, 22, 23, and 25 and described below. Figure 1 is a high-level diagram of a distributed environment 100 illustrating an overlay VCN or customer VCN hosted by CSPI, according to certain embodiments. The distributed environment shown in Figure 1 includes multiple elements in an overlay network. The distributed environment 100 shown in Figure 1 is merely an example and is not intended to unduly limit the scope of the claimed embodiments. Many variations, alternatives, and modifications are possible. For example, in some implementations, the distributed environment shown in Figure 1 may have more or fewer systems or elements than those shown in Figure 1, may combine two or more systems, or may have a different system configuration or arrangement.

[0045] As shown in the example of FIG. 1 , distributed environment 100 includes CSPI 101, which provides services and resources that customers can subscribe to and use to build a virtual cloud network (VCN). In a particular embodiment, CSPI 101 provides IaaS services to subscribing customers. Data centers within CSPI 101 may be organized into one or more regions. FIG. 1 shows an example region, "US Region" 102. A customer configures a customer VCN 104 for region 102. A customer can deploy various compute instances on VCN 104, which may include virtual machines or bare metal instances. Example instances include applications, databases, load balancers, etc.

[0046] In the embodiment shown in FIG. 1 , customer VCN 104 includes two subnets, "Subnet-1" and "Subnet-2," each with its own CIDR IP address range. In FIG. 1 , the overlay IP address range of Subnet-1 is 10.0 / 16, and the address range of Subnet-2 is 10.1 / 16. VCN virtual router 105 represents the logical gateway of the VCN, enabling communication between subnets of VCN 104 and with other endpoints outside the VCN. VCN VR 105 is configured to route traffic between VNICs in VCN 104 and the gateway associated with VCN 104. VCN VR 105 provides a port for each subnet of VCN 104. For example, VR 105 can provide a port with IP address 10.0.0.1 to Subnet-1 and a port with IP address 10.1.0.1 to Subnet-2.

[0047] Multiple compute instances can be deployed on each subnet. In this case, the compute instances may be virtual machine instances and / or bare metal instances. The compute instances within a subnet may be hosted by one or more host machines within CSPI 101. A compute instance joins a subnet through a VNIC associated with the compute instance. For example, as shown in FIG. 1, compute instance C1 is part of subnet-1 through a VNIC associated with the compute instance. Similarly, compute instance C2 is part of subnet-1 through a VNIC associated with C2. Similarly, multiple compute instances, which may be virtual machine instances or bare metal instances, may be part of subnet-1. Each compute instance is assigned a private overlay IP address and MAC address through the associated VNIC. For example, in FIG. 1, compute instance C1 has an overlay IP address of 10.0.0.2 and a MAC address of M1, and compute instance C2 has a private overlay IP address of 10.0.0.3 and a MAC address of M2. Each compute instance in Subnet-1, including compute instances C1 and C2, has a default route to VCN VR105 using IP address 10.0.0.1, which is the IP address of a port in VCN VR105 in Subnet-1.

[0048] Subnet-2 may have multiple compute instances deployed, including virtual machine instances and / or bare metal instances. For example, as shown in FIG. 1, compute instances D1 and D2 are part of subnet-2 via VNICs associated with the respective compute instances. In the embodiment shown in FIG. 1, compute instance D1 has an overlay IP address of 10.1.0.2 and a MAC address of MM1, and compute instance D2 has a private overlay IP address of 10.1.0.3 and a MAC address of MM2. Each compute instance in subnet-2, including compute instances D1 and D2, has a default route to VCN VR105 using IP address 10.1.0.1, which is the IP address of a port in VCN VR105 in subnet-2.

[0049] VCN A 104 may also include one or more load balancers. For example, a load balancer may be provided for a subnet and configured to load balance traffic among multiple compute instances on the subnet. A load balancer may also be provided to load balance traffic among subnets within the VCN.

[0050] A particular compute instance deployed on VCN 104 can communicate with a variety of different endpoints. These endpoints may include endpoints hosted by CSPI 200 and endpoints external to CSPI 200. Endpoints hosted by CSPI 101 may include endpoints on the same subnet as the particular compute instance (e.g., communication between two compute instances in Subnet-1), endpoints in a different subnet but within the same VCN (e.g., communication between a compute instance in Subnet-1 and a compute instance in Subnet-2), endpoints in a different VCN in the same region (e.g., communication between a compute instance in Subnet-1 and an endpoint in a VCN in the same region 106 or 110, or communication between a compute instance in Subnet-1 and an endpoint in the service network 110 in the same region), or endpoints in a VCN in a different region (e.g., communication between a compute instance in Subnet-1 and an endpoint in a VCN in a different region 108). Additionally, compute instances in a subnet hosted by CSPI 101 can communicate with endpoints not hosted by CSPI 101 (i.e., external to CSPI 101). These external endpoints include endpoints within customer on-premise networks 116, endpoints within other remote cloud host networks 118, public endpoints 114 accessible via public networks such as the Internet, and other endpoints.

[0051] Communication between compute instances on the same subnet is facilitated using VNICs associated with the source and destination compute instances. For example, compute instance C1 in Subnet-1 may want to send a packet to compute instance C2 in Subnet-1. For a packet sent from the source compute instance destined for another compute instance in the same subnet, the packet is first processed by the VNIC associated with the source compute instance. The processing performed by the VNIC associated with the source compute instance may include determining the packet's destination information from the packet header, identifying any policies (e.g., security lists) configured for the VNIC associated with the source compute instance, determining the packet's next hop, performing any packet encapsulation / decapsulation functions as needed, and forwarding / routing the packet to the next hop to facilitate communication of the packet to its intended destination. If the destination compute instance is in the same subnet as the source compute instance, the VNIC associated with the source compute instance is configured to identify the VNIC associated with the destination compute instance and forward the packet for processing to that VNIC. The VNIC associated with the destination compute instance is then executed and forwards the packet to the destination compute instance.

[0052] When communicating a packet from a compute instance in a subnet to an endpoint in a different subnet of the same VCN, the communication is facilitated by the VNICs and VCN VRs associated with the source and destination compute instances. For example, if compute instance C1 in Subnet-1 in Figure 1 wants to send a packet to compute instance D1 in Subnet-2, the packet is first processed by the VNIC associated with compute instance C1. The VNIC associated with compute instance C1 is configured to route the packet to VCN VR105 using the VCN VR's default route or port 10.0.0.1. VCN VR105 is configured to route the packet to Subnet-2 using port 10.1.0.1. The packet is then received and processed by the VNIC associated with D1, which forwards the packet to compute instance D1.

[0053] To communicate a packet from a compute instance within VCN 104 to an endpoint outside VCN 104, the communication is facilitated by a VNIC associated with the source compute instance, VCN VR 105, and a gateway associated with VCN 104. One or more types of gateways can be associated with VCN 104. A gateway is an interface between a VCN and another endpoint, where the other endpoint is outside the VCN. A gateway is a Layer 3 / IP layer concept that enables a VCN to communicate with endpoints outside the VCN. Thus, a gateway facilitates traffic flow between a VCN and other VCNs or networks. A variety of different types of gateways can be configured in a VCN to facilitate different types of communications with different types of endpoints. Through gateways, communications may occur over a public network (e.g., the Internet) or a private network. These communications may use various communication protocols.

[0054] For example, compute instance C1 may wish to communicate with an endpoint outside VCN 104. The packet may first be processed by a VNIC associated with the source compute instance C1. The VNIC processing determines that the packet's destination is outside of Cl's subnet-1. The VNIC associated with C1 may forward the packet to VCN VR105 of VCN 104. VCN VR105 then processes the packet and, as part of the processing, determines a particular gateway associated with VCN 104 as the packet's next hop based on the packet's destination. VCN VR105 may then forward the packet to the particular gateway. For example, if the destination is an endpoint within a customer's operating premises network, the packet may be forwarded by VCN VR105 to dynamic routing gateway (DRG) 122 configured for VCN 104. The packet may then be forwarded from the gateway to the next hop to facilitate communication of the packet to its intended final destination.

[0055] Various different types of gateways may be configured for a VCN. Examples of gateways that may be configured for a VCN are shown in FIG. 1 and described below. Examples of gateways associated with VCNs are also shown in FIGS. 21, 22, 23, and 24 (e.g., gateways indicated by reference numbers 2134, 2136, 2138, 2234, 2236, 2238, 2334, 2336, 2338, 2434, 2436, and 2438) and described below. As shown in the embodiment shown in FIG. 1, a dynamic routing gateway (DRG) 122 may be added to or associated with the customer VCN 104. The DRG 122 provides a path for private network traffic communication between the customer VCN 104 and another endpoint. The other endpoint may be the customer on-premises network 116, a VCN 108 in a different region of the CSPI 101, or another remote cloud network 118 not hosted by the CSPI 101. The customer on-premises network 116 may be a customer network or customer data center built using customer resources. Access to the customer on-premises network 116 is typically highly restricted. For a customer that has both the customer on-premises network 116 and one or more VCNs 104 deployed or hosted in the cloud by CSPI 101, the customer may want the on-premises network 116 and the cloud-based VCNs 104 to be able to communicate with each other. This allows the customer to build an extended hybrid environment that includes the on-premises network 116 and the customer's VCNs 104 hosted by CSPI 101. The DRG 122 enables such communication. To enable such communication, a communication channel 124 is established. In this case, one endpoint of the communication channel is located in the customer on-premises network 116, and the other endpoint is located in CSPI 101 and connected to the customer VCN 104. The communication channel 124 can traverse a public communication network, such as the Internet, or a private communication network.A variety of different communication protocols can be used, such as IPsec VPN technology over a public communication network such as the Internet, or Oracle's FastConnect technology, which uses a private network instead of a public network. The device or equipment in the customer on-premises network 116 that forms one endpoint of the communication channel 124 is called customer premises equipment (CPE), such as CPE 126 shown in Figure 1. The endpoint on the CSPI 101 side may be a host machine running DRG 122.

[0056] In certain embodiments, remote peering connections (RPCs) can be added to a DRG, allowing customers to peer one VCN with another VCN in another region. Using such RPCs, a customer VCN 104 can connect to a VCN 108 in another region using a DRG 122. The DRG 122 may also be used to communicate with other remote cloud networks 118 not hosted by CSPI 101, such as the Microsoft® Azure cloud or the Amazon® AWS cloud.

[0057] As shown in Figure 1, an Internet Gateway (IGW) 120 can be configured in a customer VCN 104 to enable compute instances on the customer VCN 104 to communicate with public endpoints 114 accessible over a public network, such as the Internet. The IGW 120 is a gateway for connecting a VCN to a public network, such as the Internet. The IGW 120 enables public subnets in a VCN, such as VCN 104 (resources in the public subnet have public overlay IP addresses) to directly access public endpoints 112 on the public network 114, such as the Internet. The IGW 120 can be used to initiate connections from subnets in the VCN 104 or from the Internet.

[0058] Customer VCN 104 can be configured with a network address translation (NAT) gateway 128. NAT gateway 128 allows cloud resources in the customer VCN that do not have dedicated public overlay IP addresses to access the Internet without exposing them to direct incoming Internet connections (e.g., L4-L7 connections). This allows private subnets in a VCN, such as private subnet-1 in VCN 104, to privately access public endpoints on the Internet. With a NAT gateway, connections can be initiated from the private subnet to the public Internet, but connections cannot be initiated from the Internet to the private subnet.

[0059] In certain embodiments, a service gateway (SGW) 126 can be configured in customer VCN 104. SGW 126 provides a pathway for private network traffic between VCN 104 and service endpoints supported by service network 110. In certain embodiments, service network 110 may be provided by a CSP and may offer a variety of services. An example of such a service network is the Oracle® Service Network, which offers a variety of services available to customers. For example, a compute instance (e.g., a database system) in a private subnet of customer VCN 104 can back up data to a service endpoint (e.g., an object store) without requiring a public IP address or access to the Internet. In some embodiments, a VCN can have only one SGW, and connections can be initiated only from subnets within the VCN, not from service network 110. When a VCN is peered with another VCN, resources in the other VCN typically cannot access the SGW. Resources in an on-premises network connected to a VCN with FastConnect or VPN Connect can also use a service gateway configured in that VCN.

[0060] In some implementations, the SGW 126 uses service classless inter-domain routing (CIDR) labels. A CIDR label is a string that represents all regional public IP address ranges for a service or group of services of interest. Customers use service CIDR labels to control traffic to services when configuring the SGW and associated routing rules. Customers can optionally use service CIDR labels when configuring security rules without having to adjust the security rules if the service's public IP addresses change in the future.

[0061] A local peering gateway (LPG) 132 is a gateway that can be added to a customer VCN 104 to enable the VCN 104 to peer with another VCN in the same region. Peering means that the VCNs communicate using private IP addresses without the traffic going over a public network such as the Internet or routing the traffic through the customer on-premises network 116. In a preferred embodiment, a VCN has a separate LPG for each peering it establishes. Local peering or VCN peering is a common practice used to establish network connectivity between different applications or infrastructure management functions.

[0062] A service provider, such as a provider of a service in service network 110, can provide access to a service using different access models. According to a public access model, the service may be exposed as a public endpoint publicly accessible by a compute instance in the customer VCN over a public network such as the Internet, or may be accessed privately through SGW 126. According to a specific private access model, the service may be accessed as a private IP endpoint in a private subnet in the customer VCN. This is called private endpoint (PE) access and allows a service provider to expose its service as an instance in the customer's private network. A private endpoint resource represents a service in a customer VCN. Each PE appears as a VNIC (called a PE-VNIC, which has one or more private IPs) that the customer selects from a subnet in the customer VCN. Thus, the PE provides a way to provide a service within the customer's private VCN subnet using a VNIC. Because the endpoint is exposed as a VNIC, the PE VNIC can utilize all the functionality associated with a VNIC, such as routing rules and security lists.

[0063] Service providers register services to make them accessible through PEs. Providers can associate policies with services that regulate the visibility of the service to customer tenants. Providers can register multiple services under a single virtual IP address (VIP), especially for multi-tenant services. There can also be multiple private endpoints (in multiple VCNs) that represent the same service.

[0064] Compute instances in the private subnet can then access the service using the private IP address or service DNS name of the PE VNIC. Compute instances in the customer VCN can access the service by sending traffic to the private IP address of the PE in the customer VCN. The Private Access Gateway (PAGW) 130 is a gateway resource that can connect to a service provider VCN (e.g., a VCN in the service network 110) and serves as the ingress / egress point for all traffic from / to the customer subnet private endpoints. The PAGW 130 allows providers to scale the number of PE connections without utilizing internal IP address resources. A provider only needs to configure one PAGW for any number of services registered in a single VCN. A provider can present services as private endpoints in multiple VCNs for one or more customers. From the customer's perspective, the PE VNIC appears not to be connected to the customer's instance but to the service the customer wants to interact with. Traffic destined for the private endpoint is routed to the service through the PAGW 130. These are called customer-to-service private connections (C2S connections).

[0065] The PE concept can also be used to extend private access of services to customer on-premises networks and data centers by allowing traffic to flow through FastConnect / IPsec links and private endpoints in the customer VCN, and to extend private access of services to customer peering VCNs by allowing traffic to flow between LPG132 and PEs in the customer VCN.

[0066] Customers can control VCN routing at the subnet level, allowing them to specify which subnets in a customer VCN, such as VCN 104, use each gateway. A VCN's route tables can be used to determine whether traffic can be routed outside the VCN through a particular gateway. For example, in a particular case, the route table for a public subnet in customer VCN 104 can send non-local traffic through IGW 120. The route table for a private subnet in the same customer VCN 104 can send traffic to CSP services through SGW 126. All remaining traffic may be sent through NAT gateway 128. Route tables only control traffic that leaves the VCN.

[0067] Security lists associated with a VCN are used to control traffic entering the VCN through inbound connections and gateways. All resources within a subnet use the same mute tables and security lists. Security lists may be used to control specific types of traffic entering and leaving instances within a VCN's subnets. Security list rules may include inbound (inbound) rules and outbound (outbound) rules. For example, inbound rules may specify allowed source address ranges, and outbound rules may specify allowed destination address ranges. Security rules may specify specific protocols (e.g., TCP, ICMP), specific ports (e.g., port 22 for SSH, port 3389 for Windows RDP), etc. In certain implementations, the instance's operating system may enforce its own firewall rules that match security list rules. Rules may be stateful (e.g., connections are tracked and responses are automatically allowed without explicit security list rules for the response traffic) or stateless.

[0068] Access from a customer VCN (i.e., resources or compute instances deployed on VCN 104) can be categorized as public access, private access, or dedicated access. Public access refers to an access model for accessing public endpoints using public IP addresses or NATs. Private access enables customer workloads in VCN 104 with private IP addresses (e.g., resources in a private subnet) to access services without traversing a public network such as the Internet. In particular embodiments, CSPI 101 enables customer VCN workloads with private IP addresses to access public service endpoints of services using a service gateway. Thus, the service gateway provides a private access model by establishing a virtual link between the customer VCN and the public endpoints of services that reside outside the customer private network.

[0069] Additionally, CSPI can offer dedicated public access using technologies such as FastConnect public peering. In this case, customer on-premises instances can access one or more services in the customer VCN using a FastConnect connection without traversing a public network such as the Internet. CSPI can also offer dedicated private access using FastConnect private peering. In this case, customer on-premises instances with private IP addresses can access customer VCN workloads using a FastConnect connection. FastConnect is a network connection used instead of using the public Internet to connect a customer on-premises network to CSPI and its services. FastConnect provides an easy, flexible, and economical way to create dedicated private connections with higher bandwidth options and a more reliable and consistent networking experience compared to Internet-based connections.

[0070] FIG. 1 and the accompanying description above illustrate various virtualized elements in an exemplary virtual network. As noted above, a virtual network is built on an underlying physical or substrate network. FIG. 2 is a simplified architecture diagram illustrating physical elements within a physical network within CSPI 200 that provides the foundation for the virtual network, according to certain embodiments. As shown, CSPI 200 provides a distributed environment including elements and resources (e.g., compute, memory, and networking resources) provided by a cloud service provider (CSP). These elements and resources are used to provide cloud services (e.g., IaaS services) to subscribing customers, i.e., customers who subscribe to one or more services offered by the CSP. Based on the services to which the customer subscribes, CSPI 200 provides some resources (e.g., compute, memory, and networking resources) to the customer. The customer can then build their own cloud-based (i.e., CSPI-hosted), customizable private virtual network using the physical compute, memory, and networking resources provided by CSPI 200. As noted above, these customer networks are referred to as virtual cloud networks (VCNs). Customers can deploy one or more customer resources, such as compute instances, into these Customer VCNs. The compute instances may be virtual machines, bare metal instances, etc. CSPI200 provides infrastructure and a set of complementary cloud services that enable customers to build and run a wide range of applications and services in a highly available hosted environment.

[0071] In the exemplary embodiment shown in FIG. 2, the physical elements of CSPI 200 include one or more physical host machines or physical servers (e.g., 202, 206, 208), network virtualization devices (NVDs) (e.g., 210, 212), top-of-rack (TOR) switches (e.g., 214, 216), a physical network (e.g., 218), and switches within physical network 218. The physical host machines or servers can host and execute various compute instances participating in one or more subnets of a VCN. The compute instances may include virtual machine instances and bare metal instances. For example, the various compute instances shown in FIG. 1 may be hosted by the physical host machines shown in FIG. 2. The virtual machine compute instances in a VCN may be executed by one host machine or by multiple different host machines. Additionally, the physical host machines may host virtual host machines, container-based hosts or functions, etc. The VICs and VCN VRs shown in FIG. 1 may be executed by the FTVDs shown in FIG. 2. The gateway shown in FIG. 1 may be implemented by the host machine and / or NVD shown in FIG.

[0072] A host machine or server may run a hypervisor (also called a virtual machine monitor or VMM) that creates and enables a virtualized environment on the host machine. Virtualization or a virtualized environment facilitates cloud-based computing. One or more computing instances may be created, executed, and managed on the host machine by the hypervisor on the host machine. The hypervisor on the host machine enables the host machine's physical computing resources (e.g., computing resources, memory resources, and networking resources) to be shared among various computing instances running on the host machine.

[0073] For example, as shown in FIG. 2, host machines 202 and 208 execute hypervisors 260 and 266, respectively. These hypervisors may be implemented using software, firmware, hardware, or a combination thereof. Typically, a hypervisor is a process or software layer that resides in a host machine's operating system (OS), which executes on the host machine's hardware processor. The hypervisor provides a virtualized environment that allows the host machine's physical computing resources (e.g., processing resources such as processors / cores, memory resources, and networking resources) to be shared among various virtual machine computing instances executed by the host machine. For example, in FIG. 2, hypervisor 260 resides in the OS of host machine 202 and allows the host machine's computing resources (e.g., processing resources, memory resources, and networking resources) to be shared among computing instances (e.g., virtual machines) executed by host machine 202. A virtual machine can have its own OS (called a guest OS). This guest OS may be the same as or different from the host machine's OS. The OS of a virtual machine executed by a host machine may be the same as or different from the OS of other virtual machines executed by the same host machine. Thus, the hypervisor can run multiple OSs in parallel while sharing the same computing resources of the host machine. The host machines shown in Figure 2 may have the same type of hypervisor or different types of hypervisors.

[0074] A compute instance may be a virtual machine instance or a bare metal instance. In Figure 2, compute instance 268 on host machine 202 and compute instance 274 on host machine 208 are examples of virtual machine instances. Host machine 206 is an example of a bare metal instance provided to a customer.

[0075] In certain examples, an entire host machine may be provided to a single customer, and one or more compute instances (either virtual machines or bare metal instances) hosted by that host machine may all belong to the same customer. In other examples, a host machine may be shared among multiple customers (i.e., multiple tenants). In such a multi-tenant scenario, a host machine may host virtual machine compute instances belonging to different customers. These compute instances may be members of different VCNs for different customers. In certain embodiments, bare metal compute instances are hosted by bare metal servers without a hypervisor. When bare metal compute instances are provided, a single customer or tenant maintains control of the physical CPU, memory, and network interfaces of the host machine hosting the bare metal instance, and the host machine is not shared with other customers or tenants.

[0076] As previously described, each compute instance that is part of a VCN is associated with a VNIC that enables the compute instance to be a member of a subnet of the VCN. The VNIC associated with a compute instance facilitates communication of packets or frames to and from the compute instance. A VNIC is associated with the compute instance when the compute instance is created. In particular embodiments, for a compute instance executed by a host machine, the VNIC associated with the compute instance is executed by an NVD connected to the host machine. For example, in FIG. 2, host machine 202 executes virtual machine compute instance 268 associated with VNIC 276, which is executed by NVD 210 connected to host machine 202. As another example, bare metal instance 272 hosted by host machine 206 is associated with VNIC 280, which is executed by NVD 212 connected to host machine 206. As yet another example, VNIC 284 is associated with compute instance 274 executed by host machine 208, which is executed by NVD 212 connected to host machine 208.

[0077] For a compute instance hosted by a host machine, the NVD connected to that host machine executes a VCN VR corresponding to the VCN of which the compute instance is a member. For example, in the embodiment shown in Figure 2, NVD 210 executes VCN VR 277 corresponding to the VCN of which compute instance 268 is a member. NVD 212 may also execute one or more VCN VRs 283 corresponding to the VCNs corresponding to the compute instances hosted by host machines 206 and 208.

[0078] A host machine may include one or more network interface cards (NICs) for connecting the host machine to other devices. The NICs on a host machine may provide one or more ports (or interfaces) for communicatively connecting the host machine to another device. For example, one or more ports (or interfaces) on the host machine and the NVD may be used to connect the host machine to the NVD. The host machine may also be connected to other devices, such as other host machines.

[0079] 2, host machine 202 is connected to NVD 210 using link 220 extending between port 234 provided by NIC 232 of host machine 202 and port 236 of NVD 210. Host machine 206 is connected to NVD 212 using link 224 extending between port 246 provided by NIC 244 of host machine 206 and port 248 of NVD 212. Host machine 208 is connected to NVD 212 using link 226 extending between port 252 provided by NIC 250 of host machine 208 and port 254 of NVD 212.

[0080] Similarly, the NVDs are connected via communication links to top-of-rack (TOR) switches, which are connected to a physical network 218 (also called a switch fabric). In particular embodiments, the links between the host machines and the NVDs and the links between the NVDs and the TOR switches are Ethernet links. For example, in FIG. 2, NVDs 210 and 212 are connected to TOR switches 214 and 216, respectively, via links 228 and 230. In particular embodiments, links 220, 224, 226, 228, and 230 are Ethernet links. A collection of host machines and NVDs connected to a TOR may be referred to as a rack.

[0081] The physical network 218 provides a communications fabric that enables the TOR switches to communicate with each other. The physical network 218 may be a multi-tier network. In a particular implementation, the physical network 218 is a multi-tier Clos network of switches, with the TOR switches 214 and 216 representing leaf-level nodes of the multi-tier and multi-node physical switching network 218. Different Clos network configurations are possible, including, but not limited to, 2-tier networks, 3-tier networks, 4-tier networks, 5-tier networks, and generally "n"-tier networks. An example of a Clos network is shown in FIG. 5 and described below.

[0082] A variety of different connection configurations are possible between a host machine and the N virtual disks, including one-to-one, many-to-one, and one-to-many configurations. In a one-to-one implementation, each host machine is connected to its own separate virtual disk. For example, in FIG. 2, host machine 202 is connected to virtual disk 210 via NIC 232 of host machine 202. In a many-to-one configuration, multiple host machines are connected to a single virtual disk. For example, in FIG. 2, host machines 206 and 208 are connected to the same virtual disk 212 via NICs 244 and 250, respectively.

[0083] In a one-to-many configuration, one host machine is connected to multiple NVDs. Figure 3 shows an example of a CSPI 300 in which a host machine is connected to multiple NVDs. As shown in Figure 3, a host machine 302 includes a network interface card (NIC) 304 including multiple ports 306 and 30S. The host machine 300 is connected to a first NVD 310 via port 306 and link 320, and to a second NVD 312 via port 308 and link 322. Ports 306 and 308 may be Ethernet ports, and links 320 and 322 between the host machine 302 and the NVDs 310 and 312 may be Ethernet links. The NVD 310 is connected to a first TOR switch 314, and the NVD 312 is connected to a second TOR switch 316. The links between the NVDs 310 and 312 and the TOR switches 314 and 316 may be Ethernet links. TOR switches 314 and 316 represent layer-0 switching devices within a multi-tier physical network 318 .

[0084] 3 provides two separate physical network paths from the physical switch network 318 to the host machine 302: a first path from the TOR switch 314 to the host machine 302 via the NVD 310, and a second path from the TOR switch 316 to the host machine 302 via the NVD 312. The separate paths provide enhanced availability (referred to as high availability) for the host machine 302. If there is a problem with one of the paths (e.g., a link on one of the paths fails) or if there is a problem with a device (e.g., a particular NVD is not functioning), the other path can be used for communications to and from the host machine 302.

[0085] In the configuration shown in Figure 3, the host machine is connected to two different NVDs using two different ports provided by the host machine's NIC. In other embodiments, the host machine may include multiple NICs allowing the host machine to connect to multiple NVDs.

[0086] Referring again to Figure 2, an NVD is a physical device or element that performs one or more network virtualization and / or storage virtualization functions. An NVD may be any device that has one or more processing units (e.g., a CPU, a network processing unit (NPU), an FPGA, a packet processing pipeline), memory including cache, and ports. Various virtualization functions may be performed by software / firmware executed by one or more processing units of the NVD.

[0087] The NVD may be implemented in a variety of different ways. For example, in a particular embodiment, the NVD is implemented as an interface card with an embedded processor, called a smart NIC or intelligent NIC. The smart NIC is a separate device from the NIC on the host machine. In Figure 2, the NVD 210 may be implemented as a smart NIC connected to the host machine 202, and the NVD 212 may be implemented as a smart NIC connected to the host machines 206 and 208.

[0088] However, a smart NIC is only one example of an NVD implementation. Various other implementations are possible. For example, in some other implementations, the NVD or one or more functions performed by the NVD may be incorporated into or performed by one or more host machines, one or more TOR switches, and other elements of CSPI 200. For example, the NVD may be integrated into a host machine. In this case, the functions performed by the NVD are performed by the host machine. As another example, the NVD may be part of a TOR switch, or a TOR switch may be configured to perform the functions performed by the NVD that enable the TOR switch to perform various complex packet transformations used in public clouds. A TOR that performs the functions of an NVD may be referred to as a smart TOR. In yet other implementations that offer customers virtual machine (VM) instances rather than bare metal (BM) instances, the functions provided by the NVD may be implemented inside the hypervisor of the host machine. In some other implementations, some of the NVD's functions may be offloaded to a centralized service running on a set of host machines.

[0089] In certain embodiments, such as when implemented as a smart NIC, as shown in FIG. 2, an NVD may include multiple physical ports that allow the NVD to connect to one or more host machines and one or more TOR switches. Ports on an NVD can be categorized as host-facing ports (also called "south ports") or network-facing or TOR-facing ports (also called "north ports"). A host-facing port of an NVD is a port used to connect the NVD to a host machine. Examples of host-facing ports in FIG. 2 include port 236 of NVD 210 and ports 248 and 254 of NVD 212. A network-facing port of an NVD is a port used to connect the NVD to a TOR switch. Examples of network-facing ports in FIG. 2 include port 256 of NVD 210 and port 258 of NVD 212. As shown in FIG. 2, the NVD 210 is connected to the TOR switch 214 via link 228 extending from port 256 of NVD 210 to the TOR switch 214. Similarly, the NVD 212 is connected to the TOR switch 216 via a link 230 that extends from a port 258 of the NVD 212 to the TOR switch 216 .

[0090] The NVD can receive packets and frames (e.g., packets and frames generated by compute instances hosted by the host machine) from the host machine via its host-facing port, perform any necessary packet processing, and then forward the packets and frames to the TOR switch via the NVD's network-facing port. The NVD can receive packets and frames from the TOR switch via the NVD's network-facing port, perform any necessary packet processing, and then forward the packets and frames to the host machine via the NVD's host-facing port.

[0091] In certain embodiments, multiple ports and associated links may be provided between the NVD and the TOR switch. These ports and links can be aggregated to form a link aggregator group (LAG) of multiple ports or links. Link aggregation allows multiple physical links between two endpoints (e.g., between the NVD and the TOR switch) to be treated as a single logical link. All physical links within a given LAG can operate at the same speed and in full-duplex mode. LAGs help increase the bandwidth and reliability of the connection between two endpoints. If one of the physical links in a LAG fails, traffic is dynamically and transparently reassigned to another physical link within the LAG. The aggregated physical link provides higher bandwidth than individual links. Multiple ports associated with a LAG are treated as a single logical port. Traffic can be load-balanced across the multiple physical links in the LAG. One or more LAGs can be configured between two endpoints. The two endpoints may be, for example, between the NVD and the TOR switch, or between a host machine and the NVD.

[0092] The NVD implements or performs network virtualization functions. These functions are performed by software / firmware executed by the NVD. Examples of network virtualization functions include, but are not limited to, packet encapsulation and decapsulation functions, functions for creating VCN networks, functions for implementing network policies such as VCN security list (firewall) functions, functions for facilitating routing and forwarding of packets to and from compute instances within the VCN, etc. In particular embodiments, upon receiving a packet, the NVD is configured to execute a packet processing pipeline that processes the packet and determines how to forward or route the packet. As part of this packet processing pipeline, the NVD provides functions such as running VNICs associated with compute instances within the VCN, running virtual routers (VRs) associated with the VCN, encapsulating and decapsulating packets to facilitate forwarding or routing within the virtual network, running specific gateways (e.g., local peering gateways), implementing security lists, network security groups, network address translation (NAT) functions (e.g., public IP to private IP translation per host), throttling functions, and other functions.

[0093] In some embodiments, the packet processing data path in the NVD may include multiple packet pipelines. Each packet pipeline consists of a series of packet transformation stages. In some implementations, upon receiving a packet, the packet is parsed and sorted into a single pipeline. The packet is then processed stage by stage in a linear fashion until it is discarded or sent out through an interface of the NVD. These stages provide packet processing building blocks of basic functions (e.g., validating headers, performing throttling, inserting new Layer 2 headers, performing L4 firewalling, VCN encapsulation / decapsulation), such that new pipelines can be constructed by assembling existing stages, and new functionality can be added by creating and inserting new stages into existing pipelines.

[0094] The NVD can perform both control plane and data plane functions corresponding to the VCN's control plane and data plane. Examples of the VCN control plane are shown in Figures 21, 22, 23, and 24 (see reference numbers 2116, 2216, 2316, and 2416) and described below. Examples of the VCN data plane are shown in Figures 21, 22, 23, and 24 (see reference numbers 2118, 2218, 2318, and 2418) and described below. Control plane functions include functions used to configure the network to control how data is forwarded (e.g., setting routes and route tables, configuring VNICs). In certain embodiments, a VCN control plane is provided that centrally computes and exposes all overlay-to-substrate mappings to the NVD and virtual network edge devices (e.g., various gateways such as DRGs, SGWs, and IGWs). Firewall rules can also be exposed using the same mechanism. In certain embodiments, the NVD retrieves only mappings relevant to the NVD. The data plane functions include those that perform the actual routing / forwarding of packets based on the configurations established using the control plane. The VCN data plane is implemented by encapsulating customer network packets before they traverse the backbone network. The encapsulation / decapsulation functions are implemented in the NVD. In certain embodiments, the NVD is configured to intercept all network packets entering and leaving the host machine and perform network virtualization functions.

[0095] As described above, the NVD performs various virtualization functions, including VNICs and VCN VRs. The NVD can execute VNICs associated with compute instances hosted by one or more host machines connected to the VNICs. For example, as shown in FIG. 2, NVD 210 executes the functions of VNIC 276 associated with compute instance 268 hosted by host machine 202 connected to NVD 210. As another example, NVD 212 executes VNIC 280 associated with bare metal compute instance 272 hosted by host machine 206 and VNIC 284 associated with compute instance 274 hosted by host machine 208. The host machines can host compute instances that belong to different VCNs that belong to different customers. The NVDs connected to the host machines can execute VNICs (i.e., execute functions associated with the VNICs) corresponding to the compute instances.

[0096] NVDs also execute VCN virtual routers corresponding to the VCNs of the compute instances. For example, in the embodiment shown in FIG. 2, NVD 210 executes VCN VR 277 corresponding to the VCN to which compute instance 268 belongs. NVD 212 executes one or more VCN VRs 283 corresponding to one or more VCNs to which compute instances hosted on host machines 206 and 208 belong. In particular embodiments, a VCN VR corresponding to a VCN is executed by all NVDs connected to a host machine that hosts at least one compute instance belonging to that VCN. If a host machine hosts compute instances that belong to different VCNs, the NVDs connected to that host machine may execute VCN VRs corresponding to the different VCNs.

[0097] In addition to VNICs and VCN VRs, an NVD may include one or more hardware elements that run various software (e.g., daemons) and facilitate various network virtualization functions performed by the NVD. For simplicity, these various elements are grouped as "packet processing elements" shown in FIG. 2. For example, NVD 210 includes packet processing element 286, and NVD 212 includes packet processing element 288. For example, the packet processing element of an NVD may include a packet processor configured to monitor all packets received and communicated using the NVD by interacting with the NVD's ports and hardware working interfaces and store network information. The network information may include, for example, network flow information for identifying different network flows processed by the NVD and information about each flow (e.g., statistics for each flow). In certain embodiments, the network flow information may be stored on a per-VNIC basis. As another example, the packet processing element may include a replication agent configured to replicate information stored by the NVD to one or more different replication target stores. As yet another example, the packet processing element may include a logging agent configured to perform logging functions for the NVD, and may include software for monitoring the performance and health of the NVD, and possibly the status and health of other elements connected to the NVD.

[0098] FIG. 1 illustrates elements of an exemplary virtual or overlay network, including a VCN, subnets within the VCN, compute instances deployed on the subnets, VNICs associated with the compute instances, VRs for the VCN, and a set of gateways configured for the VCN. The overlay elements illustrated in FIG. 1 may be executed or hosted by one or more of the physical elements illustrated in FIG. 2. For example, compute instances within a VCN may be executed or hosted by one or more host machines illustrated in FIG. 2. For compute instances hosted by a host machine, the VNICs associated with the compute instance are typically executed by an NVD connected to the host machine (i.e., the VNIC functionality is provided by an NVD connected to the host machine). The VCN VR functionality is performed by all NVDs connected to the host machines that host or execute compute instances that are part of the VCN. Gateways associated with a VCN may be executed by one or more different types of NVDs. For example, some gateways may be executed by smart NICs, and other gateways may be executed by one or more host machines or other implementations of NVDs.

[0099] As described above, compute instances within a customer VCN can communicate with a variety of different endpoints. These endpoints may be in the same subnet as the source compute instance, in a different subnet but in the same VCN as the source compute instance, or may include endpoints outside the VCN of the source compute instance. These communications are facilitated using VNICs associated with the compute instance, VCN VRs, and gateways associated with the VCN.

[0100] Communication between two compute instances on the same subnet within a VCN is facilitated using VNICs associated with the source and destination compute instances. The source and destination compute instances may be hosted by the same host machine or different host machines. A packet originating from a source compute instance may be forwarded from the host machine hosting the source compute instance to an NVD connected to that host machine. In the NVD, the packet is processed using a packet processing pipeline, which may include the execution of a VNIC associated with the source compute instance. Because the packet's destination endpoint is in the same subnet, the execution of the VNIC associated with the source compute instance forwards the packet to an NVD running the VNIC associated with the destination compute instance, which processes the packet and forwards it to the destination compute instance. The VNICs associated with the source and destination compute instances may run on the same NVD (e.g., if both the source and destination compute instances are hosted by the same host machine) or on different NVDs (e.g., if the source and destination compute instances are hosted by different host machines connected to different NVDs). The VNIC can use the routing / forwarding tables stored by the NVD to determine the next hop for a packet.

[0101] When communicating a packet from a compute instance in a subnet to an endpoint in a different subnet within the same VCN, the packet originating from the source compute instance is communicated from the host machine hosting the source compute instance to the NVD connected to that host machine. In the NVD, the packet is processed using a packet processing pipeline that may include running one or more VNICs and VRs associated with the VCN. For example, the NVD executes or invokes a function corresponding to a VNIC associated with the source compute instance (also referred to as executing a VNIC) as part of the packet processing pipeline. The function executed by the VNIC may include examining the VLAN tag on the packet. Because the packet's destination is outside the subnet, a VCN VR function is invoked and executed by the NVD. The VCN VR then routes the packet to the NVD executing the VNIC associated with the destination compute instance. The VNIC associated with the destination compute instance then processes the packet and forwards the packet to the destination compute instance. The VNICs associated with the source compute instance and the destination compute instance may run on the same NVD (e.g., if both the source compute instance and the destination compute instance are hosted by the same host machine) or may run on different NVDs (e.g., if the source compute instance and the destination compute instance are hosted by different host machines connected to different NVDs).

[0102] If the packet's destination is outside the VCN of the source compute instance, the packet originating from the source compute instance is communicated from the host machine hosting the source compute instance to the NVD connected to that host machine. The NVD runs the VNIC associated with the source compute instance. Because the packet's destination endpoint is outside the VCN, the packet is processed by the VCN VR for that VCN. The NVD invokes a VCN VR function, which may result in the packet being forwarded to an NVD running the appropriate gateway associated with the VCN. For example, if the destination is an endpoint in a customer's on-premises network, the packet may be forwarded by the VCN VR to an NVD running a DRG gateway configured for the VCN. The VCN VR may run on the same NVD as the NVD running the VNIC associated with the source compute instance, or it may be run by a different NVD. The gateway may be run by a smart NIC, a host machine, or another NVD implementation. The packet is then processed by the gateway and forwarded to the next hop that facilitates communication of the packet to the intended destination endpoint. 2, a packet originating from compute instance 268 may be communicated from host machine 202 to NVD 210 over link 220 (using NIC 232). VNIC 276 on NVD 210 is called out because it is the VNIC associated with source compute instance 268. VNIC 276 is configured to inspect information encapsulated within the packet, determine a next hop for forwarding the packet to facilitate communication of the packet to its intended destination endpoint, and forward the packet to the determined next hop.

[0103] Compute instances deployed on a VCN can communicate with a variety of different endpoints. These endpoints may include endpoints hosted by CSPI 200 and endpoints external to CSPI 200. Endpoints hosted by CSPI 200 may include instances within the same VCN or other VCNs (which may be customer VCNs or VCNs not belonging to the customer). Communication between endpoints hosted by CSPI 200 may be performed over physical network 218. Compute instances can also communicate with endpoints not hosted by or external to CSPI 200. Examples of these endpoints include endpoints within a customer on-premises network or data center, or public endpoints accessible over a public network such as the Internet. Communication with endpoints external to CSPI 200 may be performed over a public network (e.g., the Internet) (not shown in FIG. 2) or a private network (not shown in FIG. 2) using various communication protocols.

[0104] The architecture of CSPI 200 shown in FIG. 2 is merely an example and is not intended to be limiting. Variations, substitutions, and modifications are possible in alternative embodiments. For example, in some implementations, CSPI 200 may have more or fewer systems or elements than those shown in FIG. 2, may combine two or more systems, or may have a different system configuration or arrangement. The systems, subsystems, and other elements shown in FIG. 2 may be implemented in software (e.g., code, instructions, programs) executed by one or more processing units (e.g., processors, cores) of the respective systems, hardware, or a combination thereof. Software may be stored in a non-transitory storage medium (e.g., a memory device).

[0105] FIG. 4 illustrates connections between host machines and an NVD to provide I / O virtualization to support multi-tenancy, according to certain embodiments. As shown in FIG. 4, a host machine 402 runs a hypervisor 404 that provides a virtualized environment. The host machine 402 runs two virtual machine instances: VM1 406, which belongs to customer / tenant #1, and VM2 408, which belongs to customer / tenant #2. The host machine 402 includes a physical NIC 410 connected to an NVD 412 via link 414. Each of the compute instances is connected to a VNIC run by the NVD 412. In the embodiment of FIG. 4, VM1 406 is connected to VNIC-VM1 420, and VM2 408 is connected to VNIC-VM2 422.

[0106] 4, NIC 410 includes two logical NICs: logical NIC A 416 and logical NIC B 418. Each virtual machine is connected to and configured to operate with its own logical NIC. For example, VM1 406 is connected to logical NIC A 416, and VM2 408 is connected to logical NIC B 418. The logical NICs allow each tenant's virtual machine to believe it owns its own host machine and NIC, even though host machine 402 consists of only one physical NIC 410 shared by multiple tenants.

[0107] In particular embodiments, each logical NIC is assigned its own VLAN ID. Thus, logical NIC A 416 for tenant #1 is assigned a particular VLAN ID, and logical NIC B 418 for tenant #2 is assigned a different VLAN ID. When a packet is communicated from VM1 406, the hypervisor attaches a tag assigned to tenant #1 to the packet before communicating the packet from host machine 402 to NVD 412 over link 414. Similarly, when a packet is communicated from VM2 408, the hypervisor attaches a tag assigned to tenant #2 to the packet before communicating the packet from host machine 402 to NVD 412 over link 414. Thus, a packet 424 communicated from host machine 402 to NVD 412 has an associated tag 426 that identifies the particular tenant and associated VM. When a packet 424 is received on the NVD from host machine 402, the tag 426 associated with the packet is used to determine whether the packet should be processed by VNIC-VM1 420 or VNIC-VM2 422. The packet is then processed by the corresponding VNIC. The configuration shown in Figure 4 allows each tenant's compute instance to believe it owns its own host machine and NIC. The configuration shown in Figure 4 provides I / O virtualization to support multi-tenancy.

[0108] FIG. 5 is a schematic block diagram illustrating a physical network 500 according to a particular embodiment. The embodiment illustrated in FIG. 5 is constructed as a Clos network. A Clos network is a particular type of network topology designed to provide connection redundancy while maintaining high bisection bandwidth and maximum resource utilization. A Clos network is a type of non-blocking, multi-stage or multi-layer switching network, and the number of stages or layers may be 2, 3, 4, 5, etc. The embodiment illustrated in FIG. 5 is a three-layer network, including layers 1, 2, and 3. TOR switch 504 represents a layer-0 switch in the Clos network. One or more NVDs are connected to the TOR switch. The layer-0 switch is also referred to as an edge device of the physical network. The layer-0 switch is connected to a layer-1 switch, also referred to as a leaf switch. In the embodiment illustrated in FIG. 5, “n” layer-0 TOR switches are connected to “n” layer-1 switches to form a pod. Each layer-0 switch in a pod is interconnected to all layer-1 switches in the pod, but switches between pods are not connected. In a specific implementation, the two pods are referred to as blocks. Each block is served by or connected to "n" layer-2 switches (also referred to as spine switches). A physical network topology may include multiple blocks. Similarly, the layer-2 switches are connected to "n" layer-3 switches (also referred to as super-spine switches). Communication of packets through the physical network 500 is typically performed using one or more layer-3 communication protocols. Typically, all layers of the physical network, except for the TOR layer, are n-way redundant, thus achieving high availability. The physical network can be scaled by specifying policies on pods and blocks to control the mutual visibility of switches in the physical network.

[0109] A characteristic of Clos networks is that the maximum hop count from one tier-0 switch to another tier-0 switch (or from an NVD connected to a tier-0 switch to another NVD connected to a tier-0 switch) is constant. For example, in a three-tier Clos network, a packet requires a maximum of seven hops to travel from one NVD to another. In this case, the source NVD and target NVD are connected to the leaf layers of the Clos network. Similarly, in a four-tier Clos network, a packet requires a maximum of nine hops to travel from one NVD to another. In this case, the source NVD and target NVD are connected to the leaf layers of the Clos network. Therefore, the Clos network architecture maintains constant overall network latency, which is important for intra- and inter-datacenter communications. Clos topologies are horizontally scalable and cost-effective. The network bandwidth / throughput capacity can be easily increased by adding more switches (e.g., more leaf switches and spine switches) at each tier and by increasing the number of links between switches in adjacent tiers.

[0110] In certain embodiments, each resource in the CSPI is assigned a unique identifier called a Cloud Identifier (CID). This identifier is included as part of the resource's information. This identifier can be used to manage the resource, for example, through a console or API. An exemplary syntax for a CID is as follows:

[0111] ocid1.<RESOURCE TYPE> . <realm>.[REGION][.FUTURE USE].<UNIQUE ID> In the formula, "ocid1" is a string that indicates the version of the CID.

[0112] "RESOURCE TYPE" represents the type of resource (e.g., instance, volume, VCN, subnet, user, group).

[0113] "REALM" represents the realm in which the resource resides. Example values ​​include "c1" representing the commercial realm, "c2" representing the government cloud realm, or "c3" representing the federal cloud realm. Each realm can have its own domain name.

[0114] "REGION" represents the region that the resource belongs to. If a region does not apply to the resource, this part may be blank.

[0115] "FUTURE USE" indicates that the item is reserved for future use. "UNIQUE ID" is the unique ID part. This format may vary depending on the type of resource or service.

[0116] Packet flows based on cached and non-cached configuration information 6 illustrates an example network architecture that supports packet flows based on cached configuration information, according to one embodiment. The network architecture represents, at least in part, a CSP's cloud infrastructure. As shown, the network architecture includes a host machine 660, an NVD 670, a substrate network 680, and a host machine 690.

[0117] Generally, host machine 660 hosts one or more compute instances, such as customer compute instance 662, and is connected to NVD 670. The connection can be an Ethernet link. NVD 670 hosts one or more VNICs, including VNIC 672 attached to compute instance 662, among other virtual functions. Typically, a VNIC can be associated with a single compute instance and can provide virtual network interface functionality to the compute instance. The virtual network interface functionality includes providing an interface to a virtual network (e.g., compute instance 662's interface to virtual network 682). NVD 670 is also connected to substrate network 680 via a switch (e.g., a top-of-rack (TOR) switch supporting Ethernet links). Substrate network 680 is also connected to host machine 690; this connection can be through another NVD (not shown in FIG. 6 for simplicity). Host machine 690 can host one or more compute instances, including endpoints 692. Compute instance 662 can communicate with endpoint 692, such as by sending and receiving packets over virtual network 682 via VNIC 672 (and similarly via a VNIC attached to endpoint 692, also not shown in FIG. 6 for simplicity). Virtual network 682 can be a software-based or software-defined network overlaid on top of substrate network 680. As shown by the dashed box, customer VCN 600 can include compute instance 662, VNIC 672, and endpoint 692 (although it is possible that endpoint 692 need not be part of VCN 600).

[0118] In one example, the flow of packets to and from compute instances of host machine 660 may depend on configuration information 674 stored in the memory of NVD 670. Configuration information 674 may include any or all of security policies (e.g., policies based on firewall rules for inbound and / or outbound traffic), routing rules, overlay-to-board IP address mappings, routing tables, and other types of information for sending and / or receiving packets. In one example, configuration information 674 may include multiple sets, each set cached for a VNIC, associated with a corresponding compute instance, and specific to information needed for the flow of outbound or inbound packets to that compute instance. In this example, when compute instance 662 sends a packet, host machine 660 forwards the packet to NVD 670. VNIC 672 then looks up the applicable configuration information, processes the packet (e.g., applies rules, updates packet headers, etc.), and transmits the processed packet over virtual network 682. Sending a packet over virtual network 682 involves NVD 670 further processing the packet and sending it over substrate network 680 in a manner transparent to VNIC 672 and compute instance 662. Ultimately, the packet is received by endpoint 692. Conversely, a packet destined for compute instance 662 is first received by NVD 670, processed by VNIC 672 based on applicable configuration information, and then sent from NVD 670 to host machine 660 to reach compute instance 662.

[0119] As described hereinabove, the network architecture of Figure 6 may require a large amount of configuration information to be stored and maintained in the NVD 670. Furthermore, the VNICs hosted by the NVD 670 also use the processing and memory resources of the NVD 670. Thus, the density of VNICs on the NVD 670 (e.g., the number of VNICs the NVD 670 can host) may be relatively limited.

[0120] 7 illustrates an example network architecture that supports packet flows based on uncached configuration information, according to an embodiment. This network architecture generally allows for increased VNIC density per NVD, but may impact network bandwidth. Rather than storing configuration information with the NVD, the NVD hosts stripped-down versions of VNICs (called micro-VNICs), and the configuration information is stored by a network interface service that provides a centralized network interface service. As shown, the network architecture includes a host machine 710, an NVD 720, a server fleet 730, a board network 740, and a host machine 750.

[0121] Generally, a host machine 710 hosts one or more compute instances, such as a customer's compute instance 712, and is connected to an NVD 720. The connection can be an Ethernet link. The NVD 720 hosts one or more VNICs, including a VNIC 722 attached to the compute instance 712, among other virtual functions. Unlike the VNICs hosted on the NVD 670 of FIG. 6, the VNICs hosted on the NVD 670 can be micro VNICs. Typically, micro VNICs are simpler to implement than VNICs and may not require configuration information to attach to a compute instance and process packets; instead, they may be configured to send the compute instance's outbound packets directly to a network interface service 732 on the server fleet 730 via the NVD 720, and to send inbound packets directly to the compute instance via the NVD 720. The NVD 720 is connected to the server fleet 730 (e.g., via a tunnel or some other peering mechanism).

[0122] Server fleet 730 may be a set of servers or host machines or enclaves that host network interface services 732 for one or more customers. Network interface services 732 provide network interface functionality for connectivity between compute instances. In one example, network interface services 732 may be VNICs as a Service (VNICaaS). For example, network interface services 732 include multiple VNICs, including VNIC 734, which is associated with micro VNIC 722. For clarity, these VNICs may be referred to as service VNICs (as opposed to micro VNICs). Typically, a service VNIC may be associated with multiple micro VNICs, each of which is then associated with a single compute instance. In other words, a service VNIC may be associated with multiple compute instances through their corresponding micro VNICs and provide virtual network interface functionality to these compute instances. The virtual network interface functionality includes providing an interface for a compute instance to a virtual network (e.g., an interface for compute instance 712 to virtual network 742).

[0123] Server fleet 730 is connected to substrate network 740 through a set of switches. Substrate network 740 is also connected with host machine 750, and the connection may be through another NVD, server fleet 730, and / or another server fleet (not shown in FIG. 7 for simplicity). Host machine 750 can host one or more compute instances including endpoints 752. Compute instance 712 can communicate with endpoint 752, such as by sending and receiving packets over virtual network 742 via micro VNIC 722 and service VNIC 734 (and similarly via a VNIC attached to endpoint 752 or a pair of micro VNIC and service VNIC attached to endpoint 752, also not shown in FIG. 7 for simplicity). Virtual network 742 may be a software-based or software-defined network overlaid on top of substrate network 740. As shown by the dashed box, the customer's VCN 760 may include a compute instance 712, a micro VNIC 722, a service VNIC 734, and an endpoint 752 (although it is possible that the endpoint 752 may not be part of the VCN 760).

[0124] In one example, the flow of packets to and from compute instances of the host machine 710 may depend on configuration information 736 stored by the network interface service 732. The configuration information 736 may include any or all of security policies (e.g., policies based on firewall rules for inbound and / or outbound traffic), routing rules, overlay-to-board IP address mappings, routing tables, and other types of information for sending and / or receiving packets. In one example, the configuration information 736 may include multiple sets, each set being cached for a service VNIC, a micro VNIC, and / or a compute instance. In this example, when the compute instance 712 sends a packet, the host machine 710 forwards the packet to the NVD 720. The micro VNIC 722 then sends the packet via the NVD 720 to the network interface service 732. The server fleet 730 receives the packet from the NVD 670 via the tunnel and forwards it to the network interface service 732. The network interface service 732 selects the service VNIC 734 for further processing of the packet. Service VNIC 734 then looks up the applicable configuration information, processes the packet (e.g., applies rules, updates the packet's header, etc.), and sends the processed packet on virtual network 742. Sending the packet on virtual network 742 involves server fleet 730 sending the packet on substrate network 740 in a manner transparent to service VNIC 734, micro VNIC 722, and compute instance 712. Ultimately, the packet is received by endpoint 752. Conversely, a packet destined for compute instance 712 is processed by service VNIC 734 based on the applicable configuration information and sent via server fleet 730 and NVD 720 to micro VNIC 722, and then from micro VNIC 722 via NVD 720 and host machine 710 to compute instance 712.

[0125] 7 may require packets to flow to and from the host machine 710 through the network interface services 732. This flow may impact network bandwidth, especially as the volume of packets increases.

[0126] FIG. 8 illustrates an example network architecture that supports packet flows based on cached flow information and uncached configuration information, according to an embodiment. Here, this network architecture can optimize the balance between memory usage (and VNIC density on the NVD) and network bandwidth. In one example, a network interface service can maintain configuration information available for managing packet flows. The NVD also hosts VNICs and may store only a relevant subset of the configuration information, referred to herein as flow information. This subset may be a minimal or reduced amount of configuration information required to send and / or receive packets directly from the virtual network without the need to utilize the network interface service. Only if this subset is unavailable or stale does the packet flow need to involve the network interface service. As shown, the network architecture includes a host machine 810, an NVD 820, a server fleet 830, a substrate network 840, and a host machine 850.

[0127] Generally, host machine 810 hosts one or more compute instances, such as customer compute instance 812, and is connected to NVD 820. The connection can be an Ethernet link. NVD 820 hosts, among other virtual functions, one or more VNICs, including VNIC 822 attached to compute instance 812. NVD 820 also stores flow information 824 that can be used by such VNICs to send and / or receive packets on one or more virtual networks, such as virtual network 842. NVD 820 is connected to server fleet 830 (e.g., via a tunnel or some other peering mechanism). NVD 820 is also connected to backbone network 840 (e.g., via a TOR switch).

[0128] Server fleet 830 may be a set of servers or host machines or enclaves that host network interface services 832 for one or more customers. Network interface services 832 provide network interface functionality for connectivity between compute instances. In one example, network interface services 832 may be VNICaaS. For example, network interface services 832 includes multiple service VNICs, including VNIC 822 and service VNIC 834 associated with compute instance 812. These service VNICs rely on configuration information 836 stored by network interface services 832 to process and transmit packets.

[0129] Server fleet 830 is connected to substrate network 840 through a set of switches. Substrate network 840 is also connected with host machine 850; the connection may be through another NVD, server fleet 830, and / or another server fleet (not shown in FIG. 8 for simplicity). Host machine 850 may host one or more compute instances, including endpoints 852. Compute instance 812 may communicate with endpoint 852, such as by sending and receiving packets over virtual network 842 via VNIC 822 and / or service VNIC 834 (and similarly via VNICs attached to endpoint 852 or other service VNICs, also not shown in FIG. 8 for simplicity). Virtual network 842 may be a software-based or software-defined network overlaid on substrate network 840. As shown by the dashed box, the customer's VCN 860 may include a compute instance 812, a VNIC 822, a service VNIC 834, and an endpoint 852 (although the endpoint 852 may not be part of the VCN 860).

[0130] In one example, the flow of packets to and from compute instances of host machine 810 may depend on configuration information 836 stored by network interface service 832 and flow information 824 stored by NVD 820. Configuration information 836 may include any or all of security policies (e.g., policies based on firewall rules for inbound and / or outbound traffic), routing rules, overlay-to-board IP address mappings, routing tables, and other types of information for sending and / or receiving packets. In comparison, flow information 824 may include a subset of configuration information 836, which is a minimal or reduced amount of configuration information required to process and manage the flow of packets through NVD 820. For example, flow information 824 includes any or combination of one or more security policies, one or more overlay-to-board IP address mappings, one or more routing tables, and / or one or more aggregations thereof. Flow information 824 may be granular to a particular resolution, such as packet type (e.g., it may indicate a rewriting rule applicable to a particular packet type), or it may be global to a higher level of resolution (e.g., it may apply to multiple packet types and indicate, for example, how a packet should be processed given a firewall rule). Generally, because flow information 824 is specific to the flow of packets from and / or to one or more compute instances of host machine 810 connected to NVD 820, the amount of flow information 824 is smaller than the amount of configuration information 836, whereas configuration information 836 is applicable to packet flows to and / or from a much larger number of compute instances hosted on multiple host machines. In one example, configuration information 836 may include multiple sets, each set cached for a service VNIC, a VNIC attached to a compute instance, and / or the compute instance.Also in this example, flow information 824 may include multiple sets, each set cached for a VNIC attached to a compute instance and / or for that compute instance.

[0131] To illustrate, consider two scenarios. In the first scenario, compute instance 812 sends a packet to endpoint 852, and flow information 824 is available to support this transmission. In this scenario, the packet is sent directly by VNIC 822 over virtual network 842 (as shown by the upper dotted line) and correspondingly sent directly by NVD 820 over substrate network 840 (as shown by the lower solid line). In a second, alternative scenario, flow information 824 is unavailable. In this case, the packet is processed by service VNIC 834 and sent over virtual network 842 (as shown by the lower dotted line) and correspondingly sent over substrate network 840 by server fleet 830 (as shown by the upper solid line).

[0132] In the first scenario, when compute instance 812 sends a packet, host machine 810 forwards the packet to NVD 820. VNIC 822 then looks up applicable flow information, processes the packet (e.g., applies rules, updates the packet's header, etc.), and sends the processed packet over virtual network 842. Sending the packet over virtual network 842 involves NVD 820 further processing the packet and sending it over underlay network 840 in a manner transparent to VNIC 822 and compute instance 812. Ultimately, the packet is received by endpoint 852.

[0133] In the second scenario, when compute instance 812 sends a packet, host machine 810 forwards the packet to NVD 820. VNIC 822 then determines that no flow information is available to process the packet and instead sends the packet to network interface service 832 via NVD 820. Server fleet 830 receives the packet from NVD 820 via the tunnel and forwards it to network interface service 832. Network interface service 832 selects service VNIC 834 for further processing of the packet. Service VNIC 834 then looks up applicable configuration information to process the packet (e.g., apply rules, update the packet's header, etc.) and sends the processed packet over virtual network 842. Sending the packet over virtual network 842 involves server fleet 830 sending the packet over substrate network 840 in a manner transparent to service VNIC 834, VNIC 822, and compute instance 812. Ultimately, the packet is received by endpoint 852.

[0134] As described herein above, with respect to the network architecture of Figure 6, the NVD 820 stores a smaller amount of configuration information (e.g., flow information 824), thereby enabling a higher density of VNICs. For the network architecture of Figure 7, packet flows involve the network interface services 832 only as needed (e.g., when flow information is not available), thereby reducing bandwidth burden. Aspects of flow information generation, distribution, updating, and management of packet flows are further described in the following figures.

[0135] As discussed above in this specification, the flow of packets on a virtual network not only involves overlay resources, but also includes a corresponding packet flow on the substrate network via the hardware resources that host the overlay resources. In the following figures, and for clarity and brevity, when describing packet flow on a virtual network, details regarding packet flow on the substrate network may be omitted; such details may be implied by reference to the above description.

[0136] FIG. 9 illustrates an example of an outbound packet flow that results in cached flow information, according to an embodiment. An NVD 910, which hosts a VNIC 912, is connected to a server fleet 920, which hosts a network interface service 922. Initially, the NVD 910 does not store flow information that would allow the VNIC 912 to send packets directly on the virtual network. Instead, packets are sent indirectly on the virtual network through the network interface service 922 (e.g., through a specific service VNIC). The NVD 910 then receives and stores applicable flow information from the network interface service 922, thereby enabling the VNIC 912 to send packets directly on the virtual network from that point, as further described in FIG. 10. The NVD 910, the VNIC 912, the server fleet 920, and the network interface service 922 are examples of the NVD 820, the VNIC 822, the server fleet 830, and the network interface service 832, respectively. In FIG. 9, numbers shown in circles refer to the sequence of events.

[0137] In one example, the VNIC 912 receives a packet, for example, from the compute instance to which the VNIC 912 is attached, and the packet is to be transmitted on a virtual network (e.g., an outbound packet). Given the header information in the packet, the VNIC 912 looks up the cache 914 of the NVD 910 to determine whether cached flow information is available to process the packet. This lookup may use an n-tuple match of source information, destination information, source subnet, destination subnet, packet type, etc. The result of the lookup is a cache miss. Therefore, the VNIC 912 sends the packet to the network interface service 922. Then, given the configuration information 924, the network interface service 922 processes the packet (e.g., applies rules, rewrites header information, performs encapsulation, etc.) and transmits the processed packet on the underlay network. Additionally, the network interface service 922 generates a cache entry including the flow information and version information associated with the flow information. The flow information may be a subset of specific configuration information used to transmit packets on the board number. For example, the flow information includes at least one of a security policy, an overlay-to-board IP address mapping, or a route rule. The version information may indicate the current version of the flow information or the current version of a portion of the flow information. For example, the version information is an indicator of the current version of the flow information or an indicator of the current version for each portion of the flow information (e.g., the current version of the security policy, the current version of the overlay-to-board IP address mapping, and the current version of the route rule). The version information can be used to invalidate cache entries, as further described in the next figure. The network interface service 922 sends the cache entry to the NVD 910. The NVD 910 then stores the flow information 916 and version information 918 from the received cache entry as an entry in the cache 914.

[0138] FIG. 10 illustrates an example of outbound packet flows based on cached flow information, according to an embodiment. The example of FIG. 10 continues from the example of FIG. 9. An NVD 1010, which hosts a VNIC 1012, is connected to a server fleet 1020, which hosts a network interface service 1022. The NVD 1010 stores flow information 1016 and version information 1018 in a cache 1014. In comparison, the network interface service 1022 stores configuration information 1024. The NVD 1010, the VNIC 1012, the cache 1014, the flow information 1016, the version information 1018, the server fleet 1020, the network interface service 1022, and the configuration information 1024 are examples of the NVD 910, the VNIC 912, the cache 914, the flow information 916, the version information 918, the server fleet 920, the network interface service 922, and the configuration information 924, respectively.

[0139] In one example, VNIC 1012 receives a packet, for example, from a compute instance to which VNIC 1012 is attached, and the packet is to be transmitted over the virtual network (e.g., an outbound packet). Given the header information in the packet, VNIC 1012 looks up cache 1014 to determine whether cached flow information is available to process the packet. The lookup results in a cache hit with flow information 1016. Version information 1018 indicates that flow information 1016 is valid. Thus, VNIC 1012 transmits the packet directly over the virtual network without having to send the packet through network interface service 1022.

[0140] FIG. 11 illustrates an example of inbound packet flows that result in cached flow information, according to an embodiment. While FIG. 9 describes generating and caching flow information based on outbound traffic flows, the example of FIG. 11 describes this process based on inbound traffic flows. An NVD 1110 hosting a VNIC 1112 is connected to a server fleet 1120 hosting a network interface service 1122. Initially, the NVD 1110 does not store flow information that would allow the VNIC 1112 to directly receive packets from the virtual network, as further described in FIG. 12. Instead, the packets are further processed by the network interface service 1122 (e.g., via a particular service VNIC). The NVD 1110 then receives the processed packets from the network interface service 1122 and stores the applicable flow information, thereby enabling the VNIC 1112 to directly transmit packets on the virtual network. NVD 1110, VNIC 1112, server fleet 1120, and network interface service 1122 are examples of NVD 820, VNIC 822, server fleet 830, and network interface service 832, respectively.

[0141] In one example, VNIC 1112 receives an inbound packet on a virtual network. Given the header information in the packet, VNIC 1112 looks up the cache 1114 of NVD 1110 to determine whether cached flow information is available to process the packet. The result of the lookup is a cache miss. Therefore, VNIC 1112 sends the packet to network interface services 1122. Then, given configuration information 1124, network interface services 1122 processes the packet (e.g., applies rules, rewrites header information, performs encapsulation, etc.) and sends the processed packet back to VNIC 1112, which then sends the packet to the compute instance to which VNIC 1112 is attached. In addition, network interface services 1122 generates a cache entry that includes flow information and version information associated with the flow information. The flow information may be a subset of the specific configuration information used to send the packet on the underlying network. For example, the flow information includes at least one of a security policy, an overlay-to-board IP address mapping, or a route rule. The version information can indicate the current version of the flow information or the current version of a portion of the flow information. For example, the version information is an indicator of the current version of the flow information or an indicator of the current version for each portion of the flow information (e.g., the current version of the security policy, the current version of the overlay-to-board IP address mapping, and the current version of the route rule). The version information can be used to invalidate cache entries, as further described in the next figure. The network interface service 1122 sends the cache entry to the NVD 1110. The NVD 1110 then stores the flow information 1116 and version information 1118 from the received cache entry as an entry in the cache 1114.

[0142] As described above in connection with FIG. 11, the receiving NVD (e.g., NVD 1110) caches flow information based on inbound traffic. Variations may exist. For example, upon receiving a packet from a VNIC, the network interface service sends the associated flow information to both the receiving NVD and the sending NVD for caching therein. For example, referring again to FIGS. 9 and 11, assume that the sending NVD is NVD 910 and the receiving NVD is NVD 1110. When VNIC 912 sends a packet (e.g., via sending NVD 910), the network interface service (e.g., network interface service 922 or 1122) updates both the sending NVD 910 and the receiving NVD 1110.

[0143] 9 and 11, upon a cache miss, a packet is sent to the network interface service, triggering the network interface service to send a cache entry containing the necessary flow information. Variations here may also exist. For example, the flow information may be included in the header of a packet sent by the network interface service to the NVD, or in the header of a packet sent from the sending NVD to the receiving NVD. In situations where the network interface service updates both the receiving NVD and the sending NVD, various pieces of flow information may be included in different packets (e.g., flow information is included in a packet sent to the receiving NVD, and flow information is included in a response packet sent to the sending NVD). In one exemplary variation, the sending NVD includes the flow information in the header of the receiving NVD for subsequent use by the receiving NVD. In this example, the receiving NVD may not have an incoming security policy. Instead, a signature may be included in the header and used by the receiving NVD to authenticate that the flow information was sent by an authorized entity.

[0144] Figure 12 illustrates an example of inbound packet flows based on cached flow information, according to one embodiment. The example of Figure 12 follows on from the example of Figure 11. An NVD 1210, which hosts a VNIC 1212, is connected to a server fleet 1220, which hosts a network interface service 1222. The NVD 1210 stores flow information 1216 and version information 1218 in a cache 1214. In comparison, the network interface service 1222 stores configuration information 1224. NVD 1210, VNIC 1212, cache 1214, flow information 1216, version information 1218, server fleet 1220, network interface services 1222, and configuration information 1224 are examples of NVD 1110, VNIC 1112, cache 1114, flow information 1116, version information 1118, server fleet 1120, network interface services 1122, and configuration information 1124, respectively.

[0145] In one example, VNIC 1212 receives an inbound packet on a virtual network. Given the header information in the packet, VNIC 1212 looks up cache 1214 to determine whether cached flow information is available to process the packet. The lookup results in a cache hit with flow information 1216. Version information 1218 indicates that flow information 1216 is valid. Therefore, VNIC 1212 sends the packet to the compute instance to which VNIC 1212 is attached without having to send the packet to network interface services 1222 for further processing.

[0146] FIG. 13 illustrates an example of invalidating and updating cached flow information, according to an embodiment. In this example, invalidation can be managed by a network interface service, and updates can follow a proactive strategy (e.g., by implementing a push mechanism, updated flow information is sent to associated VNICs). Alternatively, updates can follow a passive strategy (e.g., by implementing a pull mechanism, updated flow information is sent to VNICs upon an update request from the VNIC). NVD 1310, which hosts VNIC 1312, is connected to server fleet 1320, which hosts network interface service 1322. Network interface service 1322 is connected to control plane 1330. Customer computing devices (not shown in FIG. 13) can connect to control plane 1330 (e.g., application programming interface (API) calls) and update the configuration of the customer's VCN. The updates can result in changes to configuration information usable for managing the flow of packets.

[0147] In one example, the network interface service 1322 stores configuration information 1324 and version information 1326 associated with the configuration information 1324. The configuration information 1324 can include multiple portions (e.g., security policies, overlay-to-board IP address mappings, route rules). The version information 1326 can include an indicator of the current version of the entire configuration information 1324 and / or an indicator of the current version for each portion of the configuration information 1324 (e.g., the current version of the security policies, the current version of the overlay-to-board IP address mappings, and the current version of the route rules). For clarity, the following paragraphs describe the use of a version indicator for each portion of the configuration information 1324. However, this process applies equally when a single version indicator is used for the entire configuration information 1324.

[0148] The network interface service 1322 sends an update request to the control plane 1330 requesting an update to the configuration information 1324. For example, the update request may include a vector clock populated with version indicators. The control plane 1330 determines which of the version indicators are out of date. If none of them are out of date, no update to the configuration information 1324 is needed. Alternatively, if a version indicator is out of date, the corresponding portion of the configuration information 1324 is out of date, and the control plane 1330 sends a replacement configuration information portion with an updated version indicator. For example, if the version indicator for the security policy indicates that it is still the latest version, but the version indicator for the overlay-to-board IP address mapping is out of date, the control plane 1330 simply sends the updated overlay-to-board IP address mapping and its updated version indicator.

[0149] Network interface service 1322 then receives the update from control plane 1330. Network interface service 1322 tracks VNICs that use the flow information portion that corresponds to the old configuration information portion, generates updated flow information portions and corresponding version indicators, and pushes the updated flow information portions and corresponding version indicators to the NVDs that host those VNICs. For example, NVD 1310 receives updated flow information 1316 and updated version information 1318 and stores them in its cache 1314 for use by VNIC 1312.

[0150] As described above in this specification, a passive strategy can be used instead. Here, rather than pushing updates to the NVD, network interface service 1322 notifies affected VNICs that their flow information (or portions thereof) is stale and only sends updates upon request. For example, upon receiving notification that flow information 1316 (or portions thereof) is stale, VNIC 1312 can request updates thereto from network interface service 1322. In response, network interface service 1322 sends the updated flow information (or portions thereof) and corresponding version information.

[0151] FIG. 14 illustrates another example of invalidating and updating cached flow information, according to an embodiment. In this example, invalidation can be managed by the NVD, and updates can follow a proactive strategy (e.g., by requesting an update before receiving a packet to which the flow information applies). Alternatively, updates can follow a passive strategy (e.g., by sending a packet to a network interface service for further processing and then receiving updated flow information). NVD 1410, which hosts VNIC 1412, is connected to server fleet 1420, which hosts network interface service 1422. Network interface service 1422 is connected to control plane 1430. Customer computing devices (not shown in FIG. 14) can connect to control plane 1430 and update the configuration of the customer's VCN. The updates can result in changes to configuration information that can be used to manage the flow of packets.

[0152] In one example, the NVD 1410 stores in the cache 1414 flow information 1416 and version information 1418 associated with the flow information 1416. The flow information 1416 can include multiple portions (e.g., security policies, overlay-to-board IP address mappings, route rules). The version information 1418 can include an indicator of the current version of the flow information 1416 as a whole and / or an indicator of the current version for each portion of the flow information 1416 (e.g., the current version of the security policies, the current version of the overlay-to-board IP address mappings, and the current version of the route rules). For clarity, the following paragraphs describe the use of a version indicator for each portion of the flow information 1416. However, this process applies equally when a single version indicator is used for the entire flow information 1416.

[0153] The NVD 1410 sends an update request to the control plane 1430 requesting an update to the flow information 1416. For example, the update request may include a vector clock populated with version indicators. The control plane 1430 determines which of the version indicators are out of date. If none of them are out of date, no update to the flow information 1416 is needed. Otherwise, if a version indicator is out of date, the corresponding portion of the flow information 1416 is out of date, and the control plane 1430 sends a notification to the NVD 1410 indicating that an update is available. This notification may indicate which portions of the flow information 1416 are out of date.

[0154] In a passive strategy, the NVD 1410 does not immediately update the flow information 1416. Instead, the VNIC 1412 receives the packet and determines (e.g., based on a cache hit) that the flow information 1416 applies to it. The VNIC also determines that the version information 1418 is out of date (e.g., some of the flow information 1416 is out of date according to a notification from the control plane 1430). Alternatively, upon receiving the notification, the NVD 1410 invalidates the cached flow information 1416 so that when a cache lookup is performed, the lookup results in a cache miss. In both cases, the VNIC 1412 sends the packet to the network interface services 1422 for further processing. This packet transmission represents a passive update request. The network interface services 1422 processes the packet using the configuration information 1424 with the latest version information 1426 (which can be updated from the control plane 1430 in a manner similar to FIG. 13 ). Additionally, network interface service 1422 sends an update to NVD 1410, the update including the updated flow information (e.g., the entire flow information, but not portions thereof) and the corresponding version indicator. NVD 1410 then replaces existing flow information 1416 (or older portions thereof) and version information 1418 (or older version indicator) with the updated flow information portions and the corresponding version indicator.

[0155] For example, only the overlay-to-board IP mapping is out of date in flow information 1416. NVD 1410 receives notification from control plane 1430 that the corresponding version indicator is out of date. VNIC 1412 then sends the packet that would otherwise have been sent directly on the virtual network to network interface service 1422. In response, network interface service 1422 sends updated flow information and the corresponding version indicator. NVD 1410 replaces the outdated overlay-to-board IP address mapping in flow information 1416 and the corresponding version indicator in version information 1418 with the updated overlay-to-board IP address mapping and the corresponding version indicator, respectively.

[0156] As described above in this specification, a proactive strategy can be used instead. Here, rather than sending a packet to the network interface service 1422 and then receiving an update back, the NVD 1410 can automatically send an update request to the network interface service 1422. This request can indicate which portions of the flow information 1416 are out of date, and the network interface service 1422 may send only the updated portions. Alternatively, no such indication may be given, and the network interface service 1422 may send the entire flow information.

[0157] 15-20 illustrate example methods for distributing configuration information and managing packet flow. The operations of these methods may be performed by a system including an NVD and / or a network service interface. Some or all of the instructions for performing the operations may be implemented as hardware circuits and / or stored as computer-readable instructions on a non-transitory computer-readable medium of the system. When implemented, the instructions represent modules containing code executable by a processor of the circuit or system. The use of such instructions configures the system to perform specific operations described herein. Each circuit or code in combination with an associated processor represents a means for performing the respective operation. While the operations are shown in a particular order, it should be understood that no particular order is required and one or more operations may be omitted, skipped, performed in parallel, and / or reordered.

[0158] FIG. 15 illustrates an example method for transmitting a packet that results in cached flow information, according to an embodiment. Here, the packet is an outbound packet transmitted on a virtual network. In one example, the method includes operation 1502, in which a VNIC hosted on an NVD receives a packet. The packet may be transmitted from a compute instance to which the VNIC is attached. At operation 1504, the VNIC determines whether flow information is available from the memory of the NVD to process the packet. For example, the VNIC performs a cache lookup based on information from the packet's header. If the lookup results in a cache hit, the flow information is available. In this case, operation 1510 follows operation 1504. Otherwise, operation 1520 follows operation 1504. At operation 1510, the VNIC directly transmits the packet on the virtual network. For example, the VNIC processes the packet according to the flow information and then transmits the processed packet (e.g., with updated header information) on the virtual network. Transmitting on the virtual network may include the NVD further processing the packet for transmission on the underlying substrate network. At operation 1520, no flow information is available (e.g., not cached) to process the packet locally at the NVD. Instead, the VNIC sends the packet to a network interface service that stores configuration information usable to process the packet. At operation 1522, the NVD receives flow information. This flow information may be a subset of the configuration information used by the network interface service (e.g., by a service VNIC hosted therein) to process the packet and transmit it on the virtual network. Additionally, version information associated with the flow information may be received. At operation 1524, the NVD stores the flow information and, if applicable, the version information in memory (e.g., as a cache entry). The flow may be repeated to transmit additional packets on the virtual network, and the additional packets are received from the compute instance.

[0159] FIG. 16 illustrates an example method for receiving a packet that results in cached flow information, according to an embodiment. Here, the packet is an inbound packet received on a virtual network. In one example, the method includes operation 1602, in which a VNIC hosted on an NVD receives a packet. The packet can be transmitted from an endpoint on the virtual network and is destined for a compute instance to which the VNIC is attached. In operation 1604, the VNIC determines whether flow information is available from the memory of the NVD to process the packet. For example, the VNIC performs a cache lookup based on information from the packet's header. If the lookup results in a cache hit, the flow information is available. In this case, operation 1610 follows operation 1604. Otherwise, operation 1620 follows operation 1604. In operation 1610, the VNIC sends the packet directly on the compute instance. For example, the VNIC processes the packet according to the flow information and then sends the processed packet (e.g., with updated header information) to the compute instance via the NVD and the compute instance's host machine. At operation 1620, no flow information is available (e.g., not cached) at the NVD to process the packet locally. Instead, the VNIC sends the packet to a network interface service, which stores configuration information that can be used to process the packet. At operation 1622, the NVD receives the packet processed by the network interface service and the flow information. This flow information may be a subset of the configuration information used by the network interface service (e.g., by the service VNIC hosted therein) to process the packet. In addition, version information associated with the flow information may be received. At operation 1624, the NVD stores the flow information and, if applicable, the version information in memory (e.g., as a cache entry). The received packet is also sent directly to a compute instance, as indicated by the loop from operation 1624 to operation 1610.The flow may be repeated to send additional packets to the compute instance, and the additional packets are received on the virtual network.

[0160] FIG. 17 illustrates an example method for generating and transmitting flow information, according to an embodiment. Here, the method may be implemented by a network interface service. In one example, the method includes operation 1702, in which the network interface service receives a packet. In one example, the packet is an outbound packet originating from a compute instance and transmitted on a virtual network. In another example, the packet is an inbound packet received on the virtual network and destined for the compute instance. In both examples, the network interface service may receive the packet from a VNIC through an NVD hosting the VNIC, where the NVD does not store (cached) flow information that would enable the VNIC to process the packet. In operation 1704, the network interface service (e.g., a service VNIC hosted therein) determines configuration information to process the packet. This determination may be based on header information of the packet. In operation 1706, the network interface service (e.g., the service VNIC) transmits the packet based on the configuration information. Referring back to the first example, the service VNIC processes the packet according to the configuration information and transmits the processed packet on the virtual network. In the second example, the service VNIC also processes the packet according to the configuration information, but sends the processed packet to the NVD's VNIC. In operation 1708, the network interface service generates flow information. For example, the flow information is a subset of the configuration information, enabling the NVD's VNIC to locally process packets of the same type or similar packets. This flow information can correspond to the latest configuration information and can be associated with version information. In operation 1710, the network interface service sends the flow information to the NVD. Additionally, version information can be sent to the NVD.

[0161] FIG. 18 illustrates an example method for invalidating and updating cached flow information, according to an embodiment. Here, the method may be implemented by an NVD hosting a VNIC. In one example, the method includes operation 1802, in which the VNIC receives a packet. In one example, the packet is an outbound packet originating from a compute instance and transmitted on a virtual network. In another example, the packet is an inbound packet received on the virtual network and destined for a compute instance. In operation 1804, the VNIC determines whether flow information is stale, where the flow information is available from the memory of the NVD for processing the packet. For example, the VNIC determines whether version information of the flow information is stale (or a version indicator of part of the flow information is stale). This determination may include sending a vector clock to the control plane and receiving a notification regarding the updated version information. Although operation 1804 is shown as following operation 1802, operation 1804 may be performed independently of operation 1804, where the cached flow information is invalidated before receiving the packet. If the flow information is not outdated, operation 1810 follows operation 1804. Otherwise, operation 1820 follows operation 1804. At operation 1810, the VNIC sends the packet directly onto the virtual network in the case of outbound traffic, or directly to the compute instance in the case of inbound traffic. At operation 1820, valid flow information is not available to process the packet locally at the NVD. Instead, the VNIC sends the packet to a network interface service, which stores configuration information usable to process the packet. At operation 1822, the NVD receives the flow information (and, in the case of inbound traffic, the packet). This flow information may be a subset of the configuration information used by the network interface service (e.g., by a service VNIC hosted therein) to process the packet and send it back to the VNIC on the overlay substrate. Additionally, version information associated with the flow information may be received.At operation 1824, the NVD stores the flow information and, if applicable, version information in memory (e.g., as a cache entry). The flow may be repeated to send additional packets, and additional packets may be received from the compute instance or from the virtual network.

[0162] FIG. 19 illustrates another example of a method for invalidating and updating cached flow information according to an embodiment. Here, the method may be implemented by a network interface service. In one example, the method includes operation 1902, in which the network interface service determines that configuration information is out of date. For example, the network interface service determines that version information of the configuration information is out of date (or that a version indicator of a portion of the flow information is out of date). This determination may include sending a vector clock to the control plane, receiving a notification about updated version information, and / or receiving updated configuration information (or an updated portion thereof). At operation 1904, the network interface service determines VNICs that need to be updated based on an update from the control plane. For example, the network interface service tracks, for each VNIC on the NVD, the corresponding configuration information and its associated version information. If the update affects the configuration information used for a VNIC, the network interface service determines that the VNIC should be updated. At operation 1906, the network interface service sends the updated flow information, and optionally the version information, to the affected VNIC. The updated flow information may be generated as a subset from the updated configuration information. The entire updated flow information may be pushed, or alternatively, only the updated portion of it may be pushed along with a latest version indicator.

[0163] FIG. 20 illustrates another example of a method for invalidating and updating cached flow information according to an embodiment. Here, the method may be implemented by a network interface service and completes the update according to a passive strategy. In one example, the method includes operation 2002, where the network interface service determines that configuration information is stale, similar to operation 1902. In operation 2004, the network interface service determines a VNIC that needs to be updated, similar to operation 1904. In operation 2006, the network interface service indicates to the VNIC that its flow information is stale (or a portion of it is stale). For example, a notification may be sent that results in invalidation of the corresponding cached entry. In operation 2008, the network interface service receives a packet from the VNIC. The packet may be an inbound packet or an outbound packet. In operation 2010, the network interface service processes and transmits the packet using the updated configuration information. For outbound traffic, the packet is transmitted on the virtual network (e.g., via the service VNIC). For inbound traffic, the packet is sent back to the VNIC. In operation 2012, the network interface service sends updated flow information and optionally version information to the VNIC. The updated flow information may be generated as a subset from the updated configuration information. The entire updated flow information may be sent, or alternatively, only the updated portion thereof may be sent along with a latest version indicator.

[0164] Exemplary Infrastructure-as-a-Service Architecture As mentioned above, Infrastructure as a Service (IaaS) is a specific type of cloud computing. IaaS may be configured to provide virtualized computing resources over a public network (e.g., the Internet). In the IaaS model, a cloud computing provider may host infrastructure elements (e.g., servers, storage devices, network nodes (e.g., hardware), deployment software, platform virtualization (e.g., hypervisor layer), etc.). In some cases, an IaaS provider may offer various services (e.g., billing, monitoring, logging, security, load balancing, clustering, etc.) associated with the infrastructure elements. Therefore, because these services may be policy-driven, IaaS users can implement policies to drive load balancing to maintain application availability and performance.

[0165] In some examples, IaaS customers can access resources and services over a wide area network (WAN), such as the Internet, and use the cloud provider's services to install the remaining elements of their application stack. For example, a user can log into an IaaS platform to create virtual machines (VMs), install an operating system (OS) on each VM, deploy middleware such as databases, create storage buckets for workloads and backups, and install enterprise software on the VMs. Customers can use the provider's services to perform a variety of functions, including balancing network traffic, troubleshooting applications, monitoring performance, managing disaster recovery, and more.

[0166] In most cases, the cloud computing model requires the participation of a cloud provider, which can be, but does not have to be, a third-party service provider specializing in providing (e.g., offering, renting, or selling) IaaS. Alternatively, an enterprise can deploy a private cloud and become a provider of infrastructure services.

[0167] In some examples, IaaS deployment is the process of deploying a new application or a new version of an application onto a provisioned application server, etc. IaaS deployment may include the process of provisioning the server (e.g., installing libraries, daemons, etc.). IaaS deployment is often managed by the cloud provider below the hypervisor layer (e.g., servers, storage, network hardware, and virtualization). Thus, customers can perform OS, middleware, and / or application deployment (e.g., self-service virtual machines (e.g., that can be spun up on demand)).

[0168] In some instances, IaaS provisioning may include obtaining the computer or virtual host to be used and installing the necessary libraries or services on the computer or virtual host. In most cases, deployment does not include provisioning, which must be performed first.

[0169] In some cases, IaaS provisioning presents two distinct challenges. First, there is the challenge of provisioning an initial set of infrastructure before anything can be done. Second, there is the challenge of evolving the existing infrastructure (e.g., adding new services, modifying services, removing services) after everything has been provisioned. In some cases, these two challenges can be addressed by allowing the configuration of the infrastructure to be defined declaratively. In other words, the infrastructure (e.g., which elements are needed and how these elements interact) may be defined by one or more configuration files. Thus, the overall topology of the infrastructure (e.g., which resources depend on which and how they work together) can be described declaratively. In some instances, once the topology is defined, workflows can be generated to create and / or manage the different elements described in the configuration files.

[0170] In some examples, the infrastructure can include many interconnected elements. For example, there may be one or more virtual private clouds (VPCs) (e.g., potentially on-demand pools of configurable and / or shared computing resources), also known as a core network. In some examples, there may be one or more security group rules and one or more virtual machines (VMs) that are provisioned to define how the network's security is configured. Other infrastructure elements, such as load balancers, databases, etc., may also be provisioned. The infrastructure can evolve incrementally as more infrastructure elements are desired and / or added.

[0171] In some examples, continuous deployment techniques may be employed to enable deployment of infrastructure code across various virtual computing environments. The described techniques may also enable infrastructure management within these environments. In some examples, a service team may write code that is desired to be deployed to one or more, typically many, different production environments (e.g., across a variety of different geographic locations, sometimes spanning the entire world). However, in some examples, the infrastructure for deploying the code must first be set up. In some examples, provisioning may be done manually, with provisioning tools used to provision resources and / or deployment tools used to deploy the code after the infrastructure has been provisioned.

[0172] 21 is a block diagram 2100 illustrating an example pattern of an IaaS architecture, according to at least one embodiment. A service operator 2102 may be communicatively coupled to a secure host tenancy 2104, which may include a virtual cloud network (VCN) 2106 and a secure host subnet 2108. In some examples, the service operator 2102 may use one or more client computing devices. The one or more client computing devices may run software such as Microsoft Windows Mobile® and / or various mobile operating systems such as iOS, Windows Phone, Android®, BlackBerry 8, and Palm OS, and may be Internet, email, short message service (SMS), BlackBerry®, or other communication protocol-enabled handheld mobile devices (e.g., iPhone®, mobile phone, iPad®, tablet, personal digital assistant (PDA), or wearable device (e.g., Google® Glass® head-mounted display). The client computing devices may be general-purpose personal computers, including, by way of example, personal computers and / or laptop computers running various versions of the Microsoft Windows® operating system, the Apple Macintosh® operating system, and / or the Linux® operating system. Alternatively, the client computing devices may be workstation computers running various commercially available UNIX® or UNIX-like operating systems, including, but not limited to, various GNU / Linux operating systems, e.g., Google Chrome® OS.Alternatively or additionally, the client computing device may be other electronic devices capable of communicating over VCN 2106 and / or a network with access to the Internet, such as a thin client computer, an Internet-enabled gaming system (e.g., a Microsoft Xbox® game console with or without a Kinect® gesture input device), and / or a personal messaging device.

[0173] The VCN 2106 may include a local peering gateway (LPG) 2110 that can be communicatively coupled to a secure shell (SSH) VCN 2112 via an LPG 2110 included in the SSH VCN 2112. The SSH VCN 2112 can include an SSH subnet 2114, which can be communicatively coupled to a control plane VCN 2116 via an LPG 2110 included in the control plane VCN 2116. The SSH VCN 2112 can also be communicatively coupled to a data plane VCN 2118 via the LPG 2110. The control plane VCN 2116 and the data plane VCN 2118 may be included in a service tenancy 2119, which may be owned and / or operated by the IaaS provider.

[0174] The control plane VCN 2116 may include a control plane demilitarized zone (DMZ) tier 2120 that functions as a perimeter network (e.g., a portion of an enterprise network between the enterprise intranet and an external network). DMZ-based servers may have a particular level of trust and contain security breaches. Additionally, the DMZ tier 2120 may include one or more load balancer (LB) subnets 2122, a control plane app tier 2124 that may include an app subnet 2126, and a control plane data tier 2128 that may include a database (DB) subnet 2130 (e.g., a front-end DB subnet and / or a back-end DB subnet). LB subnet 2122 included in control plane DMZ layer 2120 may be communicatively coupled to app subnet 2126 included in control plane app layer 2124 and to Internet gateway 2134, which may be included in control plane VCN 2116, and applisubli 2126 may be communicatively coupled to DB subnet 2130, service gateway 2136, and network address translation (NAT) gateway 2138 included in control plane data layer 2128. Control plane VCN 2116 may include service gateway 2136 and NAT gateway 2138.

[0175] The control plane VCN 2116 can include a data plane mirror app layer 2140, which can include an app subnet 2126. The app subnet 2126 included in the data plane mirror app layer 2140 can include a virtual network interface controller (VNIC) 2142 that can run a compute instance 2144. The compute instance 2144 can communicatively couple the app subnet 2126 of the data plane mirror app layer 2140 to the app subnet 2126 that can be included in the data plane app layer 2146.

[0176] The data plane VCN 2118 may include a data plane app layer 2146, a data plane DMZ layer 2148, and a data plane data layer 2150. The data plane DMZ layer 2148 may include a LB subnet 2122 that may be communicatively coupled to an app subnet 2126 of the data plane app layer 2146 and an internet gateway 2134 of the data plane VCN 2118. The app subnet 2126 may be communicatively coupled to a service gateway 2136 of the data plane VCN 2118 and a NAT gateway 2138 of the data plane VCN 2118. Additionally, the data plane data layer 2150 may include a DB subnet 2130 that may be communicatively coupled to the app subnet 2126 of the data plane app layer 2146.

[0177] The internet gateway 2134 of the control plane VCN 2116 and the internet gateway 2134 of the data plane VCN 2118 may be communicatively coupled to a metadata management service 2152, which may be communicatively coupled to the public internet 2154. The public internet 2154 may be communicatively coupled to a NAT gateway 2138 of the control plane VCN 2116 and the NAT gateway 2138 of the data plane VCN 2118. The service gateway 2136 of the control plane VCN 2116 and the service gateway 2136 of the data plane VCN 2118 may be communicatively coupled to cloud services 2156.

[0178] In some examples, the service gateway 2136 of the control plane VCN 2116 or the data plane VCN 2118 can make application programming interface (API) calls to the cloud services 2156 without traversing the public internet 2154. The API calls from the service gateway 2136 to the cloud services 2156 can be one-way. The service gateway 2136 can make API calls to the cloud services 2156, and the cloud services 2156 can send request data to the service gateway 2136. However, the cloud services 2156 may not initiate the API calls to the service gateway 2136.

[0179] In some examples, secure host tenancy 2104 may be directly connected to service tenancy 2119, which may be an orphan. Secure host subnet 2108 can communicate with SSH subnet 2114 via LPG 2110, which allows bidirectional communication with the orphan system. By connecting secure host subnet 2108 to SSH subnet 2114, secure host subnet 2108 can access other entities in service tenancy 2119.

[0180] The control plane VCN 2116 allows users of the service tenancy 2119 to configure or provision desired resources. The desired resources provisioned in the control plane VCN 2116 may be deployed or used in the data plane VCN 2118. In some examples, the control plane VCN 2116 may be isolated from the data plane VCN 2118, and the data plane mirror app layer 2140 of the control plane VCN 2116 can communicate with the data plane app layer 2146 of the data plane VCN 2118 via a VNIC 2142, which may be included in the data plane mirror app layer 2140 and the data plane app layer 2146.

[0181] In some examples, a user or customer of the system may make a request, such as a create, read, update, or delete (CRUD) operation, via the public internet 2154, which may communicate the request to the metadata management service 2152. The metadata management service 2152 may communicate the request to the control plane VCN 2116 via the internet gateway 2134. The request may be received by the LB subnet 2122 included in the control plane DMZ tier 2120. The LB subnet 2122 may determine that the request is valid, and in response to this determination, the LB subnet 2122 may send the request to the app subnet 2126 included in the control plane app tier 2124. If the request is validated and requires a call to the public internet 2154, the call to the public internet 2154 may be sent to the NAT gateway 2138, which may make the call to the public internet 2154. Memory for storing the request may be stored in the DB subnet 2130.

[0182] In some examples, the data plane mirror app layer 2140 can facilitate direct communication between the control plane VCN 2116 and the data plane VCN 2118. For example, it may be desirable for changes, updates, or other suitable modifications to a configuration to be applied to resources included in the data plane VCN 2118. The control plane VCN 2116 can communicate directly with the resources included in the data plane VCN 2118 via the VNIC 2142 and thus can effect the changes, updates, or other suitable modifications to the configuration.

[0183] In some embodiments, the control plane VCN 2116 and the data plane VCN 2118 may be included in the service tenancy 2119. In this case, a user or customer of the system may not own or operate either the control plane VCN 2116 or the data plane VCN 2118. Instead, an IaaS provider may own or operate the control plane VCN 2116 and the data plane VCN 2118, both of which may be included in the service tenancy 2119. This embodiment can prevent users or customers from interacting with other users' or customers' resources by enabling network isolation. This embodiment can also enable users or customers of the system to store databases privately without having to rely on the public internet 2154, which may not have the desired level of security for storage.

[0184] In another embodiment, the LB subnet 2122 included in the control plane VCN 2116 may be configured to receive signals from the service gateway 2136. In this embodiment, the control plane VCN 2116 and the data plane VCN 2118 may be configured to be called by the IaaS provider's customers without calling the public internet 2154. Customers of the IaaS provider may desire this embodiment because databases used by the customers may be stored in the service tenancy 2119, which may be controlled by the IaaS provider and isolated from the public internet 2154.

[0185] Figure 22 is a block diagram 2200 illustrating another example parameter of an IaaS architecture, according to at least one embodiment. A service operator 2202 (e.g., service operator 2102 of Figure 21) may be communicatively coupled to a secure host tenancy 2204 (e.g., secure host tenancy 2104 of Figure 21), which may include a virtual cloud network (VCN) 2206 (e.g., VCN 2106 of Figure 21) and a secure host subnet 2208 (e.g., secure host subnet 2108 of Figure 21). VCN 2206 may include a local peering gateway (LPG) 2210 (e.g., LPG 2110 of Figure 21), which may be communicatively coupled to a secure shell (SSH) VCN 2212 (e.g., SSH VCN 2112 of Figure 21) via an LPG 2110 included in SSH VCN 2212. SSH VCN 2212 can include SSH subnet 2214 (e.g., SSH subnet 2114 in FIG. 21 ), and SSH VCN 2212 can be communicatively coupled to control plane VCN 2216 (e.g., control plane VCN 2116 in FIG. 21 ) via LPG 2210 included in control plane VCN 2216. Control plane VCN 2216 can be included in service tenancy 2219 (e.g., service tenancy 2119 in FIG. 21 ), and data plane VCN 2218 (e.g., data plane VCN 2118 in FIG. 21 ) can be included in customer tenancy 2221, which can be owned or operated by a user or customer of the system.

[0186] Control plane VCN 2216 may include a control plane DMZ tier 2220 (e.g., control plane DMZ tier 2120 in FIG. 21 ) that may include a LB subnet 2222 (e.g., LB subnet 2122 in FIG. 21 ), a control plane app tier 2224 (e.g., control plane app tier 2124 in FIG. 21 ) that may include an app subnet 2226 (e.g., app subnet 2126 in FIG. 21 ), and a control plane data tier 2228 (e.g., control plane data tier 2128 in FIG. 21 ) that may include a database (DB) subnet 2230 (e.g., similar to DB subnet 2130 in FIG. 21 ). LB subnet 2222 included in control plane DMZ tier 2220 may be communicatively coupled to app subnet 2226 included in control plane app tier 2224 and to an Internet gateway 2234 (e.g., Internet gateway 2134 in FIG. 21 ), which may be included in control plane VCN 2216. App subnet 2226 may be communicatively coupled to DB subnet 2230, service gateway 2236 (e.g., service gateway in FIG. 21 ), and network address translation (NAT) gateway 2238 (e.g., NAT gateway 2138 in FIG. 21 ) included in control plane data layer 2228. Control plane VCN 2216 may include service gateway 2236 and NAT gateway 2238.

[0187] Control plane VCN 2216 can include a data plane mirror app layer 2240 (e.g., data plane mirror app layer 2140 of FIG. 21 ), which can include an app subnet 2226. App subnet 2226 included in data plane mirror app layer 2240 can include a virtual network interface controller (VNIC) 2242 (e.g., VNIC 2142) on which compute instance 2244 (e.g., similar to compute instance 2144 of FIG. 21 ) can run. Compute instance 2244 can facilitate communication between app subnet 2226 of data plane mirror app layer 2240 and app subnet 2226, which can be included in data plane app layer 2246 (e.g., data plane app layer 2146 of FIG. 21 ), via VNIC 2242 included in data plane mirror app layer 2240 and VNIC 2242 included in data plane app layer 2246.

[0188] An internet gateway 2234 included in the control plane VCN 2216 may be communicatively coupled to a metadata management service 2252 (e.g., metadata management service 2152 of FIG. 21 ), which may be communicatively coupled to a public internet 2254 (e.g., public internet 2154 of FIG. 21 ). The public internet 2254 may be communicatively coupled to a NAT gateway 2238 included in the control plane VCN 2216. A service gateway 2236 included in the control plane VCN 2216 may be communicatively coupled to cloud services 2256 (e.g., cloud services 2156 of FIG. 21 ).

[0189] In some examples, data plane VCN 2218 may be included in customer tenancy 2221. In this case, the IaaS provider may provide a control plane VCN 2216 for each customer, and the IaaS provider may configure a unique compute instance 2244 for each customer, which is included in service tenancy 2219. Each compute instance 2244 may allow communication between the control plane VCN 2216 included in service tenancy 2219 and the data plane VCN 2218 included in customer tenancy 2221. The compute instance 2244 may allow resources provisioned in the control plane VCN 2216 included in service tenancy 2219 to be deployed or used in the data plane VCN 2218 included in customer tenancy 2221.

[0190] In another example, a customer of an IaaS provider may have a database that resides in customer tenancy 2221. In this example, control plane VCN 2216 may include data plane minor app tier 2240, which may include app subnet 2226. Data plane mirror app tier 2240 may reside in data plane VCN 2218, but data plane mirror app tier 2240 may not reside in data plane VCN 2218. That is, data plane mirror app tier 2240 may have access to customer tenancy 2221, but data plane mirror app tier 2240 may not reside in data plane VCN 2218 and may not be owned or operated by the IaaS provider's customer. Data plane mirror app tier 2240 may be configured to make calls to data plane VCN 2218, but may not be configured to make calls to any entities included in control plane VCN 2216. A customer may desire to deploy or use resources in data plane VCN 2218 provisioned to control plane VCN 2216, and data plane mirror application tier 2240 may facilitate the desired deployment or other use of the customer's resources.

[0191] In some embodiments, the IaaS provider's customer can apply filters to data plane VCN 2218. In this embodiment, the customer can determine what data plane VCN 2218 can access, and the customer can restrict access from data plane VCN 2218 to the public internet 2254. The IaaS provider may not be able to apply filters or control access from data plane VCN 2218 to any external networks or databases. Applying filters and controls to the data plane VCN 2218 included in customer tenancy 2221 can help isolate the data plane VCN 2218 from other customers and the public internet 2254.

[0192] In some embodiments, cloud services 2256 can be called by service gateway 2236 to access services that may not reside on the public internet 2254, on the control plane VCN 2216, or on the data plane VCN 2218. The connection between cloud service 2256 and control plane VCN 2216 or data plane VCN 2218 may not be live or continuous. Cloud services 2256 may reside on a separate network owned or operated by the IaaS provider. Cloud services 2256 may be configured to receive calls from service gateway 2236 and may not be configured to receive calls from the public internet 2254. Some cloud services 2256 may be isolated from other cloud services 2256, and control plane VCN 2216 may be isolated from cloud services 2256 that may not be located in the same region as control plane VCN 2216. For example, control plane VCN 2216 may be located in “Region 1,” and cloud service “Deployment 21” may be located in “Region 1” and “Region 2.” If a call to deployment 21 is made by service gateway 2236 included in control plane VCN 2216 located in region 1, the call may be sent to deployment 21 in region 1. In this example, control plane VCN 2216 or deployment 21 in region 1 may not be communicatively coupled with deployment 21 in region 2.

[0193] 23 is a block diagram 2300 illustrating another example pattern of an IaaS architecture, according to at least one embodiment. A service operator 2302 (e.g., service operator 2102 of FIG. 21 ) may be communicatively coupled to a secure host tenancy 2304 (e.g., secure host tenancy 2104 of FIG. 21 ), which may include a virtual cloud network (VCN) 2306 (e.g., VCN 2106 of FIG. 21 ) and a secure host subnet 2308 (e.g., secure host subnet 2108 of FIG. 21 ). VCN 2306 may include an LPG 2310 (e.g., LPG 2110 of FIG. 21 ) that may be communicatively coupled to an SSH VCN 2312 (e.g., SSH VCN 2112 of FIG. 21 ) via an LPG 2310 included in SSH VCN 2312. SSH VCN 2312 can include SSH subnet 2314 (e.g., SSH subnet 2114 in FIG. 21 ), which may be communicatively coupled to control plane VCN 2316 (e.g., control plane VCN 2116 in FIG. 21 ) via LPG 2310 included in control plane VCN 2316, and may be communicatively coupled to data plane VCN 2318 (e.g., data plane 2118 in FIG. 21 ) via LPG 2310 included in data plane VCN 2318. Control plane VCN 2316 and data plane VCN 2318 may be included in service tenancy 2319 (e.g., service tenant 2119 in FIG. 21 ).

[0194] Control plane VCN 2316 may include a control plane DMZ layer 2320 (e.g., control plane DMZ layer 2120 in FIG. 21 ) that may include a load balancer (LB) subnet 2322 (e.g., LB subnet 2122 in FIG. 21 ), a control plane app layer 2324 (e.g., control plane app layer 2124 in FIG. 21 ) that may include an app subnet 2326 (e.g., similar to app subnet 2126 in FIG. 21 ), and a control plane data layer 2328 (e.g., control plane data layer 2128 in FIG. 21 ) that may include a DB subnet 2330. LB subnet 2322 included in control plane DMZ layer 2320 may be communicatively coupled to app subnet 2326 included in control plane app layer 2324 and to an Internet gateway 2334 (e.g., Internet gateway 2134 in FIG. 21 ), which may be included in control plane VCN 2316. App subnet 2326 may be communicatively coupled to DB subnet 2330 included in control plane data layer 2328, and to service gateway 2336 (e.g., service gateway in FIG. 21 ) and network address translation (NAT) gateway 2338 (e.g., NAT gateway 2138 in FIG. 21 ). Control plane VCN 2316 may include service gateway 2336 and NAT gateway 2338.

[0195] Data plane VCN 2318 may include data plane app layer 2346 (e.g., data plane app layer 2146 in FIG. 21 ), data plane DMZ layer 2348 (e.g., data plane DMZ layer 2148 in FIG. 21 ), and data plane data layer 2350 (e.g., data plane data layer 2150 in FIG. 21 ). Data plane DMZ layer 2348 may include LB subnet 2322, which may be communicatively coupled to trusted app subnet 2360 and untrusted app subnet 2362 of data plane app layer 2346 and internet gateway 2334 included in data plane VCN 2318. Trusted app subnet 2360 may be communicatively coupled to service gateway 2336 included in data plane VCN 2318, NAT gateway 2338 included in data plane VCN 2318, and DB subnet 2330 included in data plane data layer 2350. The untrusted app subnet 2362 may be communicatively coupled to a service gateway 2336 included in the data plane VCN 2318 and to a DB subnet 2330 included in the data plane data layer 2350. The data plane data layer 2350 may include a DB subnet 2330 that may be communicatively coupled to a service gateway 2336 included in the data plane VCN 2318.

[0196] The untrusted app subnet 2362 may include one or more primary VNICs 2364(1)-(N), which may be communicatively coupled to tenant virtual machines (VMs) 2366(1)-(N). Each tenant VM 2366(1)-(N) may be communicatively coupled to a respective app subnet 2367(1)-(N), which may be included in a respective container egress VCN 2368(1)-(N), which may be included in a respective customer tenancy 2370(1)-(N). Each secondary VNIC 2372(1)-(N) may facilitate communication between the untrusted app subnet 2362 included in the data plane VCN 2318 and the app subnet included in the container egress VCN 2368(1)-(N). Each container egress VCN 2368(1)-(N) may include a NAT gateway 2338, which may be communicatively coupled to the public internet 2354 (e.g., public internet 2154 in FIG. 21 ).

[0197] The internet gateway 2334 included in the control plane VCN 2316 and the internet gateway 2334 included in the data plane VCN 2318 may be communicatively coupled to a metadata management service 2352 (e.g., metadata management system 2152 of FIG. 21 ), which may be communicatively coupled to the public internet 2354. The public internet 2354 may be communicatively coupled to a NAT gateway 2338 included in the control plane VCN 2316 and the NAT gateway 2338 included in the data plane VCN 2318. The service gateway 2336 included in the control plane VCN 2316 and the service gateway 2336 included in the data plane VCN 2318 may be communicatively coupled to cloud services 2356.

[0198] In some embodiments, data plane VCN 2318 may be integrated into customer tenancy 2370. This integration may be useful or desirable for an IaaS provider's customer in some cases, such as when they may want support when running their code. Customers may provide code that, when executed, may be disruptive, communicate with other customer resources, or cause undesirable effects. Thus, the IaaS provider can determine whether or not to run code that a customer has provided to the IaaS provider.

[0199] In some examples, an IaaS provider's customer can grant temporary network access to the IaaS provider and request a feature to be added to data plane app layer 2346. The code to perform the feature may run in VMs 2366(1)-(N) but cannot be configured to run elsewhere on data plane VCN 2318. Each VM 2366(1)-(N) may be connected to one customer tenancy 2370. Each container 2371(1)-(N) included in a VM 2366(1)-(N) may be configured to run code. In this case, double isolation may exist (e.g., container 2371(1)-(N) may run code, and container 2371(1)-(N) may be included in at least one VM 2366(1)-(N) included in untrusted app subnet 2362), which can help prevent erroneous or unwanted code from damaging the IaaS provider's network or from damaging a different customer's network. Containers 2371(1)-(N) may be communicatively coupled to customer tenancy 2370 and may be configured to send or receive data from customer tenancy 2370. Containers 2371(1)-(N) may not be configured to send or receive data from any other entity in data plane VCN 2318. Once code execution is complete, the IaaS provider can kill or discard containers 2371(I)-(N).

[0200] In some embodiments, trusted app subnet 2360 can execute code that may be owned or operated by the IaaS provider. In this embodiment, trusted app subnet 2360 can be communicatively coupled to DB subnet 2330 and configured to perform CRUD operations on DB subnet 2330. Untrusted app subnet 2362 can be communicatively coupled to DB subnet 2330, but in this embodiment, the untrusted app subnet can be configured to perform read operations within DB subnet 2330. Containers 2371(1)-(N) included in each customer's VMs 2366(1)-(N) and capable of executing code from the customer may not be communicatively coupled to DB subnet 2330.

[0201] In other embodiments, the control plane VCN 2316 and the data plane VCN 2318 may not be directly communicatively coupled. In this embodiment, there may be no direct communication between the control plane VCN 2316 and the data plane VCN 2318. However, there may be indirect communication by at least one method. An LPG 2310 may be established by an IaaS provider that can facilitate communication between the control plane VCN 2316 and the data plane VCN 2318. In another example, the control plane VCN 2316 or the data plane VCN 2318 can make a call to a cloud service 2356 through the service gateway 2336. For example, a call from the control plane VCN 2316 to the cloud service 2356 can include a request for a service that can communicate with the data plane VCN 2318.

[0202] 24 is a block diagram 2400 illustrating another example parameter of an IaaS architecture, according to at least one embodiment. A service operator 2402 (e.g., service operator 2102 of FIG. 21) may be communicatively coupled to a secure host tenancy 2404 (e.g., secure host tenancy 2104 of FIG. 21), which may include a virtual cloud network (VCN) 2406 (e.g., VCN 2106 of FIG. 21) and a secure host subnet 2408 (e.g., secure host subnet 2108 of FIG. 21). VCN 2406 may include an LPG 2410 (e.g., LPG 2110 of FIG. 21), which may be communicatively coupled to an SSH VCN 2412 (e.g., SSH VCN 2112 of FIG. 21) via an LPG 2410 included in SSH VCN 2412. SSH VCN 2412 can include SSH subnet 2414 (e.g., SSH subnet 2114 in FIG. 21 ), which may be communicatively coupled to control plane VCN 2416 (e.g., control plane VCN 2116 in FIG. 21 ) via LPG 2410 included in control plane VCN 2416, and may be communicatively coupled to data plane VCN 2418 (e.g., data plane 2118 in FIG. 21 ) via LPG 2410 included in data plane VCN 2418. Control plane VCN 2416 and data plane VCN 2418 may be included in service tenancy 2419 (e.g., service tenancy 2119 in FIG. 21 ).

[0203] Control plane VCN 2416 may include a control plane DMZ layer 2420 (e.g., control plane DMZ layer 2120 of FIG. 21 ) that may include a LB subnet 2422 (e.g., LB subnet 2122 of FIG. 21 ), a control plane app layer 2424 (e.g., control plane app layer 2124 of FIG. 21 ) that may include an app subnet 2426 (e.g., app subnet 2126 of FIG. 21 ), and a control plane data tier 2428 (e.g., control plane data tier 2128 of FIG. 21 ) that may include a DB subnet 2430 (e.g., DB subnet 2330 of FIG. 23 ). LB subnet 2422 included in control plane DMZ layer 2420 may be communicatively coupled to app subnet 2426 included in control plane app layer 2424 and to an Internet gateway 2434 (e.g., Internet gateway 2134 of FIG. 21 ), which may be included in control plane VCN 2416. The app subnet 2426 may be communicatively coupled to a DB subnet 2430 included in the control plane data layer 2428, and to a service gateway 2436 (e.g., the service gateway in FIG. 21 ) and a network address translation (NAT) gateway 2438 (e.g., the NAT gateway 2138 in FIG. 21 ). The control plane VCN 2416 may include the service gateway 2436 and the NAT gateway 2438.

[0204] Data plane VCN 2418 may include a data plane app layer 2446 (e.g., data plane app layer 2146 in FIG. 21 ), a data plane DMZ layer 2448 (e.g., data plane DMZ layer 2148 in FIG. 21 ), and a data plane data layer 2450 (e.g., data plane data layer 2150 in FIG. 21 ). Data plane DMZ layer 2448 may include trusted app subnet 2460 (e.g., trusted app subnet 2360 in FIG. 23 ) and untrusted app subnet 2462 (e.g., untrusted app subnet 2362 in FIG. 23 ) of data plane app layer 2446 and LB subnet 2422, which may be communicatively coupled to Internet gateway 2434 included in data plane VCN 2418. The trusted app subnet 2460 may be communicatively coupled to a service gateway 2436 included in the data plane VCN 2418, a NAT gateway 2438 included in the data plane VCN 2418, and a DB subnet 2430 included in the data plane data layer 2450. The untrusted app subnet 2462 may be communicatively coupled to the service gateway 2436 included in the data plane VCN 2418 and a DB subnet 2430 included in the data plane data layer 2450. The data plane data layer 2450 may include a DB subnet 2430 that may be communicatively coupled to the service gateway 2436 included in the data plane VCN 2418.

[0205] The untrusted app subnet 2462 may include primary VNICs 2464(1)-(N), which may be communicatively coupled to tenant virtual machines (VMs) 2466(1)-(N) residing in the untrusted app subnet 2462. Each tenant VM 2466(1)-(N) may execute code in a respective container 2467(1)-(N), which may be communicatively coupled to an app subnet 2426, which may be included in a data plane app layer 2446, which may be included in a container egress VCN 2468. Each secondary VNIC 2472(1)-(N) may facilitate communication between the untrusted app subnet 2462, which is included in the data plane VCN 2418, and the app subnet included in the container egress VCN 2468. The container egress VCN may include a NAT gateway 2438, which may be communicatively coupled to the public internet 2454 (e.g., public internet 2154 in FIG. 21 ).

[0206] An internet gateway 2434 included in the control plane VCN 2416 and the data plane VCN 2418 may be communicatively coupled to a metadata management service 2452 (e.g., metadata management system 2152 of FIG. 21 ), which may be communicatively coupled to the public internet 2454. The public internet 2454 may be communicatively coupled to a NAT gateway 2438 included in the control plane VCN 2416 and the data plane VCN 2418. A service gateway 2436 included in the control plane VCN 2416 and the data plane VCN 2418 may be communicatively coupled to cloud services 2456.

[0207] In some examples, the pattern illustrated by the architecture of block diagram 2400 in FIG. 24 may be considered an exception to the pattern illustrated by the architecture of block diagram 2300 in FIG. 23 and may be desirable for an IaaS provider's customers when the IaaS provider cannot communicate directly with the customers (e.g., in unconnected regions). The customers can access each of the containers 2467(1)-(N) included in each customer's VMs 2466(1)-(N) in real time. The containers 2467(1)-(N) may be configured to call each of the secondary VNICs 2472(1)-(N) included in the app subnet 2426 of the data plane app layer 2446, which may be included in the container egress VCN 2468. The secondary VNICs 2472(1)-(N) can send the call to the NAT gateway 2438, which can send the call to the public Internet 2454. In this example, containers 2467(1)-(N) that a customer can access in real time may be isolated from control plane VCN 2416 and may be isolated from other entities included in data plane VCN 2418. Containers 2467(1)-(N) may also be isolated from resources of other customers.

[0208] In another example, a customer can invoke cloud service 2456 using containers 2467(1)-(N). In this example, the customer can execute code in containers 2467(1)-(N) that requests a service from cloud service 2456. Containers 2467(1)-(N) can send the request to secondary VNICs 2472(1)-(N), which can send the request to a NAT gateway that can send the request to public internet 2454. Public internet 2454 can send the request to LB subnet 2422, which is included in control plane VCN 2416, via internet gateway 2434. In response to determining that the request is valid, LB subnet 2426 can send the request to app subnet 2426, which can send the request to cloud service 2456 via service gateway 2436.

[0209] It should be noted that the illustrated IaaS architectures 2100, 2200, 2300, and 2400 may include elements other than those shown. Furthermore, the illustrated embodiments are only examples of some of the cloud infrastructure systems that may incorporate embodiments of the present disclosure. In other embodiments, the IaaS system may have more or fewer elements than those shown, may combine two or more elements, or may have a different configuration or arrangement of elements.

[0210] In certain embodiments, the IaaS system described herein may include a suite of application, middleware, and database services that are delivered to customers in a self-service, subscription-based, elastically scalable, reliable, highly available, and secure manner. One example of such an IaaS system is the Oracle® Cloud Infrastructure (OCI) offered by the present applicant.

[0211] 25 illustrates an exemplary computer system 2500 upon which various embodiments may be implemented. System 2500 may be used to implement any of the computer systems described above. As shown, computer system 2500 includes a processing unit 2504 that communicates with a number of peripheral subsystems via a bus subsystem 2502. These peripheral subsystems may include a processing acceleration unit 2506, an I / O subsystem 2508, a storage subsystem 2518, and a communications subsystem 2524. Storage subsystem 2518 includes a tangible computer-readable storage medium 2522 and a system memory 2510.

[0212] Bus subsystem 2502 provides a mechanism for allowing the various components and subsystems of computer system 2500 to communicate with each other as intended. While bus subsystem 2502 is shown schematically as a single bus, alternative embodiments of the bus subsystem may utilize multiple buses. Bus subsystem 2502 may be any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. For example, such architectures may include an Industry Standard Architecture (ISA) bus, a MicroChannel Architecture (MCA), an Bus Extended ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus, which may be implemented as a mezzanine bus manufactured in accordance with the IEEE P1386.1 standard.

[0213] Processing unit 2504, which may be implemented as one or more integrated circuits (e.g., conventional microprocessors or microcontrollers), controls the operation of computer system 2500. Processing unit 2504 may include one or more processors. These processors may include single-core or multi-core processors. In some embodiments, processing unit 2504 may be implemented as one or more independent processing units 2532 and / or 2534, with each processing unit including a single-core or multi-core processor. In other embodiments, processing unit 2504 may be implemented as a quad-core processing unit formed by integrating two dual-core processors into a single chip.

[0214] In various embodiments, processing unit 2504 may execute various programs in response to program code and may maintain multiple programs or processes executing simultaneously. At any given time, some or all of the program code being executed may reside in processor 2504 and / or storage subsystem 2518. Processor 2504, through appropriate programming, may provide the various functionality discussed above. Computer system 2500 may further include a processing acceleration unit 2506, which may include a digital signal processor (DSP), a special purpose processor, and / or the like.

[0215] The I / O subsystem 2508 may include user interface input devices and user interface output devices. User interface input devices may include a keyboard, a pointing device such as a mouse or trackball, a touchpad or touchscreen integrated into a display, a scroll wheel, a click wheel, a dial, buttons, switches, a keypad, a voice input device with a voice command recognition system, a microphone, and other types of input devices. User interface input devices may also include a motion detection and / or gesture recognizer, such as a Microsoft Kinect® motion sensor. The Microsoft Kinect® motion sensor can control and interact with input devices such as a Microsoft Xbox® 360 game controller through a natural user interface (NUI) that utilizes gestures and voice commands. User interface input devices may also include an eye gesture recognizer, such as a Google Glass® blink detector. The Google Glass® blink detector detects a user's eye activity (e.g., "blinks" when taking a picture and / or selecting a menu) and translates the eye activity into input for input into an input device (e.g., Google Glass®). Additionally, the user interface input device may include a voice recognition detection device that allows a user to interact with a voice recognition system (e.g., Siri® Navigator) via voice commands.

[0216] User interface input devices may also include, but are not limited to, three-dimensional (3D) mice, joysticks or pointing sticks, gamepads, graphics tablets, audio / visual devices such as speakers, digital cameras, digital video cameras, portable media players, webcams, image scanners, fingerprint scanners, barcode readers, 3D scanners, 3D printers, laser range finders, and eye-tracking devices. User interface input devices may also include medical imaging input devices, such as computed tomography (CT) scanners, magnetic resonance imaging (MRI) scanners, ultrasound emission tomography (EMC) scanners, or medical ultrasound scanners. User interface input devices may also include audio input devices, such as MIDI keyboards and electronic musical instruments.

[0217] User interface output devices may include non-visual displays such as a display subsystem, indicator lights, or audio output devices. The display subsystem may be, for example, a flat-panel device using a cathode ray tube (CRT), liquid crystal display (LCD), or plasma display, a projection device, or a touchscreen. In general, the term "output device" is intended to include all possible types of devices and mechanisms for outputting information from computer system 2500 to a user or to another computer. For example, user interface output devices include, but are not limited to, various display devices that visually convey text, images, and audio / video information, such as monitors, printers, speakers, headphones, car navigation systems, plotters, voice output devices, and modems.

[0218] Computer system 2500 may include a storage subsystem 2518. Storage subsystem 2518 comprises software elements, which are illustratively located in system memory 2510. System memory 2510 may store program instructions loadable and executable by processing unit 2504, as well as data generated by the execution of these programs.

[0219] Depending on the configuration and type of computer system 2500, the system memory 2510 may be volatile memory (e.g., random access memory (RAM)) and / or non-volatile memory (e.g., read-only memory (ROM), flash memory). Generally, RAM contains data and / or program modules that are immediately accessible to and / or currently being operated on and executed by the processing unit 2504. In some implementations, the system memory 2510 may include several different types of memory, such as static random access memory (SRAM) or dynamic random access memory (DRAM). In some implementations, a basic input / output system (BIOS), containing the basic routines that help to transfer information between elements within the computer system 2500, such as during start-up, may typically be stored in ROM. By way of example and not limitation, system memory 2510 also shows application programs 2512, program data 2514, and an operating system 2516, which may include client applications, web browsers, mid-tier applications, relational database management systems (RDBMS), and the like.By way of example, operating system 2516 may include various versions of the Microsoft Windows®, Apple Macintosh® and / or Linux® operating systems, various commercially available UNIX® or UNIX-like operating systems (including, but not limited to, various GNU / Linux operating systems, Google Chrome® OS, etc.), and / or mobile operating systems such as iOS, Windows® Phone, Android® OS, BlackBerry® 25 OS, and Palm® OS operating systems.

[0220] Additionally, storage subsystem 2518 may provide a tangible, computer-readable storage medium for storing the basic programming and data constructs that provide the functionality of some embodiments. Software (programs, code modules, instructions) that, when executed by a processor, provide the above-described functionality may be stored in storage subsystem 2518. These software modules or instructions may be executed by processing unit 2504. Storage subsystem 2518 may also provide a repository for storing data used in accordance with the present disclosure.

[0221] Storage subsystem 2500 may also include computer-readable storage medium reader 2520 further connectable to computer-readable storage medium 2522. Computer-readable storage medium 2522 may comprehensively represent remote, local, fixed, and / or removable storage devices, as well as storage media for temporarily and / or permanently containing, storing, transmitting, and retrieving computer-readable information, together with, or optionally in combination with, system memory 2510.

[0222] Additionally, the computer-readable storage medium 2522 containing the code or portions of code may include any suitable medium known or used in the art, including storage and communication media such as, but not limited to, volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storing and / or transmitting information. This may include tangible computer-readable storage media such as RAM, ROM, electronically erasable programmable ROM (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical storage devices, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or other tangible computer-readable media. This may also include intangible computer-readable media such as data signals, data transmissions, or other media usable to transmit the desired information and accessible by computer system 2500.

[0223] By way of example, computer-readable storage medium 2522 may include a hard disk drive that reads from or writes to non-removable, non-volatile magnetic media, a magnetic disk drive that reads from or writes to removable, non-volatile magnetic disks, and an optical disk drive that reads from or writes to removable, non-volatile optical disks such as CD-ROMs, DVDs, and Blu-ray disks or other optical media. Computer-readable storage medium 2522 may include, but is not limited to, Zip drives, flash memory cards, universal serial bus (USB) flash drives, secure digital (SD) cards, DVD disks, digital video tapes, etc. The computer-readable storage media 2522 may also include flash memory-based SSDs, enterprise flash drives, solid-state drives (SSDs) based on non-volatile memory such as solid-state ROM, SSDs based on volatile memory such as solid-state RAM, dynamic RAM, static RAM, DRAM-based SSDs, magnetoresistive RAM (MRAM) SSDs, and hybrid SSDs using a combination of DRAM and flash memory-based SSDs. Disk drives and their associated computer-readable media can provide non-volatile storage of computer-readable instructions, data structures, program modules, and other data for the computer system 2500.

[0224] The communications subsystem 2524 provides an interface with other computer systems and networks. The communications subsystem 2524 serves as an interface for receiving data from other systems and transmitting data from the computer system 2500 to other systems. For example, the communications subsystem 2524 may enable the computer system 2500 to connect to one or more devices via the Internet. In some embodiments, the communications subsystem 2524 may include radio frequency (RF) transceiver components for accessing wireless voice and / or data networks (e.g., using cellular technologies, advanced data network technologies such as 3G, 4G, or EDGE (enhanced data rates for global evolution)), WiFi (IEEE 802.11 family of standards or other mobile communications technologies, or any combination thereof), global positioning system (GPS) receiver components, and / or other components. In some embodiments, the communications subsystem 2524 may provide a wired network connection (e.g., Ethernet) in addition to or instead of a wireless interface.

[0225] Additionally, in some embodiments, the communications subsystem 2524 may receive incoming communications in the form of structured and / or unstructured data feeds 2526, event streams 2528, event updates 2530, etc., on behalf of one or more users who may be using the computer system 2500.

[0226] As an example, the communications subsystem 2524 may be configured to receive data feeds 2526, such as web feeds such as Twitter® feeds, Facebook® updates, Rich Site Summary (RSS) feeds, etc., in real time from users of social networks and / or other communications services, and / or to receive real-time updates from one or more third-party sources.

[0227] Communications subsystem 2524 may also be configured to receive data in the form of a continuous data stream, which may include an event stream 2528 of real-time events that may be continuous or essentially unbounded with no clear ends and / or event updates 2530. Examples of applications that generate continuous data may include, for example, sensor data applications, financial tickers, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, automobile traffic monitoring, etc.

[0228] The communications subsystem 2524 may also be configured to output structured and / or unstructured data feeds 2526, event streams 2528, event updates 2530, etc. to one or more databases that may communicate with one or more streaming data source computers coupled to the computer system 2500.

[0229] The computer system 2500 may be one of a variety of types, including a handheld mobile device (e.g., an iPhone® mobile phone, an iPad® computing tablet, a PDA), a wearable device (e.g., a Google Glass® head-mounted display), a PC, a workstation, a mainframe, a kiosk, a server rack, or other data processing system.

[0230] In the foregoing description, for purposes of explanation, specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure. However, it will be apparent that various embodiments can be practiced without these specific details. The following description provides examples only and is not intended to limit the scope, applicability, or configuration of the present disclosure. Rather, the following description of the embodiments provides one of ordinary skill in the art with an enabling description for practicing the embodiments. It should be understood that various changes can be made in the function and arrangement of elements without departing from the spirit and scope of the present disclosure, as defined by the appended claims. The drawings and descriptions are not intended to be limiting. Circuits, systems, networks, processes, and other components may be shown as components in block diagram form so as not to obscure the embodiments in unnecessary detail. In other embodiments, well-known circuits, processes, algorithms, structures, and techniques may be shown without unnecessary detail so as not to obscure the embodiments. The teachings of the present disclosure may also be applied to various types of applications, such as mobile applications, non-mobile applications, desktop applications, web applications, enterprise applications, etc. Additionally, the teachings of the present disclosure are not limited to a particular operating environment (eg, operating system, device, platform, etc.) but may be applied to multiple different operating environments.

[0231] Also, note that each embodiment is described as a process that is depicted as a flowchart, flow diagram, data flow diagram, structure diagram, or block diagram. While a flowchart describes operations as sequential processes, many operations may be performed in parallel or simultaneously. Moreover, the order of operations may be rearranged. A process terminates when the operation is completed, but may include additional steps not shown in the figures. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination may correspond to a return of the calling function or the main function.

[0232] The words "example" and "exemplary" are used herein to mean "serving as an example, instance, or illustration." Any embodiment or design described herein as "exemplary" or "example" is not necessarily to be construed as preferred or advantageous over other embodiments or designs.

[0233] The terms “machine-readable storage medium” or “computer-readable storage medium” include, but are not limited to, portable or non-portable storage devices, optical storage devices, and various other media capable of storing, storing, or transporting instructions and / or data. Machine-readable storage media or computer-readable storage media may also include non-transitory media capable of storing data and that do not involve carrier waves and / or transitory electronic signals propagating via wireless or wired connections. Examples of non-transitory media may include, but are not limited to, magnetic disks or tapes, optical storage media such as compact discs (CDs) or digital versatile discs (DVDs), flash memory, memory, or memory devices. A computer program product may include code and / or machine-executable instructions, which may represent any combination of procedures, functions, subprograms, programs, routines, subroutines, modules, software packages, classes, instructions, data structures, or program statements. A code segment may be coupled to another code segment or a hardware circuit by transferring and / or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc. may be conveyed, forwarded, or transmitted via any suitable means such as memory sharing, message passing, token passing, network transmission, etc.

[0234] Furthermore, embodiments may be implemented by hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware, or microcode, program code or code segments (e.g., a computer program product) that perform the necessary operations may be stored on a machine-readable medium. A processor may perform the necessary operations. The systems illustrated in some of the figures may be provided in various configurations. In some examples, the system may be configured as a distributed system in which one or more elements of the system are distributed across one or more networks in a cloud computing system. When an element is described as being "configured" to perform particular operations, such configuration may be achieved, for example, by designing electronic circuitry or other hardware to perform the operations, by programming or controlling electronic circuitry (e.g., a microprocessor or other suitable electronic circuitry) to perform the operations, or by any combination thereof.

[0235] While specific embodiments of the present disclosure have been described, various modifications, variations, alternative configurations, and equivalents are encompassed within the scope of the present disclosure. The embodiments of the present disclosure are not limited to operating in a particular data processing environment, but can freely operate in multiple data processing environments. Furthermore, while embodiments of the present disclosure have been described using a particular series of actions and steps, it will be apparent to those skilled in the art that the scope of the present disclosure is not limited to the series of actions and steps described. Various features and aspects of the above-described embodiments can be used individually or jointly.

[0236] Furthermore, while embodiments of the present disclosure have been described using a particular combination of hardware and software, it should be appreciated that other combinations of hardware and software are within the scope of the present disclosure. Embodiments of the present disclosure may be implemented using only hardware, only software, or a combination thereof. The various processes described in this disclosure may run on the same processor or any combination of different processors. Thus, when a component or module is described as being configured to perform a particular process, that configuration may be achieved, for example, by designing electronic circuitry to perform the process, by programming a programmable electronic circuit (such as a microprocessor) to perform the process, or a combination thereof. Processes may communicate using a variety of techniques, including, but not limited to, conventional techniques for inter-process communication. Different pairs of processes may use different techniques, or the same pair of processes may use different techniques at different times.

[0237] Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense. It will be apparent, however, that additions, subtractions, deletions, and other modifications and changes may be made without departing from the broad spirit and scope defined by the appended claims. Accordingly, while specific embodiments of the present disclosure have been described, these embodiments are not intended to be limiting. Various modifications and equivalents thereof are intended to be encompassed within the scope of the appended claims.

[0238] The indefinite articles "a" / "an," the definite article "the," and similar references used in the context of describing this disclosure (particularly in the context of the claims) should be construed to include both the singular and the plural unless otherwise stated herein or the context clearly indicates otherwise. The terms "comprising," "having," "including," and "containing" should be construed as open-ended terms (i.e., meaning "including, but not limited to") unless otherwise stated. The term "connected" should be construed as partly or wholly contained within, attached to, or joined together, even if there is something intervening. The recitation of ranges of values ​​herein is intended merely as a shorthand method of referring to each individual value falling within the range, and unless otherwise stated herein, each individual value is incorporated herein as if set forth individually herein. Unless otherwise stated herein or the context clearly indicates otherwise, all methods described herein can be performed in any suitable order. The use of any and all examples or exemplary language (e.g., "such as") herein is intended to further clarify embodiments of the present disclosure and does not limit the scope of the disclosure unless otherwise specified. No language in the specification should be construed as indicating any non-claimed element essential to the practice of the disclosure.

[0239] Disjunctive language, such as the phrase "at least one of X, Y, or Z," is intended to be understood in context as generally used to indicate that an item, term, etc. may be either X, Y, or Z, or any combination thereof (e.g., X, Y, and / or Z), unless otherwise specified. Thus, such disjunctive language is not generally intended to, and does not imply, that a particular embodiment requires that at least one of X, at least one of Y, or at least one of Z be present.

[0240] Examples of embodiments of the present disclosure can be described in terms of the following clauses: Item 1. A method, comprising: a virtual network interface card hosted on a network virtualization device receiving a first packet from a compute instance associated with the virtual network interface card; the virtual network interface card determining that flow information for transmitting the first packet on a virtual network is unavailable from a memory of the network virtualization device; and the virtual network interface card transmitting the first packet to a network interface service via the network virtualization device, the network interface service maintaining configuration information for transmitting packets on the virtual network and configured to transmit the first packet on the virtual network based on the configuration information; the method further comprising the network virtualization device receiving the flow information from the network interface service, the flow information being a subset of the configuration information; and the method further comprising the network virtualization device storing the flow information in the memory.

[0241] Clause 2. The method of clause 1, further comprising: the virtual network interface card receiving a second packet from the compute instance; the virtual network interface card determining that the flow information is available from the memory for transmitting the second packet on the virtual network; and the virtual network interface card transmitting the second packet on the virtual network based on the flow information via the network virtualization device.

[0242] Clause 3. The method of any one of clauses 1 and 2, wherein the configuration information includes a security policy, an overlay-to-board Internet Protocol (IP) address mapping, and a route rule for one or more packet flows, and the flow information includes at least one of a security policy, an overlay-to-board IP address mapping, or a route rule.

[0243] Clause 4. The method of any one of clauses 1 to 3, wherein the flow information is received as a cache entry and stored in a cache associated with the virtual network interface card, and the flow information is determined to be unavailable based on a cache miss.

[0244] Clause 5. The method of clause 4, further comprising the network virtualization device receiving an indicator of a version of the flow information from the network interface service, and the network virtualization device storing the indicator in the cache.

[0245] Clause 6. The method of clause 4, wherein the flow information includes multiple parts, and the method further includes the network virtualization device receiving a version indicator for each part of the flow information from the network interface service, and the network virtualization device storing the indicator in the cache.

[0246] Clause 7. The method of any one of clauses 1 to 6, further comprising: the network virtualization device storing version information associated with the flow information, the version information being stored in the memory; the method further comprising: the virtual network interface card receiving a second packet from the compute instance; the virtual network interface card determining that the flow information is available for sending the second packet on the virtual network; the virtual network interface card determining, based on the version information, that at least a portion of the flow information is out of date; and the virtual network interface card sending the second packet to the network interface service via the network virtualization device.

[0247] Clause 8. The method of clause 7, wherein the network virtualization device further includes receiving updated flow information and updated version information, the updated flow information including an update that replaces at least the portion of the flow information, and the method further includes the network virtualization device storing the updated flow information and the updated version information in the memory, the virtual network interface card receiving a third packet from the compute instance, and the virtual network interface card transmitting the third packet on the virtual network via the network virtualization device based on the updated flow information and the updated version information.

[0248] Clause 9. The method of any one of clauses 1 to 6, further comprising: the network virtualization device storing version information associated with the flow information, the version information being stored in the memory; the method further comprising: the virtual network interface card determining, before receiving a second packet to which the flow information is applicable, that at least a portion of the flow information is outdated based on the version information; the network virtualization device requesting updated flow information before receiving the second packet; the network virtualization device receiving the updated flow information and updated version information, the updated flow information including an update that replaces at least the portion of the flow information; the method further comprising: the virtual network interface card receiving the second packet from the compute instance; and the virtual network interface card transmitting, via the network virtualization device, the second packet on the virtual network based on the updated flow information and the updated version information.

[0249] Clause 10. The method of any one of clauses 1 to 6, wherein the network virtualization device further includes receiving an indication from the network interface service that at least a portion of the flow information is stale; the virtual network interface card receiving updated flow information before receiving a second packet to which the flow information is applicable; and the network virtualization device storing the updated flow information in the memory, the updated flow information including an update that replaces at least the portion of the flow information; and the method further includes the virtual network interface card receiving the second packet from the compute instance; and the virtual network interface card transmitting the second packet on the virtual network via the network virtualization device based on the updated flow information.

[0250] Clause 11. The method of any one of clauses 1 to 6, further comprising: the network virtualization device receiving an indication from the network interface service that at least a portion of the flow information is stale; the virtual network interface card receiving a second packet to which the flow information is applicable; the virtual network interface card sending the second packet to the network interface service via the network virtualization device; the network virtualization device receiving updated flow information; the network virtualization device storing the updated flow information in the memory; the virtual network interface card receiving a third packet from the compute instance; and the virtual network interface card sending the third packet on the virtual network based on the updated flow information via the network virtualization device.

[0251] Clause 12. The method of any one of clauses 1 to 6, wherein the method further includes: the virtual network interface card receiving a second packet from the virtual network destined for the compute instance; the virtual network interface card determining that second flow information for sending the second packet to the compute instance is unavailable from the memory of the network virtualization device; the virtual network interface card sending the second packet to the network interface service via the network virtualization device; and the virtual network interface card receiving a third packet destined for the compute instance from the network interface service via the network virtualization device, wherein the third packet corresponds to an updated version of the second packet based on a second subset of the configuration information; and the method further includes the virtual network interface card sending the third packet to the compute instance.

[0252] Clause 13. The method of clause 12, further comprising: the network virtualization device receiving the second flow information, the second flow information being the second subset of the configuration information; the method further comprising: the network virtualization device storing the second flow information in the memory; the virtual network interface card receiving a fourth packet destined for the compute instance from the virtual network via the network virtualization device; the virtual network interface card determining that the second flow information is available from the memory for transmitting the fourth packet to the compute instance; and the virtual network interface card transmitting the fourth packet on the compute instance based on the second flow information.

[0253] Item 14. A system comprising: a network virtualization device hosting a virtual network interface card and connected to a substrate network; a host machine connected to the network virtualization device and hosting a compute instance; and a set of servers connected to the substrate network and hosting a network interface service, wherein the virtual network interface card is configured to receive a first packet from the compute instance and determine that flow information for transmitting the first packet on a virtual network is unavailable from a memory of the network virtualization device, the virtual network being an overlay on the substrate network; the virtual network interface card is further configured to transmit the first packet to the network interface service via the network virtualization device; the network interface service is configured to maintain configuration information for transmitting packets on the virtual network, transmit the first packet on the virtual network based on the configuration information, and generate the flow information based on the configuration information; and the network virtualization device is configured to receive the flow information from the network interface service and store the flow information in the memory.

[0254] Item 15. The system of Item 14, wherein maintaining the configuration information includes tracking configuration information and version information associated with the configuration information or portions of the configuration information for each virtual network interface card hosted on the network virtualization device.

[0255] Clause 16. The system of any one of clauses 14 and 15, wherein the virtual network interface card is further configured to receive a second packet from the compute instance, determine that the flow information is available for sending the second packet on the virtual network, determine that at least a portion of the flow information is out of date based on version information associated with the flow information and stored in the memory of the network virtualization device, and send the second packet to the network interface service via the network virtualization device.

[0256] Clause 17. The system of any one of clauses 14 to 16, wherein the network virtualization device is further configured to receive updated flow information and updated version information, the updated flow information including an update that replaces at least a portion of the flow information, and the network virtualization device is further configured to store the updated flow information and the updated version information in the memory, and is configured to receive a third packet of the computation instance from the virtual network interface card, and transmit the third packet on the virtual network based on the updated flow information and the updated version information.

[0257] Item 18. One or more non-transitory computer-readable instructions that, when executed on a system, cause the system to perform operations, the operations including: maintaining, by a network interface service, a set of configuration information for sending packets over a virtual network; and receiving, by the network interface service, a first packet of a compute instance, the first packet being received from a virtual network interface card hosted on a network virtualization device and associated with the compute instance; the operations further including: determining, by the network interface service, configuration information from the set for sending the first packet over the virtual network; sending, by the network interface service, the first packet over the virtual network based on the configuration information; and sending, by the network interface service, flow information to the network virtualization device, the flow information being a subset of the configuration information and enabling the virtual network interface card to send additional packets received from the compute instance over the virtual network.

[0258] Clause 19. The one or more non-transitory computer-readable instructions of clause 18, wherein the operations further include determining that at least a portion of the flow information is stale based on version information associated with the flow information; sending an indication to the network virtualization device that at least the portion of the flow information is stale; and pushing an update to at least the portion of the flow information to the network virtualization device.

[0259] Clause 20. The one or more non-transitory computer-readable instructions of clause 18, wherein the operations further include: determining that at least a portion of the flow information is stale based on version information associated with the flow information; sending an indication to the network virtualization device that at least the portion of the flow information is stale; receiving a second packet to which the flow information applies from the virtual network interface card via the network virtualization device; sending the second packet on the virtual network; and sending an update to at least the portion of the flow information to the network virtualization device.

[0260] Preferred embodiments of the present disclosure are described herein, including the best mode known for carrying out the disclosure. Variations of these preferred embodiments will become apparent to those skilled in the art upon reading the foregoing description. Those skilled in the art can employ such variations as appropriate, and the present disclosure may be practiced otherwise than as specifically described herein. Accordingly, this disclosure includes all modifications and equivalents of the subject matter recited in the claims appended hereto as permitted by applicable law. Moreover, any combination of the above-described elements in all possible variations thereof is encompassed by this disclosure unless otherwise indicated herein.

[0261] All references cited herein, including publications, patent applications, and patents, are incorporated by reference to the same extent as if each individual reference was individually and specifically indicated to be incorporated by reference and was set forth in its entirety herein.

[0262] In the foregoing specification, aspects of the disclosure have been described with reference to specific embodiments thereof, but those skilled in the art will recognize that the disclosure is not limited thereto. Various features and aspects of the above-described disclosure may be used individually or jointly. Moreover, the embodiments can be utilized in any number of environments and applications beyond those described herein without departing from the broader spirit and scope of the specification. Accordingly, the specification and drawings should be regarded in an illustrative rather than a restrictive sense.< / realm>

Claims

1. 1. A method comprising: receiving a first packet from a compute instance associated with a virtual network interface card hosted on a network virtualization device connected to a substrate network; determining, by the virtual network interface card, that flow information for transmitting the first packet on a virtual network is unavailable from a memory of the network virtualization device, the virtual network being an overlay on the substrate network, the method further comprising: the virtual network interface card sending the first packet to a network interface service via the network virtualization device, the network interface service configured to maintain configuration information for sending packets on the virtual network, send the first packet on the virtual network based on the configuration information, and generate the flow information based on the configuration information, the method further comprising: The method further comprises receiving the flow information from the network interface service, the flow information being a subset of the configuration information, the method further comprising: The method includes the network virtualization device storing the flow information in the memory.

2. A method comprising: receiving a first packet from a compute instance associated with a virtual network interface card hosted on a network virtualization device; determining, by the virtual network interface card, that flow information for transmitting the first packet on a virtual network is unavailable from a memory of the network virtualization device; the virtual network interface card transmitting the first packet to a network interface service via the network virtualization device, the network interface service configured to maintain configuration information for transmitting packets on the virtual network and to transmit the first packet on the virtual network based on the configuration information, the method further comprising: The method further comprises receiving the flow information from the network interface service, the flow information being a subset of the configuration information, the method further comprising: the network virtualization device storing the flow information in the memory; The method, wherein the configuration information includes a security policy, an overlay-to-board Internet Protocol (IP) address mapping, and a route rule for one or more packet flows, and the flow information includes at least one of a security policy, an overlay-to-board IP address mapping, or a route rule.

3. receiving a second packet from the compute instance by the virtual network interface card; the virtual network interface card determining that the flow information is available from the memory for transmitting the second packet on the virtual network; The method of claim 1 or 2, further comprising: the virtual network interface card transmitting, via the network virtualization device, the second packet on the virtual network based on the flow information.

4. A method comprising: receiving a first packet from a compute instance associated with a virtual network interface card hosted on a network virtualization device; determining, by the virtual network interface card, that flow information for transmitting the first packet on a virtual network is unavailable from a memory of the network virtualization device; the virtual network interface card transmitting the first packet to a network interface service via the network virtualization device, the network interface service configured to maintain configuration information for transmitting packets on the virtual network and to transmit the first packet on the virtual network based on the configuration information, the method further comprising: The method further comprises receiving the flow information from the network interface service, the flow information being a subset of the configuration information, the method further comprising: the network virtualization device storing the flow information in the memory; The flow information is received as a cache entry and stored in a cache associated with the virtual network interface card, and the flow information is determined to be unavailable based on a cache miss, the method further comprising: receiving, by the network virtualization device, an indicator of a version of the flow information from the network interface service; the network virtualization device storing the indicator in the cache.

5. A method comprising: receiving a first packet from a compute instance associated with a virtual network interface card hosted on a network virtualization device; determining, by the virtual network interface card, that flow information for transmitting the first packet on a virtual network is unavailable from a memory of the network virtualization device; the virtual network interface card transmitting the first packet to a network interface service via the network virtualization device, the network interface service configured to maintain configuration information for transmitting packets on the virtual network and to transmit the first packet on the virtual network based on the configuration information, the method further comprising: The method further comprises receiving the flow information from the network interface service, the flow information being a subset of the configuration information, the method further comprising: the network virtualization device storing the flow information in the memory; the flow information is received as a cache entry and stored in a cache associated with the virtual network interface card, and the flow information is determined to be unavailable based on a cache miss; The flow information includes a plurality of parts, and the method further comprises: receiving, by the network virtualization device, from the network interface service, a version indicator for each portion of the flow information; the network virtualization device storing the indicator in the cache.

6. A method comprising: receiving a first packet from a compute instance associated with a virtual network interface card hosted on a network virtualization device; determining, by the virtual network interface card, that flow information for transmitting the first packet on a virtual network is unavailable from a memory of the network virtualization device; the virtual network interface card transmitting the first packet to a network interface service via the network virtualization device, the network interface service configured to maintain configuration information for transmitting packets on the virtual network and to transmit the first packet on the virtual network based on the configuration information, the method further comprising: The method further comprises receiving the flow information from the network interface service, the flow information being a subset of the configuration information, the method further comprising: the network virtualization device storing the flow information in the memory; and storing, by the network virtualization device, version information associated with the flow information, wherein the version information is stored in the memory, the method further comprising: receiving a second packet from the compute instance by the virtual network interface card; the virtual network interface card determining that the flow information is available for transmitting the second packet on the virtual network; The virtual network interface card determines, based on the version information, that at least a portion of the flow information is old; the virtual network interface card sending the second packet to the network interface service via the network virtualization device.

7. The method further comprises: and the network virtualization device receiving updated flow information and updated version information, the updated flow information including an update that replaces at least the portion of the flow information, the method further comprising: the network virtualization device storing the updated flow information and the updated version information in the memory; receiving a third packet from the compute instance by the virtual network interface card; and transmitting, via the network virtualization device, the third packet over the virtual network based on the updated flow information and the updated version information.

8. A method comprising: receiving a first packet from a compute instance associated with a virtual network interface card hosted on a network virtualization device; determining, by the virtual network interface card, that flow information for transmitting the first packet on a virtual network is unavailable from a memory of the network virtualization device; the virtual network interface card transmitting the first packet to a network interface service via the network virtualization device, the network interface service configured to maintain configuration information for transmitting packets on the virtual network and to transmit the first packet on the virtual network based on the configuration information, the method further comprising: The method further comprises receiving the flow information from the network interface service, the flow information being a subset of the configuration information, the method further comprising: the network virtualization device storing the flow information in the memory; and storing, by the network virtualization device, version information associated with the flow information, wherein the version information is stored in the memory, the method further comprising: determining, based on the version information, that at least a portion of the flow information is out of date before the virtual network interface card receives a second packet to which the flow information is applicable; the network virtualization device requesting updated flow information before receiving the second packet; receiving, by the network virtualization device, the updated flow information and updated version information, wherein the updated flow information includes an update that replaces at least the portion of the flow information; receiving the second packet from the compute instance by the virtual network interface card; and transmitting, via the network virtualization device, the second packet over the virtual network based on the updated flow information and the updated version information.

9. A method comprising: receiving a first packet from a compute instance associated with a virtual network interface card hosted on a network virtualization device; determining, by the virtual network interface card, that flow information for transmitting the first packet on a virtual network is unavailable from a memory of the network virtualization device; the virtual network interface card transmitting the first packet to a network interface service via the network virtualization device, the network interface service configured to maintain configuration information for transmitting packets on the virtual network and to transmit the first packet on the virtual network based on the configuration information, the method further comprising: The method further comprises receiving the flow information from the network interface service, the flow information being a subset of the configuration information, the method further comprising: the network virtualization device storing the flow information in the memory; the network virtualization device receiving an indication from the network interface service that at least a portion of the flow information is stale; receiving updated flow information by the virtual network interface card before receiving a second packet to which the flow information is applicable; the network virtualization device storing the updated flow information in the memory, the updated flow information including updates that replace at least the portion of the flow information, the method further comprising: receiving the second packet from the compute instance by the virtual network interface card; and transmitting, via the network virtualization device, the second packet on the virtual network based on the updated flow information.

10. A method comprising: receiving a first packet from a compute instance associated with a virtual network interface card hosted on a network virtualization device; determining, by the virtual network interface card, that flow information for transmitting the first packet on a virtual network is unavailable from a memory of the network virtualization device; the virtual network interface card transmitting the first packet to a network interface service via the network virtualization device, the network interface service configured to maintain configuration information for transmitting packets on the virtual network and to transmit the first packet on the virtual network based on the configuration information, the method further comprising: The method further comprises receiving the flow information from the network interface service, the flow information being a subset of the configuration information, the method further comprising: the network virtualization device storing the flow information in the memory; the network virtualization device receiving an indication from the network interface service that at least a portion of the flow information is stale; receiving, by the virtual network interface card, a second packet to which the flow information is applicable; and the virtual network interface card transmitting the second packet to the network interface service via the network virtualization device; receiving updated flow information by the network virtualization device; the network virtualization device storing the updated flow information in the memory; receiving a third packet from the compute instance by the virtual network interface card; and transmitting, via the network virtualization device, the third packet on the virtual network based on the updated flow information.

11. A method comprising: receiving a first packet from a compute instance associated with a virtual network interface card hosted on a network virtualization device; determining, by the virtual network interface card, that flow information for transmitting the first packet on a virtual network is unavailable from a memory of the network virtualization device; the virtual network interface card transmitting the first packet to a network interface service via the network virtualization device, the network interface service configured to maintain configuration information for transmitting packets on the virtual network and to transmit the first packet on the virtual network based on the configuration information, the method further comprising: The method further comprises receiving the flow information from the network interface service, the flow information being a subset of the configuration information, the method further comprising: the network virtualization device storing the flow information in the memory; receiving, by the virtual network interface card, a second packet from the virtual network destined for the compute instance; determining, by the virtual network interface card, that second flow information for transmitting the second packet to the compute instance is unavailable from the memory of the network virtualization device; and the virtual network interface card transmitting the second packet to the network interface service via the network virtualization device; receiving, by the virtual network interface card, a third packet from the network interface service via the network virtualization device, the third packet corresponding to an updated version of the second packet based on a second subset of the configuration information, the method further comprising: The method includes the virtual network interface card sending the third packet to the compute instance.

12. The method further comprises: The method further includes receiving, by the network virtualization device, the second flow information, the second subset of the configuration information, the network virtualization device storing the second flow information in the memory; receiving, by the virtual network interface card, a fourth packet destined for the compute instance from the virtual network via the network virtualization device; determining, by the virtual network interface card, that the second flow information is available from the memory for transmitting the fourth packet to the compute instance; The method of claim 11 , further comprising: the virtual network interface card sending the fourth packet to the compute instance based on the second flow information.

13. 1. A system comprising: a network virtualization device that hosts a virtual network interface card and is connected to an underlay network; a host machine connected to the network virtualization device and hosting a compute instance; a set of servers connected to the substrate network and hosting network interface services; The virtual network interface card is configured to receive a first packet from the computing instance; determining that flow information for transmitting the first packet on a virtual network is unavailable from a memory of the network virtualization device, the virtual network being an overlay on the substrate network, the virtual network interface card further comprising: configured to transmit the first packet to the network interface service via the network virtualization device; The network interface service configured to maintain configuration information for transmitting packets on the virtual network; configured to transmit the first packet on the virtual network based on the configuration information; configured to generate the flow information based on the configuration information; The network virtualization device configured to receive the flow information from the network interface service; The system is configured to store the flow information in the memory.

14. 14. The system of claim 13, wherein maintaining the configuration information includes tracking configuration information and version information associated with the configuration information or portions of the configuration information for each virtual network interface card hosted on the network virtualization device.

15. A computer readable program that, when executed on a system, causes the system to perform operations, the operations comprising: maintaining, by a network interface service, a set of configuration information for transmitting packets on the virtual network; receiving, by the network interface service, a first packet for a compute instance, the first packet being received from a virtual network interface card associated with the compute instance hosted on a network virtualization device connected to an underlay network, the operations further comprising: determining, by the network interface service, from the set, configuration information for transmitting the first packet over the virtual network, the virtual network being an overlay on the substrate network, the operations further comprising: transmitting, by the network interface service, the first packet on the virtual network based on the configuration information; generating, by the network interface service, flow information based on the configuration information; and transmitting, by the network interface service, the flow information to the network virtualization device, the flow information being a subset of the configuration information and enabling the virtual network interface card to transmit additional packets received from the compute instance over the virtual network.

16. A computer-readable program that, when executed on a system, causes the system to perform an operation, the operation comprising: maintaining, by a network interface service, a set of configuration information for transmitting packets on the virtual network; receiving, by the network interface service, a first packet for a compute instance, the first packet being received from a virtual network interface card hosted on a network virtualization device and associated with the compute instance, the operations further comprising: determining, by the network interface service, from the set, configuration information for transmitting the first packet on the virtual network; transmitting, by the network interface service, the first packet on the virtual network based on the configuration information; transmitting, by the network interface service, flow information to the network virtualization device, the flow information being a subset of the configuration information and enabling the virtual network interface card to transmit, over the virtual network, additional packets received from the compute instance; The operation further comprises: determining that at least a portion of the flow information is out of date based on version information associated with the flow information; sending an indication to the network virtualization device that at least the portion of the flow information is stale; and pushing updates to at least the portion of the flow information to the network virtualization device.

17. A computer-readable program that, when executed on a system, causes the system to perform an operation, the operation comprising: maintaining, by a network interface service, a set of configuration information for transmitting packets on the virtual network; receiving, by the network interface service, a first packet for a compute instance, the first packet being received from a virtual network interface card hosted on a network virtualization device and associated with the compute instance, the operations further comprising: determining, by the network interface service, from the set, configuration information for transmitting the first packet on the virtual network; transmitting, by the network interface service, the first packet on the virtual network based on the configuration information; transmitting, by the network interface service, flow information to the network virtualization device, the flow information being a subset of the configuration information and enabling the virtual network interface card to transmit, over the virtual network, additional packets received from the compute instance; The operation further comprises: determining that at least a portion of the flow information is out of date based on version information associated with the flow information; sending an indication to the network virtualization device that at least the portion of the flow information is stale; receiving a second packet from the virtual network interface card via the network virtualization device, to which the flow information applies; transmitting the second packet on the virtual network; and sending an update to the network virtualization device for at least the portion of the flow information.

Images