Layer 2 network using access control lists in virtualized cloud environments

Abstract

Techniques for communication in an L2 virtual network are described. In an example, the L2 virtual network includes a plurality of L2 compute instances hosted on a set of host machines and a plurality of L2 virtual network interfaces and L2 virtual switches hosted on a set of network virtualization devices. The L2 virtual network interfaces emulate L2 ports of the L2 virtual network. Access control list (ACL) information applicable to the L2 ports is sent to the network virtualization devices hosting the L2 virtual network interfaces.

Description

[0001] Cross-references to related applications

[0002] This international patent application claims priority to U.S. Patent Application No. 17 / 494,720, filed October 5, 2021, entitled “LAYER-2 NETWORKING USING ACCESS CONTROL LISTS IN A VIRTUALIZED CLOUD ENVIRONMENT,” which claims the benefit of U.S. Provisional Patent Application No. 63 / 132,377, filed December 30, 2020, entitled “LAYER-2 NETWORKING IN AVIRTUALIZED CLOUD ENVIRONMENT,” the contents of which are incorporated herein by reference in their entirety for all purposes. Background Technology

[0003] Cloud computing provides on-demand availability of computing resources. It can be based on data centers accessible to users via the internet. Cloud computing can provide Infrastructure as a Service (IaaS). Virtual networks can be created for user use. However, these virtual networks have limitations that restrict their functionality and value. Therefore, further improvements are expected. Summary of the Invention

[0004] This disclosure relates to virtualized cloud environments. Techniques for providing Layer 2 networking functionality in virtualized cloud environments are described. Layer 2 functionality is provided in conjunction with and complements Layer 3 networking functionality provided by the virtualized cloud environment.

[0005] Some embodiments of this disclosure relate to providing Layer 2 Virtual Local Area Networks (VLANs) to customers in a private network, such as a customer's Virtual Cloud Network (VCN). Different compute instances are connected within Layer 2 VLANs. The customer perceives a single, simulated switch connecting the compute instances. In reality, this simulated switch is implemented as an infinitely scalable, distributed switch comprising a collection of local switches. More specifically, each compute instance runs on a host machine connected to a Network Virtualization Device (NVD). For each compute instance on a host connected to the NVD, the NVD hosts a Layer 2 Virtual Network Interface Card (VNIC) and a local switch associated with the compute instance. The Layer 2 VNIC represents the port of the compute instance on the Layer 2 VLAN. The local switch connects the VNIC to other VNICs (e.g., other ports) associated with other compute instances in the Layer 2 VLAN. Various Layer 2 network services are supported, including, for example, the use of Layer 2 Access Control Lists (ACLs).

[0006] This document describes various embodiments, including methods, systems, and non-transitory computer-readable storage media that store programs, code, or instructions executable by one or more processors. Attached Figure Description

[0007] Figure 1 This is a high-level diagram of a distributed environment, illustrating a virtual or overlay cloud network hosted by a cloud service provider infrastructure according to certain embodiments.

[0008] Figure 2 A simplified architecture diagram of the physical components in the physical network within the CSPI according to certain embodiments is depicted.

[0009] Figure 3 An example arrangement within CSPI according to certain embodiments is shown, in which a host machine is connected to multiple network virtualization devices (NVDs).

[0010] Figure 4 The connectivity between the host machine and the NVD, according to certain embodiments, is described for providing I / O virtualization to support multiple tenancy.

[0011] Figure 5 A simplified block diagram of a physical network provided by CSPI according to certain embodiments is depicted.

[0012] Figure 6 This is a schematic diagram of a computing network according to certain embodiments.

[0013] Figure 7 This is a logical and hardware diagram of VLANs according to certain embodiments.

[0014] Figure 8 This is a logical diagram of multiple connected L2 VLANs according to certain embodiments.

[0015] Figure 9 This is a logical diagram of multiple connected L2 VLANs and subnet 900 according to certain embodiments.

[0016] Figure 10 This is a schematic diagram of intra-VLAN communication and learning within a VLAN according to certain embodiments.

[0017] Figure 11 This is a schematic diagram of a VLAN according to certain embodiments.

[0018] Figure 12 This is a flowchart illustrating a process 1200 for intra-VLAN communication according to certain embodiments.

[0019] Figure 13The illustration depicts an example environment suitable for defining ACLs for an L2 virtual network according to certain embodiments.

[0020] Figure 14 The illustration shows an example ACL technique in a VLAN according to certain embodiments.

[0021] Figure 15 This is a flowchart illustrating a process for distributing ACL information in a Layer 2 virtual network according to certain embodiments.

[0022] Figure 16 This is a flowchart illustrating a process for determining the applicability of ACL information to an L2 VNIC according to certain embodiments.

[0023] Figure 17 This is a flowchart illustrating a process for enforcing an ACL according to certain embodiments.

[0024] Figure 18 This is a flowchart illustrating a process for enforcing an ACL according to certain embodiments.

[0025] Figure 19 This is a block diagram illustrating a pattern for implementing cloud infrastructure as a service system according to at least one embodiment.

[0026] Figure 20 This is a block diagram illustrating another pattern for implementing cloud infrastructure as a service system according to at least one embodiment.

[0027] Figure 21 This is a block diagram illustrating another pattern for implementing cloud infrastructure as a service system according to at least one embodiment.

[0028] Figure 22 This is a block diagram illustrating another pattern for implementing cloud infrastructure as a service system according to at least one embodiment.

[0029] Figure 23 This is a block diagram illustrating an example computer system according to at least one embodiment. Detailed Implementation

[0030] In the following description, specific details are set forth for purposes of explanation in order to provide a thorough understanding of certain embodiments. However, it will be apparent that various embodiments may be practiced without these specific details. The accompanying drawings and description are not intended to be limiting. The word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any embodiment or design described herein as "exemplary" is not necessarily to be construed as preferred or superior to other embodiments or designs.

[0031] A- Example Virtual Networking Architecture

[0032] The term cloud service generally refers to services provided by a cloud service provider (CSP) to users or customers on demand (e.g., via a subscription model) using systems and infrastructure (cloud infrastructure) provided by the CSP. Typically, the servers and systems that make up the CSP's infrastructure are separate from the customer's own on-premises servers and systems. Therefore, customers can utilize cloud services provided by the CSP without having to purchase separate hardware and software resources for the service. Cloud services are designed to provide subscribers with simple, scalable access to applications and computing resources without requiring customers to invest in the infrastructure used to provide the service.

[0033] Several cloud service providers offer various types of cloud services. There are various types or models of cloud services, including Software as a Service (SaaS), Platform as a Service (PaaS), Infrastructure as a Service (IaaS), etc.

[0034] Customers can subscribe to one or more cloud services provided by a CSP. Customers can be any entity, such as individuals, organizations, or businesses. When a customer subscribes to or registers for a service provided by a CSP, a lease or account is created for that customer. The customer can then access one or more cloud resources associated with that account through their subscription.

[0035] As mentioned above, Infrastructure as a Service (IaaS) is a specific type of cloud computing service. In the IaaS model, a CSP provides infrastructure (called Cloud Service Provider Infrastructure or CSPI) that customers can use to build their own customizable networks and deploy customer resources. Therefore, the customer's resources and network are hosted in a distributed environment by the infrastructure provided by the CSP. This differs from traditional computing, where the customer's resources and network are hosted by the infrastructure provided by the customer.

[0036] CSPI can include interconnected high-performance computing resources forming a physical network, including various host machines, memory resources, and network resources. This physical network is also known as a substrate network or underlying network. Resources in CSPI can be distributed across one or more data centers, which may be geographically dispersed across one or more geographic regions. Virtualization software can be executed by these physical resources to provide a virtualized distributed environment. Virtualization creates an overlay network (also known as a software-based network, software-defined network, or virtual network) on the physical network. The CSPI physical network provides the underlying foundation for creating one or more overlay or virtual networks on top of the physical network. Virtual or overlay networks can include one or more Virtual Cloud Networks (VCNs). A virtual network is a layer of network abstraction that can run on top of a physical network, implemented using software virtualization technologies (e.g., hypervisors, functions performed by network virtualization devices (NVDs) (e.g., smartNICs), top-of-rack (TOR) switches, smart TORs that implement one or more functions performed by NVDs, and other mechanisms). Virtual networks can take many forms, including peer-to-peer networks, IP networks, etc. Virtual networks are typically either Layer 3 IP networks or Layer 2 VLANs. This method of virtual or overlay networking is often referred to as virtual or overlay layer 3 networking. Examples of protocols developed for virtual networks include IP-in-IP (or Generic Routing Encapsulation (GRE)), Virtual Scalable Local Area Network (VXLAN—IETF RFC 7348), Virtual Private Network (VPN) (e.g., MPLS layer 3 Virtual Private Network (RFC 4364)), VMware's NSX, GENEVE (Generic Network Virtualization Encapsulation), etc.

[0037] For IaaS, the infrastructure provided by a CSP (Center for Service Providers) can be configured to offer virtualized computing resources over a public network (e.g., the Internet). In the IaaS model, cloud service providers can host infrastructure components (e.g., servers, storage devices, network nodes (e.g., hardware), deployment software, platform virtualization (e.g., hypervisor layer), etc.). In some cases, IaaS providers can also provision various services to accompany those infrastructure components (e.g., billing, monitoring, logging, security, load balancing, and clustering, etc.). Therefore, since these services can be policy-driven, IaaS users can implement policies to drive load balancing to maintain application availability and performance. CSPI provides a collection of infrastructure and complementary cloud services that enable customers to build and run a wide range of applications and services in a highly available, hosted, distributed environment. CSPI provides high-performance computing resources and capabilities, as well as storage capacity, in a flexible virtual network securely accessible from various networked locations, such as the customer's on-premises network. When a customer subscribes to or enrolls in an IaaS service provided by a CSP, the lease created for that customer is a secure and isolated partition within the CSP, where the customer can create, organize, and manage their cloud resources.

[0038] Customers can build their own virtual networks using the compute, storage, and networking resources provided by CSPI. One or more customer resources or workloads, such as compute instances, can be deployed on these virtual networks. For example, a customer can use resources provided by CSPI to build one or more customizable and private virtual networks, called Virtual Cloud Networks (VCNs). A customer can deploy one or more customer resources, such as compute instances, on a customer VCN. Compute instances can take the form of virtual machines, bare metal instances, etc. Therefore, CSPI provides a collection of infrastructure and complementary cloud services that enable customers to build and run a wide range of applications and services in a highly available, virtually managed environment. Customers do not manage or control the underlying physical resources provided by CSPI, but they can control the operating system, storage devices, and deployed applications; and may have limited control over selected networking components (e.g., firewalls).

[0039] The CSP can provide a console that enables customers and network administrators to configure, access, and manage resources deployed in the cloud using CSPI resources. In some embodiments, the console provides a web-based user interface that can be used to access and manage the CSPI. In other embodiments, the console is a web-based application provided by the CSP.

[0040] CSPI can support both single-tenant and multi-tenant architectures. In a single-tenant architecture, software (e.g., applications, databases) or hardware components (e.g., host machines or servers) serve a single customer or tenant. In a multi-tenant architecture, the software or hardware components serve multiple customers or tenants. Therefore, in a multi-tenant architecture, CSPI resources are shared among multiple customers or tenants. In a multi-tenant scenario, precautions and safeguards are implemented in CSPI to ensure that each tenant's data is isolated and invisible to other tenants.

[0041] In a physical network, a network endpoint (“endpoint”) is a computing device or system that connects to and communicates with the physical network. Network endpoints in a physical network can connect to a Local Area Network (LAN), a Wide Area Network (WAN), or other types of physical networks. Examples of traditional endpoints in a physical network include modems, hubs, bridges, switches, routers and other network devices, physical computers (or host machines), etc. Each physical device in a physical network has a fixed network address that can be used to communicate with the device. This fixed network address can be a Layer 2 address (e.g., a MAC address), a fixed Layer 3 address (e.g., an IP address), etc. In a virtualized environment or virtual network, endpoints can include various virtual endpoints, such as virtual machines hosted by components of the physical network (e.g., hosted by physical host machines). These endpoints in a virtual network are addressed by overlay addresses, such as overlay Layer 2 addresses (e.g., overlay MAC addresses) and overlay Layer 3 addresses (e.g., overlay IP addresses). Network overlays provide flexibility by allowing network administrators to move around the overlay addresses associated with network endpoints using software management (e.g., via software implementing a control plane for virtual networks). Therefore, unlike physical networks, in virtual networks, overlay addresses (e.g., overlay IP addresses) can be moved from one endpoint to another using network management software. Since virtual networks are built on top of physical networks, communication between components within a virtual network involves both the virtual network and the underlying physical network. To facilitate this communication, CSPI components are configured to learn and store mappings that map overlay addresses in the virtual network to actual physical addresses in the baseboard network, and vice versa. These mappings are then used to facilitate communication. Client traffic is encapsulated to facilitate routing within the virtual network.

[0042] Therefore, physical addresses (e.g., physical IP addresses) are associated with components in a physical network, and overlay addresses (e.g., overlay IP addresses) are associated with entities in a virtual network. Both physical IP addresses and overlay IP addresses are types of real IP addresses. These are separate from virtual IP addresses, which map to multiple real IP addresses. Virtual IP addresses provide a one-to-many mapping between virtual IP addresses and multiple real IP addresses.

[0043] Cloud infrastructure, or CSPI, is physically hosted in one or more data centers in one or more regions of the world. CSPI can include components in a physical or substrate network and virtualized components (e.g., virtual networks, compute instances, virtual machines, etc.) in a virtual network built on top of the physical network components. In some embodiments, CSPI is organized and hosted in domains, regions, and availability domains. A region is typically a localized geographical area containing one or more data centers. Regions are generally independent of each other and can be geographically distant, for example, spanning countries or even continents. For example, one region might be in Australia, another in Japan, another in India, and so on. CSPI resources are partitioned between regions so that each region has its own independent subset of CSPI resources. Each region can provide a set of core infrastructure services and resources, such as compute resources (e.g., bare metal servers, virtual machines, containers, and related infrastructure); storage resources (e.g., block volume storage devices, file storage devices, object storage devices, archive storage devices); network resources (e.g., virtual cloud networks (VCNs), load balancing resources, connectivity to on-premises networks), database resources; edge networking resources (e.g., DNS); and access management and monitoring resources, etc. Each region typically has multiple paths connecting it to other regions within the domain.

[0044] Generally, applications are deployed in the areas where they are used most frequently (i.e., on infrastructure associated with that area) because using nearby resources is faster than using distant ones. Applications may also be deployed in different areas for various reasons, such as redundancy to mitigate the risk of events within a region (such as large weather systems or earthquakes), or to meet different requirements of legal jurisdictions, tax zones, and other business or social standards.

[0045] Data centers within a region can be further organized and subdivided into Availability Domains (ADs). An Availability Domain can correspond to one or more data centers located within a region. A region can consist of one or more Availability Domains. In this distributed environment, CSPI resources are either region-specific, such as Virtual Cloud Networks (VCNs), or Availability Domain-specific, such as compute instances.

[0046] Availability Zones (ADs) within a region are isolated from each other, fault-tolerant, and configured to make simultaneous failures highly unlikely. This is achieved by ensuring that ADs do not share critical infrastructure resources (such as networking, physical cabling, cable paths, cable entry points, etc.), making a failure at one AD within a region unlikely to affect the availability of other ADs in the same region. ADs within the same region can be interconnected via low-latency, high-bandwidth networks, enabling high-availability connectivity to other networks (e.g., the internet, customer on-premises networks, etc.) and allowing for replication systems across multiple ADs to achieve high availability and disaster recovery. Cloud services use multiple ADs to ensure high availability and prevent resource failures. As the infrastructure provided by an IaaS provider grows, more regions and ADs, along with additional capacity, can be added. Traffic between availability domains is typically encrypted.

[0047] In some embodiments, regions are grouped into domains. A domain is a logical collection of regions. Domains are isolated from each other and do not share any data. Regions within the same domain can communicate with each other, but regions in different domains cannot. A customer's lease or account with the CSP resides in a single domain and can be distributed across one or more regions belonging to that domain. Typically, when a customer subscribes to an IaaS service, a lease or account is created for that customer in a region within the domain that the customer designates (called the "primary" region). A customer can extend their lease to one or more other regions within the domain. A customer cannot access regions that are not within the domain where their lease is located.

[0048] IaaS providers can offer multiple domains, each catering to a specific set of customers or users. For example, a business domain can be offered for business customers. As another example, a domain can be offered for customers within a specific country. As yet another example, a government domain can be offered for governments, etc. For instance, a government domain can cater to a specific government and may have a higher level of security than the business domain. For example, Oracle Cloud Infrastructure (OCI) currently offers domains for the business region and two domains for the government cloud region (e.g., FedRAMP licensed and IL5 licensed).

[0049] In some embodiments, an Active Directory (AD) can be subdivided into one or more fault domains. A fault domain is a grouping of infrastructure resources within an AD to provide anti-affinity. Fault domains allow for the distribution of compute instances such that these instances do not reside on the same physical hardware within a single AD. This is known as anti-affinity. A fault domain refers to a collection of hardware components (computers, switches, etc.) that share a single point of failure. Compute pools are logically divided into fault domains. Therefore, a hardware failure or compute hardware maintenance event affecting one fault domain does not affect instances in other fault domains. Depending on the embodiment, the number of fault domains for each AD can vary. For example, in some embodiments, each AD contains three fault domains. Fault domains act as logical data centers within the AD.

[0050] When a customer subscribes to IaaS services, resources from CSPI are provisioned to the customer and associated with the customer's lease. Customers can use these provisioned resources to build private networks and deploy resources on those networks. Customer networks hosted in the cloud by CSPI are called Virtual Cloud Networks (VCNs). Customers can use the CSPI resources allocated to them to set up one or more VCNs. A VCN is a virtual or software-defined private network. Customer resources deployed in a customer's VCN can include compute instances (e.g., virtual machines, bare metal instances) and other resources. These compute instances can represent various customer workloads, such as applications, load balancers, databases, etc. Compute instances deployed on a VCN can communicate with publicly accessible endpoints (“public endpoints”) via public networks (such as the Internet), with other instances in the same VCN or other VCNs (e.g., other VCNs belonging to the customer or VCNs not belonging to the customer), with the customer's on-premises data center or network, and with service endpoints and other types of endpoints.

[0051] CSPs can use CSPIs to provide various services. In some cases, CSPI clients can function as service providers themselves and use CSPI resources to provide services. Service providers can expose service endpoints, which are characterized by identifying information such as IP addresses, DNS names, and ports. Client resources (e.g., compute instances) can access a specific service by accessing the service endpoints exposed by the service for that specific service. These service endpoints are generally publicly accessible to users via public communication networks (such as the Internet) using the public IP addresses associated with the endpoints. Publicly accessible network endpoints are sometimes also called public endpoints.

[0052] In some embodiments, a service provider may expose the service via an endpoint (sometimes referred to as a service endpoint) for the service. Clients of the service can then use this service endpoint to access the service. In some implementations, the service endpoint provided for the service can be accessed by multiple clients intending to consume the service. In other implementations, a dedicated service endpoint can be provided to a client, such that only that client can use that dedicated service endpoint to access the service.

[0053] In some embodiments, when a VCN is created, it is associated with a Private Overlay Classless Inter-Domain Routing (CIDR) address space, which refers to a set of private overlay IP addresses (e.g., 10.0 / 16) assigned to the VCN. A VCN includes associated subnets, routing tables, and gateways. A VCN resides within a single region but can span one or more of the availability domains within that region. A gateway is a virtual interface configured for the VCN and enables communication between the VCN and one or more endpoints outside the VCN. One or more different types of gateways can be configured for a VCN to enable communication to and from different types of endpoints.

[0054] A VCN can be subdivided into one or more subnets, such as one or more subnets. Therefore, a subnet is a configurable unit or subdivision that can be created within a VCN. A VCN can have one or more subnets. Each subnet within a VCN is associated with a contiguous range of overlay IP addresses (e.g., 10.0.0.0 / 24 and 10.0.1.0 / 24), which do not overlap with other subnets within that VCN and represent a subset of the VCN's address space.

[0055] Each compute instance is associated with a Virtual Network Interface Card (VNIC), which enables the compute instance to participate in subnets within a VCN. A VNIC is the logical representation of a physical network interface card (NIC). Generally, a VNIC is the interface between an entity (e.g., a compute instance, a service) and the virtual network. A VNIC exists within a subnet and has one or more associated IP addresses, along with associated security rules or policies. A VNIC is equivalent to a Layer 2 port on a switch. A VNIC is attached to both the compute instance and the subnet within the VCN. The VNIC associated with a compute instance makes the compute instance part of the VCN's subnet and enables the compute instance to communicate (e.g., send and receive packets) with endpoints on the same subnet as the compute instance, with endpoints in different subnets within the VCN, or with endpoints outside the VCN. Therefore, the VNIC associated with a compute instance determines how the compute instance connects to endpoints inside and outside the VCN. When a compute instance is created and added to a subnet within a VCN, a VNIC for the compute instance is created and associated with that compute instance. For a subnet that includes a set of compute instances, the subnet contains a VNIC corresponding to that set of compute instances, with each VNIC attached to a compute instance within that set of compute instances.

[0056] A private overlay IP address is assigned to each compute instance via the VNIC associated with it. This private overlay network IP address is assigned to the VNIC associated with the compute instance when the compute instance is created and is used to route traffic to and from the compute instance. All VNICs within a given subnet use the same routing table, security list, and DHCP options. As described above, each subnet within a VCN is associated with a contiguous range of overlay IP addresses (e.g., 10.0.0.0 / 24 and 10.0.1.0 / 24) that do not overlap with other subnets within that VCN and represent a subset of the address space within the VCN's address space. For a VNIC on a specific subnet of a VCN, the private overlay IP address assigned to that VNIC is an address from the contiguous range of overlay IP addresses allocated to the subnet.

[0057] In some embodiments, in addition to a private overlay IP address, a compute instance may optionally be assigned additional overlay IP addresses, such as one or more public IP addresses, for example, if in a public subnet. These multiple addresses may be assigned either on the same VNIC or on multiple VNICs associated with the compute instance. However, each instance has a primary VNIC, which is created during instance startup and associated with the overlay private IP address assigned to that instance—this primary VNIC cannot be deleted. Additional VNICs, called secondary VNICs, can be added to existing instances in the same availability domain as the primary VNIC. All VNICs are in the same availability domain as the instance. Secondary VNICs may reside in a subnet within the same VCN as the primary VNIC, or in a different subnet within the same VCN or different VCNs.

[0058] If a compute instance is in a public subnet, it can optionally be assigned a public IP address. When creating a subnet, it can be specified as either a public or private subnet. A private subnet means that resources within the subnet (such as compute instances) and associated VNICs cannot have public overriding IP addresses. A public subnet means that resources within the subnet and associated VNICs can have public IP addresses. Customers can specify that a subnet exists within a single availability domain or across multiple availability domains in regions or domains.

[0059] As described above, a VCN can be subdivided into one or more subnets. In some embodiments, a virtual router (VR) configured for the VCN (referred to as a VCN VR or simply VR) enables communication between the subnets of the VCN. For a subnet within the VCN, the VR represents a logical gateway for that subnet, enabling that subnet (i.e., the compute instances on that subnet) to communicate with endpoints on other subnets within the VCN and other endpoints outside the VCN. The VCN VR is a logical entity configured to route traffic between the VNIC within the VCN and the virtual gateway (“gateway”) associated with the VCN. The following section discusses… Figure 1Further description of the gateway. A VCN VR is a Layer 3 / IP layer concept. In one embodiment, there exists a VCN VR for a VCN, where the VCN VR has a potentially unlimited number of IP-addressable ports, one port for each subnet of the VCN. In this way, the VCN VR has a different IP address for each subnet within the VCN to which the VCN VR is attached. The VR also connects to various gateways configured for the VCN. In some embodiments, a specific overlay IP address within the overlay IP address range for a subnet is reserved for the port of the VCN VR for that subnet. For example, consider a VCN with two subnets, associated with address ranges 10.0 / 16 and 10.1 / 16. For the first subnet in the VCN with an address range of 10.0 / 16, addresses within this range are reserved for the port of the VCN VR for that subnet. In some cases, the first IP address within the range can be reserved for the VCN VR. For example, for a subnet covering the IP address range of 10.0 / 16, the IP address 10.0.0.1 could be reserved for the port of the VCN VR for that subnet. For a second subnet within the same VCN with an address range of 10.1 / 16, the VCN VR can have a port for the second subnet with the IP address 10.1.0.1. The VCN VR has a different IP address for each subnet within the VCN.

[0060] In some other embodiments, each subnet within a VCN may have its own associated VR, which can be addressed by the subnet using a reserved or default IP address associated with the VR. For example, the reserved or default IP address may be the first IP address in a range of IP addresses associated with the subnet. The VNICs within the subnet can use this default or reserved IP address to communicate with the VR associated with the subnet (e.g., send and receive packets). In this embodiment, the VR is the ingress / egress point of the subnet. The VR associated with a subnet within the VCN can communicate with other VRs associated with other subnets within the VCN. The VR can also communicate with the gateway associated with the VCN. The VR functionality of a subnet runs on or is performed by one or more NVDs that perform VNIC functionality for the VNICs within the subnet.

[0061] You can configure routing tables, security rules, and DHCP options for a VCN. A routing table is a virtual routing table used by the VCN and includes rules that route traffic from subnets within the VCN to destinations outside the VCN via gateways or specially configured instances. You can customize the VCN's routing table to control how packets are forwarded / routed to and from the VCN. DHCP options refer to configuration information automatically provided to the instance when it starts up.

[0062] Security rules configured for a VCN represent overlay firewall rules used by the VCN. Security rules can include ingress and egress rules, specifying the types of traffic allowed to enter and exit the VCN instance (e.g., based on protocol and port). Clients can choose whether a given rule is stateful or stateless. For example, a client can allow a set of incoming SSH traffic from anywhere to the instance by setting a stateful ingress rule with source CIDR 0.0.0.0 / 0 and destination TCP port 22. Security rules can be implemented using network security groups or security lists. A network security group consists of a set of security rules that apply only to resources within that group. A security list, on the other hand, includes rules applicable to all resources in any subnet using that security list. A default security list with default security rules can be provided to the VCN. DHCP options configured for the VCN provide configuration information that is automatically provided to instances within the VCN at instance startup.

[0063] In some embodiments, configuration information for a VCN is determined and stored by the VCN control plane. For example, configuration information for a VCN may include information about: the address range associated with the VCN, subnets and associated information within the VCN, one or more VRs associated with the VCN, compute instances and associated VNICs within the VCN, NVDs (e.g., VNICs, VRs, gateways) performing various virtualization network functions associated with the VCN, status information for the VCN, and other VCN-related information. In some embodiments, a VCN distribution service publishes configuration information stored by the VCN control plane or a portion thereof to the NVD. The distributed information can be used to update information stored and used by the NVD (e.g., forwarding tables, routing tables, etc.) to forward packets to or from compute instances within the VCN.

[0064] In some embodiments, the creation of VCNs and subnets is handled by the VCN control plane (CP), and the initiation of compute instances is handled by the compute control plane. The compute control plane is responsible for allocating physical resources to the compute instances and then invoking the VCN control plane to create VNICs and attach them to the compute instances. The VCN CP also maps VCN data to the VCN data plane, which is configured to perform packet forwarding and routing functions. In some embodiments, the VCN CP provides a distribution service responsible for providing updates to the VCN data plane. Examples of VCN control planes are also available in... Figure 17 , Figure 18 , Figure 19 and Figure 20 The description is as follows (see references 1716, 1816, 1916 and 2016).

[0065] Customers can create one or more VCNs using resources hosted by CSPI. Compute instances deployed on a customer's VCN can communicate with different endpoints. These endpoints can include endpoints hosted by CSPI and endpoints external to CSPI.

[0066] Figure 1 , Figure 2 , Figure 3 , Figure 4 , Figure 5 , Figure 17 , Figure 18 , Figure 19 and Figure 21 The document describes various architectures for implementing cloud-based services using CSPI, and these are described below. Figure 1 This is a high-level diagram of a distributed environment 100, illustrating an overlay or client VCN hosted by CSPI according to certain embodiments. Figure 1 The distributed environment described includes multiple components in the overlay network. Figure 1 The distributed environment 100 depicted herein is merely an example and is not intended to unduly limit the scope of the claimed embodiments. Many variations, substitutions, and modifications are possible. For example, in some implementations, Figure 1 The distributed environment described in the text can have more than Figure 1 The more or fewer systems or components shown can be combined into two or more systems, or can have different system configurations or arrangements.

[0067] like Figure 1 As illustrated in the example, distributed environment 100 includes a CSPI 101 that provides services and resources that customers can subscribe to and use to build their Virtual Cloud Network (VCN). In some embodiments, CSPI 101 provides IaaS services to subscribing customers. Data centers within CSPI 101 can be organized into one or more regions. Figure 1 The example shown is a region "Region US" 102. The customer has already configured customer VCN 104 for Region 102. The customer can deploy various compute instances on VCN 104, which can include virtual machines or bare metal instances. Examples of instances include applications, databases, load balancers, etc.

[0068] exist Figure 1 In the embodiment depicted, customer VCN 104 includes two subnets, namely "Subnet-1" and "Subnet-2", each with its own CIDR IP address range. Figure 1In this configuration, subnet-1 covers the IP address range of 10.0 / 16, and subnet-2 covers the address range of 10.1 / 16. VCN Virtual Router 105 represents the logical gateway for the VCN, enabling communication between subnets of VCN 104 and with other endpoints outside the VCN. VCN VR 105 is configured to route traffic between the VNICs within VCN 104 and the gateway associated with VCN 104. VCN VR 105 provides a port for each subnet of VCN 104. For example, VR 105 could provide a port with IP address 10.0.0.1 for subnet-1 and a port with IP address 10.1.0.1 for subnet-2.

[0069] Multiple compute instances can be deployed on each subnet, where compute instances can be virtual machine instances and / or bare metal instances. Compute instances within a subnet can be hosted by one or more host machines within CSPI 101. Compute instances participate in the subnet via a VNIC associated with the compute instance. For example, as... Figure 1 As shown, compute instance C1 becomes part of subnet-1 via the VNIC associated with it. Similarly, compute instance C2 becomes part of subnet-1 via the VNIC associated with it. In a similar manner, multiple compute instances (which can be virtual machine instances or bare metal instances) can be part of subnet-1. Each compute instance is assigned a private overlay IP address and MAC address via its associated VNIC. For example, in... Figure 1 In this subnet, compute instance C1 has an overlay IP address of 10.0.0.2 and a MAC address of M1, while compute instance C2 has a private overlay IP address of 10.0.0.3 and a MAC address of M2. Each compute instance in subnet-1 (including compute instances C1 and C2) has a default route to VCN VR 105 using IP address 10.0.0.1, which is the IP address of the port of VCN VR 105 used in subnet-1.

[0070] Multiple compute instances, including virtual machine instances and / or bare metal instances, can be deployed on subnet-2. For example, such as Figure 1 As shown, compute instances D1 and D2 become part of subnet-2 via the VNIC associated with the respective compute instance. Figure 1 In the illustrated embodiment, compute instance D1 has an overlay IP address of 10.1.0.2 and a MAC address of MM1, while compute instance D2 has a private overlay IP address of 10.1.0.3 and a MAC address of MM2. Each compute instance in subnet-2 (including compute instances D1 and D2) has a default route to VCN VR 105 using IP address 10.1.0.1, which is the IP address of the port of VCN VR 105 in subnet-2.

[0071] VCN A 104 may also include one or more load balancers. For example, a load balancer can be provided for a subnet, and the load balancer can be configured to load balance traffic across multiple compute instances on the subnet. Load balancers can also be provided to load balance traffic across subnets within a VCN.

[0072] A specific compute instance deployed on VCN 104 can communicate with a variety of different endpoints. These endpoints can include endpoints hosted by CSPI 200 and endpoints outside of CSPI 200. Endpoints hosted by CSPI 101 can include: endpoints located on the same subnet as the specific compute instance (e.g., communication between two compute instances in subnet-1); endpoints located on different subnets but within the same VCN (e.g., communication between a compute instance in subnet-1 and a compute instance in subnet-2); endpoints in different VCNs within the same region (e.g., communication between a compute instance in subnet-1 and an endpoint in a VCN in the same region 106 or 110, or communication between a compute instance in subnet-1 and an endpoint in service point 110 within the same region); or endpoints in VCNs in different regions (e.g., communication between a compute instance in subnet-1 and an endpoint in a VCN in a different region 108). Compute instances in a subnet hosted by CSPI 101 can also communicate with endpoints not hosted by CSPI 101 (i.e., outside of CSPI 101). These external endpoints include endpoints in the customer’s on-premises network 116, endpoints in other remote cloud-hosted networks 118, public endpoints 114 accessible via public networks such as the Internet, and other endpoints.

[0073] VNICs associated with source and destination compute instances facilitate communication between compute instances on the same subnet. For example, compute instance C1 in subnet-1 might want to send a packet to compute instance C2 in subnet-1. For a packet originating from the source compute instance and destined for another compute instance in the same subnet, the packet is first processed by the VNIC associated with the source compute instance. The processing performed by the VNIC associated with the source compute instance may include determining the packet's destination information from the packet header, identifying any policies (e.g., security lists) configured for the VNIC associated with the source compute instance, determining the next hop for the packet, performing any packet encapsulation / decapsulation functions as needed, and then forwarding / routing the packet to the next hop to facilitate communication to its intended destination. When the destination compute instance is in the same subnet as the source compute instance, the VNIC associated with the source compute instance is configured to identify the VNIC associated with the destination compute instance and forward the packet to that VNIC for processing. The VNIC associated with the destination compute instance then performs the processing and forwards the packet to the destination compute instance.

[0074] For packets to be transferred from compute instances in a subnet to endpoints in different subnets within the same VCN, communication is facilitated by the VNIC associated with the source and destination compute instances, as well as the VCN VR. For example, if Figure 1 If compute instance C1 in subnet-1 wants to send a packet to compute instance D1 in subnet-2, the packet is first processed by the VNIC associated with compute instance C1. The VNIC associated with compute instance C1 is configured to route the packet to VCN VR 105 using the default route or port 10.0.0.1 of the VCN VR. VCN VR 105 is configured to route the packet to subnet-2 using port 10.1.0.1. Then, the VNIC associated with D1 receives and processes the packet and forwards it to compute instance D1.

[0075] For packets to be transferred from a compute instance in VCN 104 to an endpoint outside VCN 104, communication is facilitated by the VNIC associated with the source compute instance, VCN VR 105, and the gateway associated with VCN 104. One or more types of gateways can be associated with VCN 104. A gateway is an interface between a VCN and another endpoint located outside the VCN. A gateway is a Layer 3 / IP concept and enables a VCN to communicate with endpoints outside the VCN. Therefore, a gateway facilitates traffic flow between a VCN and other VCNs or networks. Various different types of gateways can be configured for a VCN to facilitate different types of communication with different types of endpoints. Depending on the gateway, communication can occur over a public network (e.g., the Internet) or over a private network. Various communication protocols can be used for these communications.

[0076] For example, compute instance C1 might want to communicate with an endpoint outside of VCN 104. The packet can first be processed by the VNIC associated with the source compute instance C1. The VNIC processing determines that the packet's destination is outside C1's subnet-1. The VNIC associated with C1 can then forward the packet to VCN VR 105 for VCN 104. VCN VR 105 then processes the packet and, as part of the processing, determines a specific gateway associated with VCN 104 as the next hop for the packet based on its destination. VCN VR 105 can then forward the packet to the specific identified gateway. For example, if the destination is an endpoint within a customer's on-premises network, the packet can be forwarded by VCN VR 105 to the Dynamic Routing Gateway (DRG) gateway 122 configured for VCN 104. The packet can then be forwarded from the gateway to the next hop to facilitate delivery to its final intended destination.

[0077] Various types of gateways can be configured for a VCN. Examples of gateways that can be configured for a VCN are available in [link / details]. Figure 1 It is depicted in [the text] and described below. Examples of gateways associated with VCNs are also [described in the text]. Figure 17 , Figure 18 , Figure 19 and Figure 20 The gateways are depicted (e.g., referenced by reference numerals 1734, 1736, 1738, 1834, 1836, 1838, 1934, 1936, 1938, 2034, 2036, and 2038) and are described below. Figure 1As depicted in the embodiments, a Dynamic Routing Gateway (DRG) 122 can be added to or associated with a customer VCN 104 and provides a path for private network traffic communication between the customer VCN 104 and another endpoint, which can be the customer's on-premises network 116, a VCN 108 in a different region of CSPI 101, or another remote cloud network 118 not hosted by CSPI 101. The customer's on-premises network 116 can be a customer network or customer data center built using the customer's resources. Access to the customer's on-premises network 116 is generally very restricted. For customers who have both a customer's on-premises network 116 and one or more VCNs 104 deployed or hosted in the cloud by CSPI 101, the customer may want their on-premises network 116 and their cloud-based VCN 104 to be able to communicate with each other. This allows customers to build extended hybrid environments that include the customer's VCN 104 hosted by CSPI 101 and their on-premises network 116. The DRG 122 enables this communication. To enable this type of communication, a communication channel 124 is set up, with one endpoint located in the customer's on-premises network 116 and the other endpoint located in CSPI 101 and connected to the customer's VCN 104. Communication channel 124 can be over a public communication network (such as the Internet) or a private communication network. Various communication protocols can be used, such as IPsec VPN technology over a public communication network (such as the Internet), Oracle's FastConnect technology using a private network instead of a public network, etc. The device or equipment forming one endpoint of communication channel 124 in the customer's on-premises network 116 is referred to as a customer field equipment (CPE), such as... Figure 1 The CPE126 is depicted in the diagram. On the CSPI 101 side, the endpoint can be a host machine executing DRG 122.

[0078] In some embodiments, a remote peering connection (RPC) can be added to the DRG, which allows a customer to peer one VCN to another VCN in a different region. Using such an RPC, customer VCN 104 can use DRG 122 to connect to VCN 108 in another region. DRG 122 can also be used to communicate with other remote cloud networks 118 (such as Microsoft Azure, Amazon AWS, etc.) not hosted by CSPI 101.

[0079] like Figure 1As shown, an Internet Gateway (IGW) 120 can be configured for customer VCN 104, enabling compute instances on VCN 104 to communicate with a public endpoint 114 accessible via a public network, such as the Internet. IGW 1120 is a gateway connecting the VCN to a public network such as the Internet. IGW 120 enables public subnets within the VCN (such as VCN 104), where resources within the public subnet have publicly overriding IP addresses, to directly access a public endpoint 112 on the public network 114 (such as the Internet). Using IGW 120, connections can be initiated from subnets within VCN 104 or from the Internet.

[0080] A Network Address Translation (NAT) gateway 128 can be configured for a customer's VCN 104, enabling cloud resources within the customer's VCN that do not have dedicated public overlay IP addresses to access the Internet, and doing so to direct incoming Internet connections (e.g., L4-L7 connections) without exposing those resources. This allows private subnets within the VCN (such as private subnet-1 in VCN 104) to privately access public endpoints on the Internet. In a NAT gateway, connections can only originate from private subnets to the public Internet, and not from the Internet to private subnets.

[0081] In some embodiments, a Service Gateway (SGW) 126 can be configured for customer VCN 104 to provide a path for private network traffic between VCN 104 and service endpoints supported in service network 110. In some embodiments, service network 110 may be provided by a CSP and may offer a variety of services. An example of such a service network is Oracle's service network, which provides a variety of services available to customers. For example, compute instances (e.g., database systems) in a private subnet of customer VCN 104 can back up data to service endpoints (e.g., object storage devices) without requiring a public IP address or access to the Internet. In some embodiments, a VCN may have only one SGW, and connections can only originate from subnets within the VCN, not from service network 110. If a VCN is peered to another, resources in the other VCN typically cannot access the SGW. Resources in the on-premises network of a VCN connected using FastConnect or VPN Connect can also use the service gateway configured for that VCN.

[0082] In some implementations, the SGW 126 uses the concept of a Service Classless Inter-Domain Routing (CIDR) label, which is a string representing all regional public IP address ranges for the service or group of services of interest. Customers use the Service CIDR label when configuring the SGW and associated routing rules to control traffic to the service. If the public IP address of the service changes in the future, customers can optionally use it when configuring security rules without having to adjust them.

[0083] Local peering gateway (LPG) 132 is a gateway that can be added to customer VCN 104 and enable VCN 104 to peer with another VCN in the same region. Peering means that VCNs communicate using private IP addresses, and traffic does not need to traverse public networks (such as the Internet) or be routed through the customer's on-premises network 116. In a preferred embodiment, a VCN has a separate LPG for each peer it establishes. Local peering or VCN peering is a common practice for establishing network connectivity between different applications or infrastructure management functions.

[0084] Service providers (such as those providing services in service network 110) can offer access to services using different access models. Under the public access model, a service can be exposed as a public endpoint that is publicly accessible by compute instances within a customer's VCN via a public network (such as the Internet), and / or can be privately accessed via SGW 126. Under a specific private access model, a service can be accessed as a private IP endpoint within a private subnet of the customer's VCN. This is called Private Endpoint (PE) access and enables service providers to expose their services as instances within the customer's private network. A private endpoint resource represents a service within the customer's VCN. Each PE is represented as a VNIC (called a PE-VNIC, with one or more private IPs) within the customer's VCN in a subnet selected by the customer. Thus, the PE provides a way to present services within a private customer VCN subnet using a VNIC. Since the endpoint is exposed as a VNIC, all characteristics associated with the VNIC (such as routing rules, security lists, etc.) are now available for the PE VNIC.

[0085] Service providers can register their services to enable access via PE. Providers can associate policies with services, which restricts the visibility of the service to customer leases. Providers can register multiple services under a single Virtual IP address (VIP), especially for multi-tenant services. Multiple such private endpoints (across multiple VCNs) can represent the same service.

[0086] Compute instances in a private subnet can then access the service using the private IP address or service DNS name of the PE VNIC. Compute instances in a customer VCN can access the service by sending traffic to the private IP address of the PE in the customer VCN. A Private Access Gateway (PAGW) 130 is a gateway resource that can be attached to a service provider VCN (e.g., a VCN in service network 110), which acts as the ingress / egress point for all traffic from / to the private endpoints of the customer subnet. PAGW 130 enables providers to scale the number of PE connections without utilizing their internal IP address resources. A provider only needs to configure one PAGW for any number of services registered in a single VCN. A provider can represent a service as a private endpoint in multiple VCNs of one or more customers. From the customer's perspective, the PE VNIC is not an instance attached to the customer, but rather appears to be attached to the service that the customer wants to interact with. Traffic to the private endpoint is routed to the service via PAGW 130. These are referred to as customer-to-service private connections (C2S connections).

[0087] The PE concept can also be used to extend private access to services to the customer's on-premises network and data center by allowing traffic to flow through FastConnect / IPsec links and private endpoints in the customer's VCN. Private access to services can also be extended to the customer's peering VCN by allowing traffic to flow between the LPG 132 and the PE in the customer's VCN.

[0088] Customers can control routing within a VCN at the subnet level, allowing them to specify which subnets within a customer's VCN (such as VCN104) use each gateway. The VCN's routing table is used to determine whether traffic is allowed to leave the VCN via a specific gateway. For example, in a given instance, a routing table for a public subnet within customer VCN 104 might send non-local traffic via IGW 120. A routing table for a private subnet within the same customer VCN 104 might send traffic destined for a CSP service via SGW 126. All remaining traffic can be sent via NAT gateway 128. The routing table only controls traffic flowing out of the VCN.

[0089] Security lists associated with a VCN are used to control traffic entering the VCN via a gateway through inbound connections. All resources within a subnet use the same routing tables and security lists. Security lists can be used to control specific types of traffic allowed to enter or leave instances within a subnet of the VCN. Security list rules can include inbound and outbound rules. For example, inbound rules can specify allowed source address ranges, while outbound rules can specify allowed destination address ranges. Security rules can specify specific protocols (e.g., TCP, ICMP), specific ports (e.g., port 22 for SSH, port 3389 for Windows RDP), etc. In some implementations, the instance's operating system can enforce its own firewall rules that conform to the security list rules. Rules can be stateful (e.g., tracking connections and automatically allowing responses without explicit security list rules for response traffic) or stateless.

[0090] Access from a customer's VCN (i.e., through resources or compute instances deployed on VCN 104) can be categorized as public access, private access, or dedicated access. Public access refers to an access model that uses a public IP address or NAT to access a public endpoint. Private access enables customer workloads with private IP addresses within VCN 104 (e.g., resources in a private subnet) to access services without traversing a public network such as the Internet. In some embodiments, CSPI 101 enables customer VCN workloads with private IP addresses to access services (public service endpoints) using a service gateway. Thus, the service gateway provides a private access model by establishing a virtual link between the customer's VCN and the public endpoint of the service residing outside the customer's private network.

[0091] Furthermore, CSPI can provide dedicated public access using technologies such as FastConnect public peering, where on-premises instances can access one or more services within a customer's VCN using FastConnect connections without traversing public networks such as the internet. CSPI can also provide dedicated private access using FastConnect private peering, where on-premises instances with private IP addresses can access a customer's VCN workloads using FastConnect connections. FastConnect is a network connectivity alternative to using the public internet to connect a customer's on-premises network to CSPI and its services. Compared to internet-based connections, FastConnect offers a simple, flexible, and cost-effective way to create dedicated and private connections with higher bandwidth options and a more reliable and consistent network experience.

[0092] Figure 1The accompanying description above describes the various virtualized components in the example virtual network. As mentioned above, the virtual network is built on the underlying physical or substrate network. Figure 2 A simplified architecture diagram of the physical components within the physical network of the CSPI 200, which provides the underlying layer for virtual networks according to certain embodiments, is depicted. As shown, the CSPI 200 provides a distributed environment including components and resources (e.g., compute, memory, and network resources) provided by a cloud service provider (CSP). These components and resources are used to provide cloud services (e.g., IaaS services) to subscribed customers (i.e., customers who have subscribed to one or more services provided by the CSP). Based on the services subscribed to by the customer, a subset of the resources of the CSPI 200 (e.g., compute, memory, and network resources) is provisioned to the customer. The customer can then use the physical compute, memory, and networking resources provided by the CSPI 200 to build their own cloud-based (i.e., CSPI-hosted) customizable and private virtual networks. As indicated above, these customer networks are referred to as Virtual Cloud Networks (VCNs). Customers can deploy one or more customer resources, such as compute instances, on these customer VCNs. Compute instances can take the form of virtual machines, bare metal instances, etc. The CSPI 200 provides a collection of infrastructure and complementary cloud services, enabling customers to build and run a wide range of applications and services in a highly available managed environment.

[0093] exist Figure 2 In the example embodiment depicted, the physical components of CSPI 200 include one or more physical host machines or physical servers (e.g., 202, 206, 208), network virtualization devices (NVDs) (e.g., 210, 212), top-of-rack (TOR) switches (e.g., 214, 216), and a physical network (e.g., 218), as well as switches within physical network 218. The physical host machines or servers can host and execute various compute instances participating in one or more subnets of the VCN. Compute instances can include virtual machine instances and bare metal instances. For example, Figure 1 The various computational examples described in the text can be derived from... Figure 2 The physical host machine described in the diagram is used for hosting virtual machine compute instances in a VCN. Virtual machine compute instances in a VCN can be executed by one host machine or multiple different host machines. A physical host machine can also host virtual host machines, container-based hosts, or functions, etc. Figure 1 The VNIC and VCN VR described in the text can be generated by Figure 2 The NVD execution described in the text. Figure 1 The gateway described in the text can be... Figure 2 The host machine and / or NVD execution described in the document.

[0094] A host machine or server can run a hypervisor (also known as a virtual machine monitor or VMM) that creates and enables virtualized environments on the host machine. Virtualization or virtualized environments facilitate cloud-based computing. One or more compute instances can be created, executed, and managed on the host machine by a hypervisor on that host machine. The hypervisor on the host machine enables the host machine's physical computing resources (e.g., compute, memory, and network resources) to be shared among various compute instances executed by the host machine.

[0095] For example, such as Figure 2 As depicted, host machines 202 and 208 execute hypervisors 260 and 266, respectively. These hypervisors can be implemented using software, firmware, or hardware, or a combination thereof. Typically, a hypervisor is a process or software layer residing above the host machine's operating system (OS), which in turn executes on the host machine's hardware processor. Hypervisors provide a virtualized environment by enabling the host machine's physical computing resources (e.g., processing resources such as processors / cores, memory resources, network resources) to be shared among various virtual machine computing instances executed by the host machine. For example, in... Figure 2 In this configuration, the hypervisor 260 can reside on top of the operating system of the host machine 202 and enable the computing resources (e.g., processing, memory, and network resources) of the host machine 202 to be shared among computing instances (e.g., virtual machines) executed by the host machine 202. A virtual machine can have its own operating system (called a guest operating system), which may be the same as or different from the host machine's operating system. The operating system of a virtual machine executed by the host machine can be the same as or different from the operating system of another virtual machine executed by the same host machine. Therefore, the hypervisor enables multiple operating systems to be executed simultaneously, while sharing the same computing resources of the host machine. Figure 2 The host machines described in the text may have the same or different types of management programs.

[0096] A compute instance can be a virtual machine instance or a bare metal instance. Figure 2 In the diagram, compute instance 268 on host machine 202 and compute instance 274 on host machine 208 are examples of virtual machine instances. Host machine 206 is an example of a bare metal instance provided to a customer.

[0097] In some cases, an entire host machine can be provisioned to a single customer, and one or more compute instances (or virtual machines or bare metal instances) hosted by that host machine all belong to the same customer. In other cases, the host machine can be shared among multiple customers (i.e., multiple tenants). In this multi-tenancy scenario, the host machine can host virtual machine compute instances belonging to different customers. These compute instances can be members of different VCNs for different customers. In some embodiments, bare metal compute instances are hosted by bare metal servers without a hypervisor. When provisioning bare metal compute instances, a single customer or tenant maintains control over the physical CPU, memory, and network interfaces of the host machine hosting the bare metal instance, and the host machine is not shared with other customers or tenants.

[0098] As previously mentioned, each compute instance as part of a VCN is associated with a VNIC that enables that compute instance to become a member of a subnet of the VCN. The VNIC associated with a compute instance facilitates communication to and from the compute instance. The VNIC is associated with the compute instance when the compute instance is created. In some embodiments, for compute instances executed by a host machine, the VNIC associated with that compute instance is executed by an NVD connected to the host machine. For example, in Figure 2 In this example, host machine 202 executes a virtual machine compute instance 268 associated with VNIC 276, and VNIC 276 is executed by NVD 210 connected to host machine 202. As another example, a bare metal instance 272 hosted by host machine 206 is associated with VNIC 280 executed by NVD 212 connected to host machine 206. As yet another example, VNIC 284 is associated with compute instance 274 executed by host machine 208, and VNIC 284 is executed by NVD 212 connected to host machine 208.

[0099] For compute instances hosted by a host machine, NVDs connected to that host machine also execute VCN VR corresponding to the VCN where the compute instance is a member. For example, in Figure 2 In the embodiment depicted, NVD 210 executes VCN VR 277 corresponding to a VCN that is a member of compute instance 268. NVD 212 may also execute one or more VCN VR 283 corresponding to a VCN that corresponds to a compute instance hosted by host machines 206 and 208.

[0100] The host machine may include one or more network interface cards (NICs) that enable the host machine to connect to other devices. The NIC on the host machine can provide one or more ports (or interfaces) that allow the host machine to communicate with another device. For example, the host machine can use one or more ports (or interfaces) provided on the host machine and the NVD to connect to the NVD. The host machine can also connect to other devices (such as another host machine).

[0101] For example, in Figure 2 In this configuration, host machine 202 is connected to NVD 210 via link 220, which extends between port 234 provided by NIC 232 of host machine 202 and port 236 of NVD 210. Host machine 206 is connected to NVD 212 via link 224, which extends between port 246 provided by NIC 244 of host machine 206 and port 248 of NVD 212. Host machine 208 is connected to NVD 212 via link 226, which extends between port 252 provided by NIC 250 of host machine 208 and port 254 of NVD 212.

[0102] The NVD is then connected to top-of-rack (TOR) switches via communication links, which are connected to physical network 218 (also known as a switch architecture). In some embodiments, the links between the host machine and the NVD, and between the NVD and the TOR switches, are Ethernet links. For example, in Figure 2 In this configuration, NVDs 210 and 212 are connected to TOR switches 214 and 216 via links 228 and 230, respectively. In some embodiments, links 220, 224, 226, 228, and 230 are Ethernet links. The collection of host machines and NVDs connected to the TOR is sometimes referred to as a rack.

[0103] Physical network 218 provides a communication architecture that enables TOR switches to communicate with each other. Physical network 218 can be a multi-layer network. In some implementations, physical network 218 is a multi-layer Clos network of switches, where TOR switches 214 and 216 represent leaf-level nodes of the multi-layer and multi-node physical switching network 218. Different Clos network configurations are possible, including but not limited to Layer 2 networks, Layer 3 networks, Layer 4 networks, Layer 5 networks, and general "n"-layer networks. Examples of Clos networks are provided in... Figure 5 It is depicted in the middle and described below.

[0104] Various connection configurations can exist between the host machine and the NVD, such as one-to-one, many-to-one, and one-to-many configurations. In a one-to-one configuration, each host machine connects to its own individual NVD. For example, in... Figure 2 In this configuration, host machine 202 connects to NVD 210 via its NIC 232. In a many-to-one configuration, multiple host machines connect to a single NVD. For example, in... Figure 2 In this configuration, host machines 206 and 208 are connected to the same NVD 212 via NICs 244 and 250, respectively.

[0105] In a one-to-many configuration, a host machine connects to multiple NVDs. Figure 3 An example within the CSPI 300 is shown, where a host machine is connected to multiple NVDs. (Example follows) Figure 3 As shown, host machine 302 includes a network interface card (NIC) 304, which includes multiple ports 306 and 308. Host machine 300 is connected to a first NVD 310 via port 306 and link 320, and to a second NVD 312 via port 308 and link 322. Ports 306 and 308 may be Ethernet ports, and links 320 and 322 between host machine 302 and NVDs 310 and 312 may be Ethernet links. NVD 310 is further connected to a first TOR switch 314, and NVD 312 is connected to a second TOR switch 316. Links between NVDs 310 and 312 and TOR switches 314 and 316 may be Ethernet links. TOR switches 314 and 316 represent Layer 0 switching devices in a multi-layer physical network 318.

[0106] Figure 3 The arrangement depicted provides two separate physical network paths from physical switch network 318 to host machine 302: the first path passes through TOR switch 314 to NVD 310 and then to host machine 302, and the second path passes through TOR switch 316 to NVD 312 and then to host machine 302. These separate paths provide enhanced availability (referred to as high availability) for host machine 302. If one of the paths (e.g., a link in one of the paths breaks) or a device (e.g., a particular NVD is not running) experiences a problem, the other path can be used for communication with host machine 302.

[0107] exist Figure 3 In the configuration depicted, the host machine connects to two different NVDs using two different ports provided by the host machine's NIC. In other embodiments, the host machine may include multiple NICs that enable the host machine to connect to multiple NVDs.

[0108] Go back to reference Figure 2An NVD is a physical device or component that performs one or more network and / or storage virtualization functions. An NVD can be any device with one or more processing units (e.g., CPU, Network Processing Unit (NPU), FPGA, packet processing pipeline, etc.), memory (including cache), and ports. Various virtualization functions can be executed by software / firmware performed by one or more processing units of the NVD.

[0109] NVDs can be implemented in various different forms. For example, in some embodiments, an NVD is implemented as an interface card called a smartNIC or a smart NIC with an onboard embedded processor. A smartNIC is a device that is independent of the NIC on the host machine. Figure 2 In this context, NVD 210 and 212 can be implemented as smartNICs connected to host machine 202 and host machines 206 and 208, respectively.

[0110] However, smartNIC is just one example of an NVD implementation. Various other implementations are possible. For example, in some other implementations, the NVD, or one or more functions performed by the NVD, may be integrated into or performed by one or more host machines, one or more TOR switches, and other components of the CSPI 200. For instance, the NVD may be implemented within a host machine, where the functions performed by the NVD are performed by the host machine. As another example, the NVD may be part of a TOR switch, or the TOR switch may be configured to perform functions performed by the NVD, enabling the TOR switch to perform various complex packet transformations for public clouds. A TOR performing the functions of the NVD is sometimes referred to as a smart TOR. In other implementations that serve virtual machine (VM) instances rather than bare metal (BM) instances to customers, the functions performed by the NVD may be implemented within the hypervisor of the host machine. In some other implementations, some of the functions of the NVD may be offloaded to a centralized service running on a set of host machines.

[0111] In some embodiments, such as when implemented as Figure 2 As shown in the smartNIC diagram, the NVD can include multiple physical ports that enable it to connect to one or more host machines and one or more TOR switches. Ports on the NVD can be categorized as host-facing ports (also known as "south ports") or network-facing or TOR-facing ports (also known as "north ports"). The host-facing ports of the NVD are those used to connect the NVD to the host machine. Figure 2 Examples of host-facing ports include port 236 on the NVD 210 and ports 248 and 254 on the NVD 212. Network-facing ports on the NVD are used to connect the NVD to a TOR switch. Figure 2 Examples of network-facing ports include port 256 on the NVD 210 and port 258 on the NVD 212. Figure 2 As shown, NVD 210 is connected to TOR switch 214 via link 228, which extends from port 256 of NVD 210 to TOR switch 214. Similarly, NVD 212 is connected to TOR switch 216 via link 230, which extends from port 258 of NVD 212 to TOR switch 216.

[0112] The NVD receives packets and frames from the host machine (e.g., packets and frames generated by compute instances hosted by the host machine) via its host-facing port, and after performing the necessary packet processing, can forward the packets and frames to the TOR switch via its network-facing port. The NVD can also receive packets and frames from the TOR switch via its network-facing port, and after performing the necessary packet processing, can forward the packets and frames to the host machine via its host-facing port.

[0113] In some embodiments, there can be multiple ports and associated links between the NVD and TOR switches. These ports and links can be aggregated to form a link aggregation group (called a LAG) of multiple ports or links. Link aggregation allows multiple physical links between two endpoints (e.g., between the NVD and TOR switches) to be treated as a single logical link. All physical links in a given LAG can operate at the same speed in full-duplex mode. LAGs help increase the bandwidth and reliability of the connection between two endpoints. If one of the physical links in the LAG fails, traffic is dynamically and transparently reassigned to one of the other physical links in the LAG. Aggregated physical links deliver higher bandwidth than each individual link. Multiple ports associated with an LAG are treated as a single logical port. Traffic can be load-balanced across the multiple physical links of the LAG. One or more LAGs can be configured between two endpoints. These endpoints can be located between the NVD and TOR switches, between a host machine and the NVD, etc.

[0114] The NVD implements or performs network virtualization functions. These functions are performed by software / firmware executed by the NVD. Examples of network virtualization functions include, but are not limited to: packet encapsulation and decapsulation functions; functions for creating VCN networks; functions for implementing network policies, such as VCN security list (firewall) functionality; functions for facilitating packet routing and forwarding to and from compute instances within the VCN; and so on. In some embodiments, upon receiving a packet, the NVD is configured to perform a packet processing pipeline to process the packet and determine how to forward or route it. As part of this packet processing pipeline, the NVD may perform one or more virtual functions associated with the overlay network, such as performing VNICs associated with the CIS in the VCN, performing virtual routers (VRs) associated with the VCN, packet encapsulation and decapsulation to facilitate forwarding or routing in the virtual network, execution of certain gateways (e.g., local peer gateways), implementation of security lists, network security groups, Network Address Translation (NAT) functionality (e.g., per-host translation of public IPs to private IPs), throttling functions, and other functions.

[0115] In some embodiments, the packet processing data path in the NVD may include multiple packet pipelines, each consisting of a series of packet transformation stages. In some implementations, upon receiving a packet, the packet is parsed and classified into a single pipeline. The packet is then processed linearly, stage by stage, until the packet is either dropped or sent out through the NVD's interface. These stages provide basic functional packet processing building blocks (e.g., header verification, throttling, insertion of new Layer 2 headers, L4 firewall enforcement, VCN encapsulation / decapsulation, etc.) so that new pipelines can be built by combining existing stages, and new functionality can be added by creating new stages and inserting them into existing pipelines.

[0116] NVD can perform control plane and data plane functions corresponding to those of VCN's control plane and data plane. An example of the VCN control plane is also available in... Figure 17 , Figure 18 , Figure 19 and Figure 20 Depicted in (see references 1716, 1816, 1916, and 2016) and described below. An example of the VCN data plane is... Figure 17 , Figure 18 , Figure 19 and Figure 20The data plane is depicted (see reference numerals 1718, 1818, 1918, and 2018) and described below. Control plane functions include features for configuring how control data is forwarded on the network (e.g., setting routes and routing tables, configuring VNICs, etc.). In some embodiments, a VCN control plane is provided that centrally computes all overlay mappings to the baseboard and publishes them to the NVD and virtual network edge devices (such as various gateways, such as DRGs, SGWs, IGWs, etc.). Firewall rules can also be published using the same mechanism. In some embodiments, the NVD only receives mappings associated with that NVD. Data plane functions include the ability to actually route / forward packets based on the configuration set using the control plane. The VCN data plane is implemented by encapsulating client network packets before they traverse the baseboard network. Encapsulation / decapsulation functionality is implemented on the NVD. In some embodiments, the NVD is configured to intercept all network packets entering and leaving the host machine and perform network virtualization functions.

[0117] As indicated above, NVD performs various virtualization functions, including VNIC and VCN VR. NVD can execute VNICs associated with compute instances hosted on one or more host machines connected to the VNIC. For example, as... Figure 2 As depicted, NVD 210 performs the functionality of VNIC 276 associated with compute instance 268 hosted by host machine 202 connected to NVD 210. As another example, NVD 212 performs VNIC 280 associated with bare-metal compute instance 272 hosted by host machine 206, and VNIC 284 associated with compute instance 274 hosted by host machine 208. Host machines can host compute instances belonging to different VCNs (belonging to different customers), and NVDs connected to host machines can perform VNICs corresponding to compute instances (i.e., perform VNIC-related functionality).

[0118] NVD also executes a VCN virtual router corresponding to the VCN of the compute instance. For example, in Figure 2 In the embodiments depicted, NVD 210 executes VCN VR 277 corresponding to the VCN to which compute instance 268 belongs. NVD 212 executes one or more VCN VR 283 corresponding to one or more VCNs to which compute instances hosted by host machines 206 and 208 belong. In some embodiments, the VCN VR corresponding to a VCN is executed by all NVDs connected to host machines hosting at least one compute instance belonging to that VCN. If a host machine hosts compute instances belonging to different VCNs, then NVDs connected to that host machine can execute VCN VR corresponding to those different VCNs.

[0119] In addition to VNIC and VCN VR, NVD can execute various software (e.g., daemons) and include one or more hardware components that facilitate various network virtualization functions performed by NVD. For simplicity, these various components are grouped together as... Figure 2 The term "packet processing component" is shown in the diagram. For example, NVD 210 includes packet processing component 286 and NVD 212 includes packet processing component 288. For instance, a packet processing component for an NVD may include a packet processor configured to interact with the NVD's ports and hardware interfaces to monitor all packets received by and transmitted using the NVD and to store network information. This network information may include, for example, network flow information identifying different network flows handled by the NVD and per-flow information (e.g., per-flow statistics). In some embodiments, network flow information may be stored on a per-VNIC basis. The packet processor may perform per-packet manipulation and implement stateful NAT and L4 firewall (FW). As another example, a packet processing component may include a replication agent configured to replicate information stored by the NVD to one or more different replication target repositories. As yet another example, a packet processing component may include a logging agent configured to perform logging functions of the NVD. The packet processing component may also include software for monitoring the performance and health of the NVD and may also monitor the status and health of other components connected to the NVD.

[0120] Figure 1 The components of an example virtual or overlay network are shown, including a VCN, subnets within the VCN, compute instances deployed on the subnets, VNICs associated with the compute instances, VRs for the VCN, and a collection of gateways configured for the VCN. Figure 1 The overlay component described in the text can be made by Figure 2 One or more executions or hosts are described in the physical components. For example, a compute instance in a VCN can be executed or managed by... Figure 2 The VNIC described herein is executed or hosted by one or more host machines. For a compute instance hosted by a host machine, the VNIC associated with that compute instance is typically executed by an NVD connected to that host machine (i.e., VNIC functionality is provided by an NVD connected to that host machine). The VCN VR functionality for a VCN is executed by all NVDs connected to the host machine hosting or executing a compute instance as part of that VCN. The gateway associated with a VCN can be executed by one or more different types of NVDs. For example, some gateways can be executed by smartNICs, while others can be executed by one or more host machines or other implementations of NVDs.

[0121] As described above, compute instances in a client VCN can communicate with various endpoints, which can be within the same subnet as the source compute instance, in a different subnet but within the same VCN as the source compute instance, or communicate with endpoints located outside the VCN of the source compute instance. These communications are facilitated using VNICs, VCN VRs, and gateways associated with the VCNs.

[0122] For communication between two compute instances on the same subnet within a VCN, a VNIC associated with both the source and destination compute instances facilitates the communication. The source and destination compute instances can be hosted by the same host machine or different host machines. Packets originating from the source compute instance can be forwarded from the host machine hosting the source compute instance to an NVD connected to that host machine. On the NVD, packets are processed using a packet processing pipeline, which may include the execution of the VNIC associated with the source compute instance. Because the destination endpoint for the packet is within the same subnet, the execution of the VNIC associated with the source compute instance results in the packet being forwarded to the NVD executing the VNIC associated with the destination compute instance, whereupon the NVD processes the packet and forwards it to the destination compute instance. The VNIC associated with the source and destination compute instances can execute on the same NVD (e.g., when the source and destination compute instances are hosted by the same host machine) or on different NVDs (e.g., when the source and destination compute instances are hosted by different host machines connected to different NVDs). The VNIC can use a routing / forwarding table stored by the NVD to determine the next hop for the packet.

[0123] For packets destined for endpoints in different subnets within the same VCN, the packet originating from the source compute instance is forwarded from the host machine hosting the source compute instance to the NVD connected to that host machine. On the NVD, the packet is processed using a packet processing pipeline, which may include the execution of one or more VNICs and the VR associated with the VCN. For example, as part of the packet processing pipeline, the NVD executes or invokes functionality (also known as executing the VNIC) associated with the source compute instance. Functionality executed by the VNIC may include viewing the VLAN tag on the packet. Since the packet's destination is outside the subnet, the VCN VR functionality is then invoked and executed by the NVD. The VCN VR then routes the packet to the NVD executing the VNIC associated with the destination compute instance. The VNIC associated with the destination compute instance then processes the packet and forwards it to the destination compute instance. The VNICs associated with the source and destination compute instances may execute on the same NVD (e.g., when the source and destination compute instances are hosted by the same host machine) or on different NVDs (e.g., when the source and destination compute instances are hosted by different host machines connected to different NVDs).

[0124] If the destination of the packet is outside the VCN of the source compute instance, the packet originating from the source compute instance is forwarded from the host machine hosting the source compute instance to the NVD connected to that host machine. The NVD executes the VNIC associated with the source compute instance. Since the destination endpoint of the packet is outside the VCN, the packet is subsequently processed by the VCN VR used by that VCN. The NVD invokes the VCN VR functionality, which results in the packet being forwarded to the NVD executing the appropriate gateway associated with the VCN. For example, if the destination is an endpoint within a customer's on-premises network, the packet may be forwarded by the VCN VR to the NVD executing the DRG gateway configured for the VCN. The VCN VR may execute on the same NVD as the NVD executing the VNIC associated with the source compute instance, or it may be executed by a different NVD. The gateway may be executed by the NVD, which may be a smartNIC, a host machine, or another NVD implementation. The packet is then processed by the gateway and forwarded to the next hop, which facilitates the forwarding of the packet to its intended destination endpoint. For example, in Figure 2 In the embodiment depicted, packets originating from compute instance 268 can be transmitted from host machine 202 to NVD 210 via link 220 (using NIC 232). On NVD 210, VNIC 276 is invoked because it is the VNIC associated with the source compute instance 268. VNIC 276 is configured to examine the information encapsulated in the packet and determine the next hop for forwarding the packet, with the aim of facilitating the transmission of the packet to its intended destination endpoint, and then forwarding the packet to the determined next hop.

[0125] Compute instances deployed on a VCN can communicate with a variety of endpoints. These endpoints can include endpoints hosted by CSPI 200 and endpoints outside of CSPI 200. Endpoints hosted by CSPI 200 can include instances within the same VCN or other VCNs, which can be the customer's VCN or a VCN not belonging to the customer. Communication between endpoints hosted by CSPI 200 can be performed over physical network 218. Compute instances can also communicate with endpoints not hosted by CSPI 200 or outside of CSPI 200. Examples of these endpoints include endpoints within the customer's on-premises network or data center, or public endpoints accessible via public networks such as the Internet. Communication with endpoints outside of CSPI 200 can use various communication protocols over public networks (e.g., the Internet). Figure 2 (not shown in the image) or a dedicated network ( Figure 2 (Not shown in the image) to execute.

[0126] Figure 2The architecture of the CSPI 200 depicted herein is merely an example and is not intended to be limiting. Variations, alternatives, and modifications are possible in alternative embodiments. For example, in some implementations, the CSPI 200 may have a more advanced architecture than... Figure 2 The systems or components shown may include more or fewer systems or components, and may combine two or more systems, or may have different system configurations or arrangements. Figure 2 The systems, subsystems, and other components described herein may be implemented in software (e.g., code, instructions, programs) executed by one or more processing units (e.g., processors, cores) of the respective system, using hardware, or a combination thereof. The software may be stored on a non-transitory storage medium (e.g., a memory device).

[0127] Figure 4 The diagram depicts a connection between a host machine and an NVD, according to certain embodiments, for providing I / O virtualization to support multi-tenancy. For example... Figure 4 As depicted, host machine 402 executes a hypervisor 404 that provides a virtualized environment. Host machine 402 executes two virtual machine instances, VM1 406 belonging to customer / tenant #1 and VM2 408 belonging to customer / tenant #2. Host machine 402 includes a physical NIC 410 connected to NVD 412 via link 414. Each compute instance is attached to a VNIC executed by NVD 412. Figure 4 In the embodiment, VM1 406 is attached to VNIC-VM1 420 and VM2 408 is attached to VNIC-VM2 422.

[0128] like Figure 4 As shown, NIC 410 includes two logical NICs, logical NIC A 416 and logical NIC B 418. Each virtual machine is attached to its own logical NIC and configured to work with its own logical NIC. For example, VM1 406 is attached to logical NIC A 416 and VM2 408 is attached to logical NIC B 418. Although host machine 402 includes only one physical NIC 410 shared by multiple tenants, due to the logical NICs, each tenant's virtual machines believe they have their own host machine and network interface card.

[0129] In some embodiments, each logical NIC is assigned its own VLAN ID. Therefore, a specific VLAN ID is assigned to logical NIC A 416 for tenant #1, and a separate VLAN ID is assigned to logical NIC B 418 for tenant #2. When a packet is transmitted from VM1 406, the label assigned to tenant #1 is appended to the packet by the hypervisor, and the packet is then transmitted from host machine 402 to NVD 412 via link 414. Similarly, when a packet is transmitted from VM2 408, the label assigned to tenant #2 is appended to the packet by the hypervisor, and the packet is then transmitted from host machine 402 to NVD 412 via link 414. Thus, a packet 424 transmitted from host machine 402 to NVD 412 has an associated label 426 identifying the specific tenant and the associated VM. On the NVD, for a packet 424 received from the host machine 402, the tag 426 associated with the packet is used to determine whether the packet is processed by VNIC-VM1 420 or VNIC-VM2 422. The packet is then processed by the corresponding VNIC. Figure 4 The configuration described in [the document] enables each tenant's compute instance to believe that they own their own host machine and NIC. Figure 4 The setup described in [the document] provides I / O virtualization to support multi-tenancy.

[0130] Figure 5 A simplified block diagram of a physical network 500 according to certain embodiments is depicted. Figure 5 The embodiments depicted are structured as Clos networks. Clos networks are a specific type of network topology designed to provide connectivity redundancy while maintaining high bandwidth and maximum resource utilization. Clos networks are non-blocking, multi-stage or multi-layer switching networks, where the number of stages or layers can be two, three, four, five, etc. Figure 5 The embodiment depicted is a Layer 3 network, including Layer 1, Layer 2, and Layer 3. TOR switch 504 represents a Layer 0 switch in a Clos network. One or more NVDs are connected to the TOR switch. Layer 0 switches are also referred to as edge devices of the physical network. Layer 0 switches are connected to Layer 1 switches, also known as leaf switches. Figure 5In the embodiments depicted, a set of "n" Layer 0 TOR switches is connected to a set of "n" Layer 1 switches, forming a pod. Each Layer 0 switch in the pod is interconnected to all Layer 1 switches in that pod, but there is no switch connectivity between pods. In some implementations, two pods are referred to as blocks. Each block is served by or connected to a set of "n" Layer 2 switches (sometimes called backbone switches). There can be several blocks in the physical network topology. The Layer 2 switches are then connected to "n" Layer 3 switches (sometimes called super backbone switches). Packet communication over the physical network 500 is typically performed using one or more Layer 3 communication protocols. Typically, all layers of the physical network (except the TOR layer) are n-way redundant, thus allowing high availability. Policies can be specified for pods and blocks to control the visibility of switches to each other in the physical network, thereby enabling the scaling of the physical network.

[0131] A key characteristic of Clos networks is that the maximum number of hops from one Layer 0 switch to another (or from an NVD connected to a Layer 0 switch to another NVD connected to a Layer 0 switch) is fixed. For example, in a Layer 3 Clos network, a packet takes a maximum of seven hops to reach another NVD, where the source and destination NVDs are connected to the leaf layers of the Clos network. Similarly, in a Layer 4 Clos network, a packet takes a maximum of nine hops to reach another NVD, where the source and destination NVDs are connected to the leaf layers of the Clos network. Therefore, the Clos network architecture maintains consistent latency throughout the network, which is important for communication within and between data centers. Clos topologies are horizontally scalable and cost-effective. Network bandwidth / throughput capacity can be easily increased by adding more switches at each layer (e.g., more leaf switches and backbone switches) and by increasing the number of links between switches in adjacent layers.

[0132] In some embodiments, each resource within the CSPI is assigned a unique identifier called a Cloud Identifier (CID). This identifier is included as part of the resource's information and can be used to manage the resource, for example, via a console or API. An example syntax for a CID is:

[0133] ocid1.<RESOURCE TYPE>.<REALM>.[REGION][.FUTURE USE].<UNIQUE ID>

[0134] in,

[0135] ocid1: A text string indicating the version of the CID;

[0136] resource type: The type of resource (e.g., instance, volume, VCN, subnet, user, group, etc.);

[0137] realm: The realm where the resource resides. Example values ​​are "c1" for the commercial realm, "c2" for the government cloud realm, or "c3" for the federal government cloud realm, etc. Each realm can have its own domain name;

[0138] region: The region where the resource is located. This section may be empty if the region is not applicable to the resource.

[0139] future use: to be reserved for future use.

[0140] Unique ID: The unique part of the ID. The format may vary depending on the type of resource or service.

[0141] B- Example Layer 2 VLAN Architecture

[0142] This section describes techniques for providing Layer 2 networking functionality in a virtualized cloud environment. Layer 2 functionality is provided in conjunction with and complements Layer 3 networking functionality provided by the virtualized cloud environment. In some embodiments, virtual Layer 2 and Layer 3 functionality are provided by Oracle Cloud Infrastructure (OCI) provided by Oracle Corporation.

[0143] After introducing Layer 2 network functionality, this section describes a Layer 2 implementation of VLANs. Following this, a description of providing Layer 2 VLAN services, including ACLs, is presented.

[0144] introduce

[0145] The number of enterprise customers transitioning their on-premises applications to cloud environments provided by cloud service providers (CSPs) continues to grow rapidly. However, many of these customers quickly realize that the transition to a cloud environment can be very bumpy, requiring them to rebuild and redesign their existing applications to make them work in the cloud environment. This is because applications written for on-premises environments often depend on the physical network's monitoring, availability, and scalability features. Therefore, these on-premises applications need to be rebuilt and redesigned before they can work in a cloud environment.

[0146] There are several reasons why on-premises applications cannot easily transition to cloud environments. One major reason is that current cloud virtual networks operate at Layer 3 of the OSI model, such as at the IP layer, and do not provide the Layer 2 capabilities required by applications. Layer 3-based routing or forwarding involves determining where a packet should be sent (e.g., to which customer instance) based on information contained in the Layer 3 header of the packet (e.g., the destination IP address contained in the Layer 3 header of the packet). To facilitate this, the location of IP addresses in the virtualized cloud network is determined through a centralized control and orchestration system or controller. These can include, for example, IP addresses associated with customer entities or resources within the virtualized cloud environment.

[0147] Many customers run applications in their on-premises environments that have stringent requirements for Layer 2 networking capabilities, issues that current cloud offerings and IaaS providers do not currently address. For example, traffic in current cloud offerings is routed using Layer 3 protocols with Layer 3 headers and does not support the Layer 2 features required by applications. These Layer 2 features may include Address Resolution Protocol (ARP) processing, Media Access Control (MAC) address learning and Layer 2 broadcast capabilities, Layer 2 (MAC-based) forwarding, Layer 2 networking constructs, and others. By providing virtualized Layer 2 networking functionality within a virtualized cloud network, as described in this disclosure, customers can now seamlessly migrate their legacy applications to cloud environments without any substantial rebuilding or redesign. For example, the virtualized Layer 2 networking capabilities described herein enable such applications (e.g., VMware vSphere, vCenter, vSAN, and NSX-T components) to communicate at Layer 2 as they would in an on-premises environment. These applications can run the same versions and configurations in the public cloud, allowing customers to utilize their legacy on-premises applications, including existing knowledge, tools, and processes associated with those applications. Customers are also able to access native cloud services from their applications (e.g., using VMware software-defined data centers (SDDC)).

[0148] As another example, several legacy on-premises applications (e.g., enterprise clustering software applications, network virtual appliances) require Layer 2 broadcast support for failover. Example applications include Fortinet FortiGate, IBM QRadar, Palo Alto firewall, Cisco ASA, Juniper SRX, and Oracle RAC (True Application Clustering). By providing virtualized Layer 2 networking in a virtualized public cloud as described in this disclosure, these applications can now run unchanged in a virtualized public cloud environment. Virtualized Layer 2 networking functionality equivalent to on-premises deployment is provided as described herein. The virtualized Layer 2 networking functionality described in this disclosure supports traditional Layer 2 networking. This includes support for customer-defined VLANs and unicast, broadcast, and multicast Layer 2 traffic functionality. Layer 2-based packet routing and forwarding involves routing or forwarding packets using Layer 2 protocols and information contained in the Layer 2 header of the packet, such as the destination MAC address contained in the Layer 2 header. Protocols used by enterprise applications (such as clustering software applications), such as ARP, gratuitous address resolution protocol (GARP), and reverse address resolution protocol (RARP), can now also work in cloud environments.

[0149] Traditional virtualized cloud infrastructures support Layer 3 networking but not Layer 2 networking for several reasons. Layer 2 networks typically cannot scale as quickly as Layer 3 networks. Layer 2 network control protocols lack the complexity expected of scaling. For example, Layer 3 networks do not have to worry about packet looping, a problem that Layer 2 networks must solve. IP packets (i.e., Layer 3 packets) have a Time-to-Live (TTL) concept, while Layer 2 packets do not. IP addresses contained in Layer 3 packets have topological concepts such as subnets and CIDR ranges, while Layer 2 addresses (e.g., MAC addresses) do not. Layer 3 IP networks have built-in tools that facilitate troubleshooting, such as ping and traceroute for finding path information. Such tools are unavailable for Layer 2. Layer 3 networks support multipathing, which is unavailable in Layer 2 networks. Due to the lack of sophisticated control protocols specifically designed for exchanging information between entities in the network (e.g., Border Gateway Protocol (BGP) and Open Shortest Path First (OSPF)), Layer 2 networks must rely on broadcast and multicast to learn the network sequentially, which negatively impacts network performance. As networks change, the learning process used for Layer 2 must be repeated, while that is not necessary for Layer 3. For these and other reasons, cloud IaaS service providers prefer to provide infrastructure that operates at Layer 3 rather than Layer 2.

[0150] However, despite several drawbacks, many on-premises applications still require Layer 2 functionality. For example, consider a virtualized cloud configuration where a customer (customer 1) has two instances in a virtual network "V," instance A with IP1 and instance B with IP2. These instances can be compute instances (e.g., bare metal, virtual machines, or containers) or service instances (such as load balancers, NFS mount points, or other service instances). Virtual network V is a unique address space isolated from other virtual networks and the underlying physical network. This isolation can be achieved using various techniques, including packet encapsulation or NAT. For this reason, the IP address of the instance in the virtual network used by the customer is different from the address in the physical network hosting it. A centralized SDN (Software-Defined Networking) control plane is provided, which knows the physical IP and virtual interface of all virtual IP addresses. When a packet is sent from instance A to its destination IP2 in virtual network V, the virtual network SDN stack needs to know the location of IP2. It must know this in advance so that it can send the packet to the IP in the physical network hosting the virtual IP address IP2 used for V. The location of virtual IP addresses can be modified in the cloud, thereby altering the relationship between physical IP addresses and virtual IP addresses. Whenever a virtual IP address needs to be moved (e.g., moving an IP address associated with a virtual machine to another virtual machine or migrating a virtual machine to a new physical host), an API call must be made to the SDN control plane to inform the controller that the IP is moving so it can update all participants in the SDN stack, including the packet processor (data plane). However, some application classes do not make such API calls. Examples include various on-premises applications and applications provided by various virtualization software vendors (such as VMware). The value of facilitating virtual Layer 2 networking in a virtualized cloud environment lies in enabling applications not programmed to make such API calls or applications that rely on other Layer 2 networking features, such as supporting non-IP Layer 3 and MAC learning.

[0151] Virtual Layer 2 networks create broadcast domains, where learning is performed by members of the broadcast domain. Within a virtual Layer 2 domain, any IP address can be on any MAC address of any host within that domain, and the system will learn to use standard Layer 2 networking protocols. These networking primitives are virtualized without the central controller explicitly informing the MAC and IP addresses of their locations within the virtual Layer 2 network. This enables applications requiring low-latency failover, applications supporting broadcast or multicast protocols to multiple nodes, and legacy applications that don't know how to make API calls to the SDN control plane or API endpoints to determine IP and MAC address locations. Therefore, Layer 2 networking capabilities are needed in virtualized cloud environments to support functionality not available at the IP Layer 3 level.

[0152] Another technical advantage of providing virtual Layer 2 in a virtualized cloud environment is that it enables support for a variety of different Layer 3 protocols (such as IPv4 and IPv6), including non-IP protocols. For example, various non-IP protocols, such as IPX and AppleTalk, can be supported. Because existing cloud IaaS providers do not provide Layer 2 functionality in their virtualized cloud networks, they cannot support these non-IP protocols. By providing the Layer 2 networking functionality described in this disclosure, support can be provided for protocols at Layer 3, as well as for applications that require and rely on the availability of Layer 2 level functionality.

[0153] Using the techniques described in this disclosure, both Layer 3 and Layer 2 functionality are provided in a virtualized cloud infrastructure. As previously mentioned, Layer 3-based networking offers certain efficiencies, particularly suited for scalability, which Layer 2 networking does not provide. Providing Layer 2 functionality in addition to Layer 3 functionality allows full utilization of such efficiencies offered by Layer 3 (e.g., providing a more scalable solution) while providing Layer 2 functionality in a more scalable manner. For example, virtualized Layer 3 avoids the use of broadcasting for learning purposes. By providing Layer 3 for increased efficiency, while providing virtualized Layer 2 to enable applications that require it and those that cannot function without Layer 2 functionality, and supporting non-IP protocols, etc., customers are provided with complete flexibility in a virtualized cloud environment.

[0154] Customers typically have hybrid environments where Layer 2 and Layer 3 environments coexist, and the virtualized cloud environment can now support both. Customers can have Layer 3 networks such as subnets and / or Layer 2 networks such as VLANs, and these two environments can communicate with each other within the virtualized cloud environment.

[0155] Virtualized cloud environments also need to support multi-tenancy. Multi-tenancy makes it technically difficult and complex to provision both Layer 3 and Layer 2 functionality within the same virtualized cloud environment. For example, a Layer 2 broadcast domain must be managed across many different customers in the cloud provider's infrastructure. The embodiments described in this disclosure overcome these technical problems.

[0156] For virtualization providers (e.g., VMware), virtualized Layer 2 networks that emulate physical Layer 2 networks allow workloads to run without modification. Applications provided by such providers can then run on virtualized Layer 2 networks provided by cloud infrastructure. For example, such applications may include a collection of instances that require a Layer 2 network to run. When customers want to elevate such applications from their on-premises environments to virtualized cloud environments, they cannot simply acquire the application and run it in the cloud because those applications rely on the underlying Layer 2 network (e.g., Layer 2 network features are used to perform virtual machine migrations or move the location of MAC and IP addresses), which is not currently provided by virtualized cloud providers. For these reasons, such applications cannot run natively in virtualized cloud environments. Using the techniques described herein, cloud providers provide virtualized Layer 2 networks in addition to providing virtualized Layer 3 networks. Now, such application stacks can run in cloud environments without modification, and nested virtualizations can run in cloud environments. Customers can now run and manage their own Layer 2 applications in the cloud. Application providers do not need to make any changes to their software to facilitate this. Such legacy applications or workloads (e.g., legacy load balancers, legacy applications, KVM, OpenStack, clustering software) can now run unchanged in a virtualized cloud environment.

[0157] By providing the Layer 2 functionality of virtualization as described in this article, virtualized cloud environments can now support a wide range of Layer 3 protocols, including non-IP protocols. Taking Ethernet as an example, various different EtherTypes (a field in the Layer 2 header that tells the type of Layer 3 packet being sent; it tells what protocol is expected at Layer 3) can be supported, including a variety of non-IP protocols. EtherType is a two-octet field in an Ethernet frame. It is used to indicate which protocol is encapsulated in the frame's payload and is used by the data link layer at the receiving end to determine how to process the payload. EtherType also serves as the basis for 802.1Q VLAN tagging, encapsulating packets from VLANs for multiplexing with other VLAN traffic over Ethernet relay. Examples of EtherTypes include IPv4, IPv6, Address Resolution Protocol (ARP), AppleTalk, IPX, etc. Cloud networks that support Layer 2 protocols can support any protocol at Layer 3. Similarly, when cloud infrastructure provides support for Layer 3 protocols, it can support a variety of protocols at Layer 4, such as TCP, UDP, ICMP, etc. When virtualization is provided at Layer 3, the network can be unaffected by Layer 4 protocols. Similarly, when virtualization is provided at Layer 2, the network can be unaffected by Layer 3 protocols. This technology can be extended to support any Layer 2 network type, including FDDI, Infiniband, etc.

[0158] Therefore, many applications written for physical networks (especially those working with clusters of computer nodes sharing a broadcast domain) utilize Layer 2 features not supported by L3 virtual networks. The following six examples highlight the complexities that can result from not providing Layer 2 networking capabilities:

[0159] (1) Assigning MAC and IP addresses without prior API calls. Network appliances and management programs (such as VMware) are not built for cloud virtual networks. They assume they can use MAC addresses as long as they are unique, and either obtain dynamic addresses from a DHCP server or use any IP address assigned to the cluster. Often there is no mechanism to configure them to inform the control plane about the assignment of these Layer 2 and Layer 3 addresses. If the location of the MAC and IP addresses is unknown, then the Layer 3 virtual network does not know where to send traffic.

[0160] (2) Low-latency reallocation of MAC and IP addresses for high availability and live migration. Many on-premises applications use ARP to reassign IP and MAC addresses for high availability—when an instance in a cluster or HA pair stops responding, the new active instance sends a gratuitous ARP (GARP) to reassign the service IP to its MAC address or a reverse ARP (RARP) to reassign the service MAC address to its interface. This is also important when live migrating instances on a hypervisor: the new host must send RARP after a guest migration to redirect guest traffic to the new host. The assignment not only requires no API calls but also extremely low latency (sub-milliseconds). This cannot be accomplished via HTTPS calls to REST endpoints.

[0161] (3) Interface multiplexing via MAC addresses. When a hypervisor hosts multiple virtual machines on a single host, all of these virtual machines are on the same network, and guest interfaces are distinguished by their MAC addresses. This requires support for multiple MAC addresses on the same virtual interface.

[0162] (4) VLAN support. A single physical virtual machine host will need to be located on multiple broadcast domains, as indicated by VLAN tags. For example, VMware ESX uses VLANs for traffic separation (e.g., a guest virtual machine can communicate on one VLAN, store on another VLAN, and host the virtual machine on yet another VLAN).

[0163] (5) Use of broadcast and multicast traffic. ARP requires L2 broadcast, and there are examples of on-premises applications using broadcast and multicast traffic for cluster and HA applications.

[0164] (6) Support for non-IP traffic. Since L3 networks require IPv4 or IPv6 headers for communication, using any L3 protocol other than IP will not work. L2 virtualization means that the network within a VLAN can be independent of L3 protocols—the L3 header can be IPv4, IPv6, IPX or anything else—or even not exist at all.

[0165] Layer 2 VLAN Implementation

[0166] As disclosed herein, a Layer 2 (L2) network can be created within a cloud network. This virtual L2 network includes one or more Layer 2 virtual networks, such as virtualized L2 VLANs, referred to herein as VLANs. Each VLAN may include multiple compute instances, each of which may be associated with at least one L2 virtual network interface (e.g., an L2 VNIC) and an L2 virtual switch. In some embodiments, each pair of L2 virtual network interfaces and L2 virtual switches is hosted on an NVD. An NVD may host multiple such pairs, each associated with a different compute instance. A set of L2 virtual switches represents a simulated single L2 switch for a VLAN. An L2 virtual network interface represents a set of L2 ports on a simulated single L2 switch. VLANs may be connected to other VLANs, Layer 3 (L3) networks, on-premises networks, and / or other networks via VLAN Switching and Routing Service (VSRS) (also referred to herein as a Real Virtual Router (RVR) or L2 VSRS). An example of this architecture is described below.

[0167] Now for reference Figure 6The diagram illustrates one embodiment of a computing network. VCN 602 resides in CSPI 601. VCN 602 includes multiple gateways connecting VCN 602 to other networks. These gateways include DRG 604, which can connect VCN 602 to, for example, an on-premises network (such as on-premises data center 606). Gateways may also include gateway 600, which may include, for example, an LPG for connecting VCN 602 to another VCN, and / or an IGW and / or NAT gateway for connecting VCN 602 to the Internet. Gateways of VCN 602 may also include service gateway 610, which can connect VCN 602 to service network 612. Service network 612 may include one or more databases and / or repositories, including, for example, autonomous database 614 and / or object repository 616. Service network may include a conceptual network that includes an aggregation of IP ranges, such as public IP ranges. In some embodiments, these IP ranges may cover some or all of the public services provided by the CSPI 601 provider. For example, these services can be accessed through an Internet gateway or a NAT gateway. In some embodiments, the service network provides a way for services within the service network to be accessed from a local area via a dedicated gateway (service gateway) for that purpose. In some embodiments, the backends of these services may be implemented, for example, in their own private network. In some embodiments, service network 612 may include additional databases.

[0168] VCN 602 can include multiple virtual networks. Each of these networks can include one or more compute instances that can communicate within their respective networks, between networks, or outside of VCN 602. One of the virtual networks of VCN 602 is L3 subnet 620. L3 subnet 620 is a unit subdivision of a configuration created within VCN 602. Subnet 620 can include a virtual Layer 3 network within the virtualized cloud environment of VCN 602, which is hosted on the underlying physical network of CPSI 601. Although Figure 6 A single subnet 620 is depicted, but VCN 602 may have one or more subnets. Each subnet within VCN 602 may be associated with a contiguous range of IP addresses (e.g., 10.0.0.0 / 24 and 10.0.1.0 / 24) that does not overlap with other subnets within that VCN and represents a subset of the address space within that VCN's address space. In some embodiments, this IP address space may be isolated from the address space associated with CPSI 601.

[0169] Subnet 620 includes one or more computing instances, and specifically includes a first computing instance 622-A and a second computing instance 622-B. Computing instances 622-A and 622-B can communicate with each other within subnet 620, or they can communicate with other instances, devices, and / or networks outside subnet 620. Communication outside subnet 620 is enabled by a virtual router (VR) 624. VR 624 enables communication between subnet 620 and other networks of VCN 602. For subnet 620, VR 624 represents a logical gateway that enables subnet 620 (e.g., computing instances 622-A and 622-B) to communicate with endpoints on other networks within VCN 602 and with other endpoints outside VCN 602.

[0170] The VCN 602 may also include additional networks, and specifically may include one or more L2 VLANs (referred to herein as VLANs), which are examples of virtual L2 networks. These one or more VLANs may each comprise a virtual Layer 2 network located in the cloud environment of the VCN 602 and / or hosted by the underlying physical network of the CPSI 601. Figure 6In this embodiment, VCN 602 includes VLAN A 630 and VLAN B 640. Each VLAN 630, 640 within VCN 602 may be associated with a contiguous range of IP addresses (e.g., 10.0.0.0 / 24 and 10.0.1.0 / 24) that do not overlap with other networks within the VCN (such as other subnets or VLANs within the VCN) and represent a subset of the address space within the VCN's address space. In some embodiments, this IP address space of the VLAN may be isolated from the address space associated with CPSI 601. Each of VLANs 630, 640 may include one or more compute instances, and specifically, VLAN A 630 may include, for example, a first compute instance 632-A and a second compute instance 632-B. In some embodiments, VLAN A 630 may include additional compute instances. VLAN B 640 may include, for example, a first compute instance 642-A and a second compute instance 642-B. Each of compute instances 632-A, 632-B, 642-A, and 642-B can have an IP address and a MAC address. These addresses can be assigned or generated in any desired manner. In some embodiments, these addresses can be within the CIDR of the compute instance's VLAN, and in some embodiments, these addresses can be any addresses. In embodiments where the compute instance in the VLAN communicates with endpoints outside the VLAN, one or both of these addresses come from the VLAN CIDR, while when all communication is within the VLAN, these addresses are not limited to addresses within the VLAN CIDR. Unlike networks where addresses are assigned by the control plane, the IP and / or MAC addresses of compute instances within a VLAN can be assigned by the users / clients of that VLAN, and these IP and / or MAC addresses can then be discovered and / or learned by the compute instances in the VLAN according to the learning process discussed below.

[0171] Each VLAN may include a VLAN Switching and Routing Service (VSRS), and specifically, VLAN A 630 includes VSRS A 634 and VLAN B 640 includes VSRS B 644. Each VSRS 634, 644 participates in Layer 2 switching and local learning within the VLAN and also performs all necessary Layer 3 network functions, including ARP, NDP, and routing. VSRS performs ARP (a Layer 2 protocol) because VSRS must map IPs to MAC addresses.

[0172] Within these cloud-based VLANs, each virtual interface or virtual gateway can be associated with one or more Media Access Control (MAC) addresses, which can be virtual MAC addresses. Within a VLAN, one or more compute instances 632-A, 632-B, 642-A, 642-B (e.g., bare metal, VMs, or containers, and / or one or more service instances) can communicate directly with each other via virtual switches. Communication outside the VLAN, such as with other VLANs or with L3 networks, is enabled via VSRS 634, 644. VSRS 634, 644 are distributed services that provide Layer 3 functionality, such as IP routing, to the VLAN network. In some embodiments, VSRS 634, 644 are horizontally scalable, highly available routing services that can reside at the intersection of IP and L2 networks and participate in IP routing and L2 learning within the cloud-based L2 domain.

[0173] VSRS 634 and 644 can be distributed across multiple nodes within an infrastructure, and their functionality can be scalable, particularly horizontally scalable. In some embodiments, each node implementing the functionality of VSRS 634 and 644 shares and replicates the functionality of the router and / or switch with each other. Furthermore, these nodes can present themselves as a single VSRS 634 and 644 to all instances in VLANs 630 and 640. VSRS 634 and 644 can be implemented on any virtualization device within the CSPI 601, specifically within a virtual network. Therefore, in some embodiments, VSRS 634 and 644 can be implemented on any virtual network virtualization device, including NICs, SmartNICs, switches, intelligent switches, or general-purpose computing hosts.

[0174] VSRS 634 and 644 can be services residing on one or more hardware nodes supporting a cloud network. These hardware nodes may be, for example, one or more servers, such as one or more x86 servers, or one or more networking devices supporting the cloud network, such as one or more NICs, and specifically one or more SmartNICs. In some embodiments, VSRS 634 and 644 can be implemented on a server cluster. Therefore, VSRS 634 and 644 can be services distributed across a cluster of nodes, which can be a centrally managed cluster or distributed at the edge of virtual networking executors that participate in and share L2 and L3 learning and evaluate routing and security policies. In some embodiments, each VSRS instance can update other VSRS instances with new mapping information, as this new mapping information is learned by the VSRS instance. For example, when a VSRS instance learns the IP, interface, and / or MAC mappings of one or more CIs in its VLAN, the VSRS instance can provide this updated information to other VSRS instances within the VCN. Through this cross-update, the VSRS instance associated with the first VLAN can know the mappings for CIs in other VLANs (in some embodiments, for CIs in other VLANs within VCN 602), including IP, interface, and / or MAC mappings. These updates can be significantly accelerated when the VSRS resides on a server cluster and / or is distributed across nodes.

[0175] In some embodiments, VSRS 634, 644 may also host one or more higher-level services necessary for networking, including but not limited to: DHCP relay; DHCP (managed); DHCPv6; neighbor discovery protocols (such as IPv6 neighbor discovery protocol); DNS; managed DNSv6; SLAAC for IPv6; NTP; metadata services; and blockstore mount points. In some embodiments, VSRS may support one or more Network Address Translation (NAT) functions for translation between network address spaces. In some embodiments, VSRS may incorporate anti-spoofing, anti-MAC spoofing, ARP cache poisoning protection for IPv4, IPv6 Route Advertisement (RA) protection, DHCP protection, packet filtering using Access Control Lists (ACLs); and / or reverse path forwarding checks. Functions that VSRS can implement include, for example, ARP, GARP, packet filtering (ACLs), DHCP relay, and / or IP routing protocols. For example, VSRS 634 and 644 can learn MAC addresses, invalidate expired MAC addresses, handle MAC address migration, vet MAC address information, handle MAC information flooding, handle storm control, prevent loops, multicast via Layer 2 protocols such as IGMP in the cloud, collect statistics including logs, collect statistics using SNMP, monitor, and / or collect and use statistics for broadcast, total traffic, bits, cross-tree groups, etc.

[0176] Within a virtual network, VSRS 634 and 644 can manifest as distinct instantiations. In some embodiments, each of these VSRS instantiations can be associated with VLAN 630 and 640, and in some embodiments, each VLAN 630 and 640 can have an instantiation of VSRS 634 and 644. In some embodiments, each instantiation of VSRS 634 and 644 can have one or more unique tables corresponding to the VLAN 630 and 640 associated with each instantiation of VSRS 634 and 644. Each instantiation of VSRS 634 and 644 can generate and / or orchestrate a unique table associated with that instantiation of VSRS 634 and 644. Therefore, while a single service can provide VSRS 634 and 644 functionality for one or more cloud networks, individual instantiations of VSRS 634 and 644 within a cloud network can have unique Layer 2 and Layer 3 forwarding tables, while multiple such customer networks can have overlapping Layer 2 and Layer 3 forwarding tables.

[0177] In some embodiments, VSRS 634, 644 can support conflicting VLANs and IP spaces across multiple tenants. This can include multiple tenants on the same VSRS 634, 644. In some embodiments, some or all of these tenants may choose to use some or all of the following: the same IP address space, the same MAC space, and the same VLAN space. This provides users with great flexibility in address selection. In some embodiments, this multi-tenancy is supported by providing each tenant with a different virtual network, which is a private network within a cloud network. Each virtual network is assigned a unique identifier. Similarly, in some embodiments, each host may have a unique identifier, and / or each virtual interface or virtual gateway may have a unique identifier. In some embodiments, these unique identifiers, and specifically the unique identifiers of the tenant's virtual network, may be encoded in each communication. By providing a unique identifier for each virtual network and including it in communications, a single instantiation of VSRS 634, 644 can serve multiple tenants with overlapping addresses and / or namespaces.

[0178] VSRS 634 and 644 can perform these switching and / or routing functions to facilitate and / or enable the creation and / or communication with L2 networks within VLANs 630 and 640. These VLANs 630 and 640 can be found within a cloud computing environment, and more specifically, within a virtual network within that cloud computing environment.

[0179] For example, each of VLANs 630 and 640 includes multiple compute instances 632-A, 632-B, 642-A, and 642-B. VSRS 634 and 644 enable communication between compute instances in one VLAN 630 or 640 and compute instances in another VLAN 630 or 640 or subnet 620. In some embodiments, VSRS 634 and 644 enable communication between compute instances in one VLAN 630 or 640 and another VCN, or another network outside that VCN (including the Internet, on-premises data centers, etc.). In such embodiments, for example, a compute instance (such as compute instance 632-A) may send communication to an endpoint outside the VLAN (in this example, an endpoint outside VLAN A 630). Compute instance (632-A) may send communication to VSRS A 634, which may direct the communication to routers 624 and 644 or gateways 604, 608, and 610 coupled to the desired endpoint. Routers 624, 644 or gateways 604, 608, 610 that are coupled to the desired endpoint can receive communication from the computing instance (632-A) and can direct the communication to the desired endpoint.

[0180] Now for reference Figure 7 The diagram illustrates the logical and hardware schematics of VLAN 700. As can be seen, VLAN 700 includes multiple endpoints, specifically multiple compute instances and VSRS. Multiple compute instances (CIs) are instantiated on one or more host machines. In some embodiments, this can be a one-to-one relationship, such that each CI is instantiated on a unique host machine, and / or in some embodiments, this can be a many-to-one relationship, such that multiple CIs are instantiated on a single shared host machine. In various embodiments, CIs can be Layer 2 CIs by being configured to communicate with each other using the L2 protocol. Figure 7 This describes a scenario where some CIs are instantiated on a single host machine, and some CIs share a common host machine. For example... Figure 7 As seen in the diagram, instance 1 (CI1) 704-A is instantiated on host machine 1 702-A, instance 2 (CI2) 704-B is instantiated on host machine 2 702-B, and instance 3 (CI3) 704-AC and instance 4 (CI4) 704-D are instantiated on the shared host machine 702-C.

[0181] Each of CI 704-A, 704-B, 704-C, and 704-D is communicatively coupled to the other CI 704-A, 704-B, 704-C, and 704-D in VLAN 700 and to VSRS 714. Specifically, each of CI 704-A, 704-B, 704-C, and 704-D is connected to the other CI 704-A, 704-B, 704-C, and 704-D in VLAN 700 via an L2 VNIC and a switch, and is connected to VSRS 714. Each CI 704-A, 704-B, 704-C, and 704-D is associated with a unique L2 VNIC and switch. The switch can be a local L2 virtual switch and is uniquely associated with and deployed for the L2 VNIC. Specifically, CI1 704-A is associated with L2 VNIC 1 708-A and switch 1 710-A, CI2 704-B is associated with L2 VNIC 2 708-B and switch 710-B, CI3 704-C is associated with L2 VNIC 3 708-C and switch 3 710-C, and CI4 704-D is associated with L2 VNIC 4 708-D and switch 4 710-D.

[0182] In some embodiments, each L2 VNIC 708 and its associated switch 710 can be instantiated on NVD 706. This instantiation can be a one-to-one relationship, whereby a single L2 VNIC 708 and its associated switch 710 are instantiated on a unique NVD 706, or it can be a many-to-one relationship, where multiple L2 VNIC 708s and their associated switches 710s are instantiated on a single shared NVD 706. Specifically, L2 VNIC 1 708-A and switch 1 710-A are instantiated on NVD 1 706-A, L2 VNIC 2 708-B and switch 2 710-B are instantiated on NVD 2, and L2 VNIC 3 708-C and switch 3 710-C, as well as L2 VNIC 4 708-D and switch 710-D, are all instantiated on a shared NVD (i.e., NVD 706-C).

[0183] In some embodiments, the VSRS 714 can support conflicting VLANs and IP spaces across multiple tenants. This can include multiple tenants on the same VSRS 714. In some embodiments, some or all of these tenants may choose to use some or all of the following: the same IP address space, the same MAC space, and the same VLAN space. This provides users with great flexibility in address selection. In some embodiments, this multi-tenancy is supported by providing each tenant with a different virtual network, which is a private network within the cloud network. Each virtual network (e.g., each VLAN or VCN) is assigned a unique identifier, such as a VCN identifier that can be a VLAN identifier. This unique identifier may be selected by, for example, the control plane, and specifically by the CSPI control plane. In some embodiments, this unique VLAN identifier may include one or more bits that may be included and / or used in packet encapsulation. Similarly, in some embodiments, each host may have a unique identifier, and / or each virtual interface or virtual gateway may have a unique identifier. In some embodiments, these unique identifiers, and specifically the unique identifiers of the tenant's virtual network, may be encoded in each communication. By providing a unique identifier for each virtual network and including it in communications, a single instantiation of VSRS can serve multiple tenants with overlapping addresses and / or namespaces. In some embodiments, VSRS 714 can determine which tenant a packet belongs to based on the VCN identifier and / or VLAN identifier associated with the communication, specifically within the VCN header of the communication. In the embodiments disclosed herein, communications leaving or entering a VLAN may have a VCN header that may include a VLAN identifier. Based on the VCN header containing the VLAN identifier, VSRS 714 can determine the lease, or in other words, the receiving VSRS can determine which VLAN and / or which tenant to send the communication to. Furthermore, each compute instance belonging to a VLAN (e.g., an L2 compute instance) is assigned a unique interface identifier that identifies the L2 VNIC associated with the compute instance. The interface identifier can be included in traffic to and / or to the compute instance (e.g., by including it in the header of a frame) and can be used by the NVD to identify the L2 VNIC associated with the compute instance. In other words, the interface identifier can uniquely indicate the compute instance and / or its associated L2 VNIC.

[0184] like Figure 7As indicated in the document, switches 710-A, 710-B, 710-C, and 710-D can be combined to form an L2 distributed switch 712, also referred to herein as a distributed switch 712. From a customer's perspective, each switch 710-A, 710-B, 710-C, and 710-D in the L2 distributed switch 712 is a single switch connected to all CIs in the VLAN. However, the L2 distributed switch 712, simulating the user experience of a single switch, is infinitely scalable and includes a collection of local switches (e.g., in...). Figure 7 In the illustrative examples, switches 710-A, 710-B, 710-C, and 710-D are shown. Figure 7 As shown, each CI executes on a host machine connected to the NVD. For each CI on a host connected to the NVD, the NVD hosts a Layer 2 VNIC and a local switch associated with the compute instance (e.g., an L2 virtual switch, local to the NVD, associated with a Layer 2 VNIC, and a member or component of an L2 distributed switch 712). A Layer 2 VNIC represents a port of the compute instance on a Layer 2 VLAN. The local switch connects the VNIC to other VNICs (e.g., other ports) associated with other compute instances on other Layer 2 VLANs.

[0185] Each of CI 704-A, 704-B, 704-C, and 704-D can communicate with the other CI 704-A, 704-B, 704-C, and 704-D in VLAN 700, or with VSRS 714. One of CI 704-A, 704-B, 704-C, and 704-D sends a frame to another of CI 704-A, 704-B, 704-C, and 704-D or to VSRS 714 by sending the MAC address and interface identifier of the receiver in CI 704-A, 704-B, 704-C, or 704-D, or to VSRS 714. The MAC address and interface identifier can be included in the frame header. As explained above, the interface identifier can indicate the receiver of CI 704-A, 704-B, 704-C, 704-D or the L2 VNIC of VSRS 714.

[0186] In one embodiment, CI1 704-A can be the source CI, L2 VNIC 708-A can be the source L2 VNIC, and switch 710-A can be the source L2 virtual switch. In this embodiment, CI3 704-C can be the destination CI, and L2VNIC 3 708-C can be the destination L2 VNIC. The source CI can send a frame with the source MAC address and the destination MAC address. This frame can be intercepted by NVD 706-A, thereby instantiating the source VNIC and the source switch.

[0187] For VLAN 700, L2 VNICs 708-A, 708-B, 708-C, and 708-D can each learn a mapping from MAC addresses to L2 VNIC interface identifiers. This mapping can be learned based on frames and / or communications received within VLAN 700. Based on this previously determined mapping, the source VNIC can determine the interface identifier of the destination interface associated with the destination CI within the VLAN and can encapsulate the frame. In some embodiments, this encapsulation can include GENEVE encapsulation, and specifically L2 GENEVE encapsulation, which includes the L2 (Ethernet) header of the encapsulated frame. The encapsulated frame can identify the destination MAC address, destination interface identifier, source MAC address, and source interface identifier.

[0188] The source VNIC can pass the encapsulated frame to the source switch, which can then direct the frame to the destination VNIC. Upon receiving the frame, the destination VNIC can decapsulate the frame and then provide it to the destination CI.

[0189] Now for reference Figure 8 This illustrates a logical diagram of multiple connected L2 VLAN 800 connections. Figure 8 In the specific embodiment depicted, both VLANs reside within the same VCN. As can be seen, multiple connected L2 VLANs 800 may include a first VLAN (VLAN A 802-A) and a second VLAN (VLAN B 802-B). Each of these VLANs 802-A and 802-B may include one or more CIs, each CI may have an associated L2 VNIC and an associated L2 virtual switch. Additionally, each of these VLANs 802-A and 802-B may include a VSRS.

[0190] Specifically, VLAN A 802-A may include Instance 1 804-A connected to L2 VNIC 1 806-A and Switch 1 808-A, Instance 2 804-B connected to L2 VNIC 2 806-B and Switch 808-B, and Instance 3 804-C connected to L2 VNIC 3806-C and Switch 3 808-C. VLAN B 802-B may include Instance 4 804-D connected to L2 VNIC 4 806-D and Switch 4 808-D, Instance 5 804-E connected to L2 VNIC 5 806-E and Switch 808-E, and Instance 6 804-F connected to L2 VNIC 6 806-F and Switch 3 808-F. VLAN A 802-A may also include VSRS A 810-A, and VLAN B 802-B may include VSRS B 810-B. Each of CI 804-A, 804-B, and 804-C of VLAN A 802-A may be communicatively coupled to VSRS A 810-A, and each of CIS 804-D, 804-E, and 804-F of VLAN B 802-B may be communicatively coupled to VSRS B 810-B.

[0191] VLAN A 802-A can be communicatively coupled to VLAN B 802-B via their respective VSRS 810-A, 810-B. Each VSRS can also be coupled to gateway 812, which provides access to other networks outside the VCN where VLANs 802-A and 802-B reside via CI 804-A, 804-B, 804-C, 804-D, 804-E, and 804-F in each VLAN 802-A and 802-B. In some embodiments, these networks may include, for example, one or more on-premises networks, another VCN, a service network, a public network such as the Internet, etc.

[0192] Each of the CIs 804-A, 804-B, and 804-C in VLAN A 802-A can communicate with CIs 804-D, 804-E, and 804-F in VLAN B 802-B via the VSRSs 810A and 810-B of each VLAN 802-A and 802-B. For example, one of the CIs 804-A, 804-B, 804-C, 804-D, 804-E, and 804-F in one of VLANs 802-A and 802-B can send a frame to one of the CIs 804-A, 804-B, 804-C, 804-D, 804-E, or 804-F in the other VLAN 802-A and 802-B. This frame can leave the source VLAN via the VSRS of the source VLAN and enter the destination VLAN, and is routed to the destination CI via the destination VSRS.

[0193] In one embodiment, CI 1 804-A can be a source CI, VNIC 806-A can be a source VNIC, and switch 808-A can be a source switch. In this embodiment, CI 5804-E can be a destination CI, and L2 VNIC 5 806-E can be a destination VNIC. VSRS A 810-A can be a source VSRS identified as an SVSRS, and VSRS B 810-B can be a destination VSRS identified as a DVSRS.

[0194] The source CI can send a frame with a MAC address. This frame can be intercepted by the NVD of the instantiated source VNIC and the source switch. The source VNIC encapsulates the frame. In some embodiments, this encapsulation may include GENEVE encapsulation, and specifically L2GENEVE encapsulation. The encapsulated frame can identify the destination address of the destination CI. In some embodiments, this destination address may also include the destination address of the destination VSRS. The destination address of the destination CI may include the destination IP address, the destination MAC address of the destination CI, and / or the destination interface identifier of the destination VNIC associated with the destination CI. The destination address of the destination VSRS may include the IP address of the destination VSRS, the interface identifier of the destination VNIC associated with the destination VSRS, and / or the MAC address of the destination VSRS.

[0195] The source VSRS can receive frames from the source switch, look up the VNIC mapping from the frame's destination address (which can be a destination IP address), and forward the packet to the destination VSRS. The destination VSRS can receive the frame. Based on the destination address contained in the frame, the destination VSRS can forward the frame to the destination VNIC. The destination VNIC can receive and decapsulate the frame, and then provide the frame to the destination CI.

[0196] Now for reference Figure 9 This illustrates a logical diagram of multiple connected L2 VLANs and subnet 900. Figure 9 In the specific embodiment shown, the VLAN and subnet reside within the same VCN. This is indicated by the virtual routers and VSRS of the VLAN and subnet being directly connected, rather than through a gateway.

[0197] As can be seen, this can include a first VLAN (VLAN A 902-A), a second VLAN (VLAN B 902-B), and subnet 930. Each of these VLANs 902-A and 902-B can include one or more CIs, each of which can have an associated L2 VNIC and an associated L2 switch. Additionally, each of these VLANs 902-A and 902-B can include a VSRS. Similarly, subnet 930, which can be an L3 subnet, can include one or more CIs, each of which can have an associated L3 VNIC, and L3 subnet 930 can include a virtual router 916.

[0198] Specifically, VLAN A 902-A may include Instance 1 904-A connected to L2 VNIC 1 906-A and Switch 1 908-A, Instance 2 904-B connected to L2 VNIC 2 906-B and Switch 908-B, and Instance 3 904-C connected to L2 VNIC 3 906-C and Switch 3 908-C. VLAN B 902-B may include Instance 4 904-D connected to L2 VNIC 4 906-D and Switch 4 908-D, Instance 5 904-E connected to L2 VNIC 5 906-E and Switch 908-E, and Instance 6 904-F connected to L2 VNIC 6 906-F and Switch 3 908-F. VLAN A 902-A may also include VSRS A 910-A, and VLAN B 902-B may include VSRS B 910-B. Each of the CIs 904-A, 904-B, and 904-C of VLAN A 902-A may be communicatively coupled to VSRS A 910-A, and each of the CIs 904-D, 904-E, and 904-F of VLAN B 902-B may be communicatively coupled to VSRS B 910-B. L3 subnet 930 may include one or more CIs, and specifically may include instance 7 904-G, which is communicatively coupled to L3 VNIC 7 906-G. L3 subnet 930 may include virtual router 916.

[0199] VLAN A 902-A can be communicatively coupled to VLAN B 902-B via their respective VSRS 910-A, 910-B. L3 subnet 930 can be communicatively coupled to VLAN A 902-A and VLAN B 902-B via virtual router 916. Each virtual router 916 and VSRS instances 910-A, 910-B can also be coupled to gateway 912, which provides access to other networks outside the VCN where VLANs 902-A, 902-B and subnet 930 reside (CI 904-A, 904-B, 904-C, 904-D, 904-E, 904-F, 904-G) reside in each of VLANs 902-A, 902-B and subnet 930. In some embodiments, these networks may include, for example, one or more on-premises networks, another VCN, a service network, a public network such as the Internet.

[0200] Each VSRS instance 910-A, 910-B can provide an exit path for frames leaving the associated VLAN 902-A, 902-B, and an entry path for frames entering the associated VLAN 902-A, 902-B. From VSRS instances 910-A, 910-B in VLAN 902-A, 902-B, frames can be sent to any desired endpoint, including L2 endpoints (such as L2 CIs in another VLAN on the same VCN or on a different VCN or network) and / or L3 endpoints (such as L3 CIs in a subnet on the same VCN or on a different VCN or network).

[0201] In one embodiment, CI 1 904-A can be a source CI, VNIC 906-A can be a source VNIC, and switch 908-A can be a source switch. In this embodiment, CI 7904-G can be a destination CI, and VNIC 7 906-G can be a destination VNIC. VSRS A 910-A can be a source VSRS identified as an SVSRS, and virtual router (VR) 916 can be a destination VR.

[0202] The source CI can send frames with MAC addresses. These frames can be intercepted by the instantiated source VNIC and the source switch's NVD. The source VNIC encapsulates the frame. In some embodiments, this encapsulation may include Geneve encapsulation, and specifically L2Geneve encapsulation. The encapsulated frame can identify the destination address of the destination CI. In some embodiments, this destination address may also include the destination address of the VSRS of the source CI's VLAN. The destination address of the destination CI may include the destination IP address, the destination MAC address of the destination CI, and / or the destination interface identifier of the destination VNIC of the destination CI.

[0203] The source VSRS can receive frames from the source switch, look up the VNIC mapping from the frame's destination address (which can be a destination IP address), and forward the frame to the destination VR. The destination VR can receive the frame. Based on the destination address contained in the frame, the destination VR can forward the frame to the destination VNIC. The destination VNIC can receive and decapsulate the frame, and then provide the frame to the destination CI.

[0204] Learning is performed through L2 VNICs and / or L2 virtual switches within the virtual L2 network.

[0205] Now for reference Figure 10 This diagram illustrates an embodiment of intra-VLAN communication and learning within VLAN 1000. The learning here is specific to how L2 VNICs, the VSRS of the source CI VLAN, and / or L2 virtual switches learn the association between MAC addresses and L2 VNICs / VSRS VNICs (more specifically, the association between MAC addresses associated with L2 compute instances or VSRSs and identifiers associated with the L2 VNICs of these L2 compute instances or with the VSRS VNICs). Generally, learning is based on inbound traffic. For the interface-to-MAC address learning aspect, this learning differs from the learning process (e.g., ARP process) that L2 compute instances can implement to learn destination MAC addresses. Both learning processes (e.g., the learning process of L2 VNICs / L2 virtual switches and the learning process of L2 compute instances) in... Figure 12 The diagram shows a joint implementation.

[0206] As can be seen, VLAN 1000 includes compute instance 1 1000-A, communicatively coupled to NVD 1 1001-A. NVD 1 1001-A instantiates L2 VNIC 1 1002-A and L2 switch 1 1004-A. VLAN 1000 also includes compute instance 2 1000-B, communicatively coupled to NVD 2 1001-B. NVD 2 1001-B instantiates L2 VNIC 2 1002-B and L2 switch 2 1004-A. VLAN 1000 also includes VSRS 1015 running on a server cluster, which includes VSRS VNIC 1002-C and VSRS switch 1004-C. All switches 1004-A, 1004-B, and 1004-C together form L2 distributed switch 1050. VSRS 1015 is communicatively coupled to endpoint 1008, which may include a gateway and, more specifically, may include, for example, another L2 / L3 router in the form of another VSRS, or an L3 router in the form of a virtual router.

[0207] The control plane 1010 of the VCN hosting VLAN 1000 maintains information identifying each L2 VNIC on VLAN 1000 and the network placement of the L2 VNICs. For example, for an L2 VNIC, this information may include the interface identifier associated with the L2 VNIC and / or the physical IP address of the NVD hosting the L2 VNIC. The control plane 1010 uses this information to update (e.g., periodically or on demand) the interfaces in VLAN 1000. Therefore, each L2 VNIC 1002-A, 1002-B, 1002-C in VLAN 1000 receives information identifying the interfaces in the VLAN from the control plane 1010 and populates a table with this information. The table populated by the L2 VNICs can be stored locally in the NVD hosting the L2 VNICs. If VNICs 1002-A, 1002-B, and 1002-C already include a current table, VNICs 1002-A, 1002-B, and 1002-C can determine any differences between their current tables and the information / tables received from control plane 1010. In some embodiments, VNICs 1002-A, 1002-B, and 1002-C can update their tables to match the information received from control plane 1010.

[0208] like Figure 10As seen in the diagram, frames are transmitted via L2 switches 1004-A, 1004-B, and 1004-C, and received by receiving VNICs 1002-A, 1002-B, and 1002-C. When a frame is received by VNICs 1002-A, 1002-B, and 1002-C, the VNIC learns the mapping between the source interface (source VNIC) and the source MAC address of the frame. Based on a table of information received from control plane 1010, the VNIC can map the source MAC address (from the received frame) to the interface identifier of the source VNIC and the IP address of the VNIC and / or the IP address of the NVD hosting the VNIC (where the interface identifier and (one or more) IP addresses are obtainable from the table). Therefore, L2 VNICs 1002-A, 1002-B, and 1002-C learn the mapping from interface identifier to MAC address based on received communications and / or frames. Each VNIC 1002-A, 1002-B, 1002-C can update its L2 forwarding (FWD) tables 1006-A, 1006-B, 1006-C using this learned mapping information. In some embodiments, the L2 forwarding table includes a MAC address and associates it with at least one of an interface identifier or a physical IP address. In such embodiments, the MAC address refers to the address assigned to the L2 compute instance and may correspond to a port emulated by the L2 VNIC associated with the L2 compute instance. The interface identifier can uniquely identify the L2 VNIC and / or the L2 compute instance. The virtual IP address can be the address of the L2 VNIC. And the physical IP address can be the IP address of the NVD hosting the L2 VNIC. The L2 forwarding updated by the L2 VNIC can be stored locally on the NVD hosting the L2 VNIC and used by the L2 virtual switch associated with the L2 VNIC to guide frames. In some embodiments, VNICs within a shared VLAN can share all or part of their mapping tables with each other.

[0209] Given the network architecture described above, traffic flow will now be described. For clarity, traffic flow will be described in conjunction with compute instance 21000-B, L2 VNIC 2 10002-B, L2 switch 2 1004-B, and NVD 2 1001-B. This description is equivalent to traffic flowing to and / or from other compute instances.

[0210] As explained above, VLANs are implemented within a VCN as an overlay L2 network on top of an L3 physical network. L2 compute instances of a VLAN can send or receive L2 frames that include an overlay MAC address (also known as a virtual MAC address) as the source and destination MAC addresses. L2 frames can also encapsulate packets that include an overlay IP address (also known as a virtual IP address) as the source and destination IP addresses. In some embodiments, the overlay IP address of the compute instance may belong to the CIDR range of the VLAN. Another overlay network IP address may belong to the CIDR range (in which case the L2 frame is within the VLAN) or flow outside the CIDR range (in which case the L2 frame is sent to or received from another network). L2 frames may also include a VLAN tag that uniquely identifies the VLAN and can be used to differentiate multiple L2 VNICs on the same NVD. L2 frames can be received by the NVD via tunnel in encapsulated packets from the host machine of the compute instance, from another NVD, or from a cluster of servers hosting VSRS. In these different cases, the encapsulated packets can be L3 packets sent over the physical network, where the source and destination IP addresses are physical IP addresses. Different types of encapsulation are possible, including GENEVE encapsulation. NVD can decapsulate received packets to extract L2 frames. Similarly, to transmit an L2 frame, NVD can encapsulate it in an L3 packet and transmit it on the physical substrate.

[0211] For intra-VLAN outbound traffic from Instance 2 1000-B, NVD 2 1001-B receives frames from the host machine of Instance 2 1000-B via the Ethernet link. The frame includes an interface identifier that identifies L2VNIC 2 1000-B. The frame includes the overlay MAC address of Instance 2 1000-B (e.g., M.2) as the source MAC address and the overlay MAC address of Instance 1 1000-A (e.g., M.1) as the destination MAC address. Given the interface identifier, NVD 21001-B passes the frame to L2 VNIC 2 1002-B for further processing. L2 VNIC 21002-B forwards the frame to L2 switch 2 1004-B. Based on L2 forwarding table 1006-B, L2 switch 2 1004-B determines whether the destination MAC address is known (e.g., matches an entry in L2 forwarding table 1006-B).

[0212] If known, L2 switch 2 1004-B determines that L2 VNIC 1 1002-A is the relevant tunnel endpoint and forwards the frame to L2 VNIC 1 1002-A. Forwarding may include frame encapsulation in packets and packet decapsulation (e.g., GENEVE encapsulation and decapsulation), where the packet includes the frame, the physical IP address of NVD 1 1001-A (e.g., IP.1) as the destination address, and the physical IP address of NVD 2 1001-B (e.g., IP.2) as the source address.

[0213] If the information is unknown, then L2 switch 2 1004-B broadcasts the frame to the individual VNICs of the VLAN (e.g., including L2VNIC 1 1002-A and any other L2VNICs in the VLAN), where the broadcast frame is processed (e.g., encapsulated, transmitted, decapsulated) among the relevant NVDs. In some embodiments, this broadcast can be performed, or more specifically simulated, at the physical network, thereby encapsulating the frame individually into each L2 VNIC, including the VSRS in the VLAN. Thus, the broadcast is simulated via a series of replicated unicast packets at the physical network. Furthermore, each L2 VNIC receives the frame and learns the association between the interface identifier of L2 VNIC 2 1002-B and the source MAC address (e.g., M.2) and source physical IP address (e.g., IP.2).

[0214] For intra-VLAN inbound traffic from compute instance 1 1000-A to compute instance 2 1000-B, NVD 2 1001-B receives packets from NVD 1. The packet has IP.1 as the source address and a frame, where the frame includes M.2 as the destination MAC address and M.1 as the source MAC address. The frame also includes the network identifier of L2 VNIC 1 1002-A. After decapsulation, VNIC 2 receives the frame and learns that this interface identifier is associated with M.1 and / or IP.1, and if this was previously unknown, stores the learned information in the L2 forwarding table 1006-B at switch 2 for subsequent outbound traffic. Alternatively, after decapsulation, L2VNIC 2 1002-B receives the frame and learns that this interface identifier is associated with M.1 and / or IP.1, and if this information is known, refreshes the expiration time.

[0215] For egress traffic from instance 2 1000-B in VLAN 1000 to an instance in another VLAN, similar flows to the egress traffic described above can exist, in addition to using VSRS VNICs and VSRS switches. Specifically, the destination MAC address is not within the L2 broadcast of VLAN 1000 (it is within another L2 VLAN). Therefore, the destination instance's overlay destination IP address (e.g., IP.A) is used for this egress traffic. For example, L2VNIC 2 1002-B determines that IP.A is outside the CIDR range of VLAN 1000. Therefore, L2VNIC 2 1002-B sets the destination MAC address to the default gateway MAC address (e.g., M.DG). Based on M.DG, L2 switch 2 1004-B sends the egress traffic to the VSRS VNIC (e.g., via tunnel, with appropriate end-to-end encapsulation). The VSRS VNIC forwards the egress traffic to the VSRS switch. Next, the VSRS switch performs routing functions, where, based on the destination IP address (e.g., IP.A), the VSRS switch in VLAN 1000 sends egress traffic to the VSRS switch in another VLAN (e.g., via a virtual router between the two VLANs, also with appropriate end-to-end encapsulation). The VSRS switch in the other VLAN then performs switching functions by determining that IP.A is within the CIDR range of this VLAN and performs a lookup in its ARP cache based on IP.A to determine the destination MAC address associated with IP.A. If no match is found in the ARP cache, an ARP request is sent to a different L2 VNIC in the other VLAN to determine the destination MAC address. Otherwise, the VSRS switch sends the egress traffic to the relevant VNIC (e.g., via a tunnel, with appropriate encapsulation).

[0216] For inbound traffic from an instance in another VLAN to an instance in VLAN 1000, the traffic flow is similar to the above, only in the reverse direction. For outbound traffic from an instance in VLAN 1000 to the L3 network, the traffic flow is similar to the above, except that the VSRS switch of VLAN 1000 routes packets directly to the destination VNIC in the virtual L3 network via a virtual router (e.g., without routing packets through another VSRS switch). For inbound traffic from the virtual L3 network to an instance in VLAN 1000, the traffic flow is similar to the above, except that packets are received by the VSRS switch of VLAN 1000A, which then sends them as frames within the VLAN. For traffic between VLAN 1000 and other networks (either outbound or inbound), a VSRS switch is used similarly, where its routing function is used for outbound traffic to send packets via appropriate gateways (e.g., IGW, NGW, DRG, SGW, LPG), and its switching function is used for inbound traffic to send frames within VLAN 1000.

[0217] Now for reference Figure 11 The diagram illustrates an embodiment of VLAN 1100 (e.g., a cloud-based virtual L2 network), and more specifically, it shows a view of VLAN implementation methods.

[0218] As described above, a VLAN can include "n" compute instances 1102-A, 1102-B, and 1102-N, each running on a host machine. As discussed earlier, there can be a one-to-one association between compute instances and host machines, or a many-to-one association between multiple compute instances and a single host machine. Each compute instance 1102-A, 1102-B, and 1102-N can be an L2 compute instance, in which case it is associated with at least one virtual interface (e.g., an L2 VNIC) 1104-A, 1104-B, and 1104-N and switches 1106-A, 1106-B, and 1106-N. Switches 1106-A, 1106-B, and 1106-N are L2 virtual switches and together form an L2 distributed switch 1107.

[0219] The pairs of L2VNICs 1104-A, 1104-B, 1104-N and switches 1106-A, 1106-B, 1106-N associated with compute instances 1102-A, 1102-B, 1102-N on the host machine are a pair of software modules connected to NVDs 1108-A, 1108-B, 1108-N on the host machine. Each L2 VNIC 1104-A, 1104-B, 1104-N represents an L2 port of a single switch (referred to herein as vswitch) perceived by the client. Generally, host machine “i” executes compute instance “i” and connects to NVD “i”. NVD “i” then executes L2VNIC “i” and switch “i”. L2 VNIC “i” represents L2 port “i” of vswitch. “i” is a positive integer between 1 and “n”. Here, while a one-to-one association is described, other types of associations are also possible. For example, a single NVD can connect to multiple hosts, each executing one or more compute instances belonging to a VLAN. If this is the case, then the NVD hosts multiple pairs of L2 VNICs and switches, each pair corresponding to one of the compute instances.

[0220] VLANs can include instances of VSRS 1110. VSRS 1110 performs switching and routing functions and includes instances of VSRSVNIC 1112 and VSRS switch 1114. VSRS VNIC 1112 represents a port on the vswitch, where this port connects the vswitch to other networks via a virtual router. As shown in the diagram, VSRS 1110 can be instantiated on server cluster 1116.

[0221] Control plane 1118 can track and identify L2 VNICs 1104-A, 1104-B, and 1104-N and their placement within VLANs. Control plane 1110 can also provide this information to L2 interfaces 1104-A, 1104-B, and 1104-N within the VLANs.

[0222] like Figure 11 As shown, the VLAN can be a cloud-based virtual L2 network that can be built on top of the physical network 1120. In some embodiments, this physical network 1120 may include NVD 1108-A, 1108-B, and 1108-N.

[0223] Generally, the first L2 compute instance of a VLAN (e.g., compute instance 1 1102-A) can communicate with the second compute instance of the VLAN (e.g., compute instance 2 1102-B) using L2 protocols. For example, frames can be sent between these two L2 compute instances via VLAN. However, frames can be encapsulated, tunneled, routed, and / or otherwise processed so that frames can be sent through the underlying physical network 1120.

[0224] For example, compute instance 1 1102-A sends a frame destined for compute instance 2 1102-B. Depending on the network connections (e.g., TCP / IP connection, Ethernet connection, tunnel connection, etc.) between host machine 1 and NVD 1, NVD 1 and physical network 1120, physical network 1120 NVD 2, and NVD 2 and host machine 2, different types of processing can be applied to the frame. For example, the frame is received and encapsulated by NVD 1, and so on, until the frame reaches compute instance 2. It is assumed that this processing allows frames to be sent between underlying physical resources, and for the sake of brevity, its description is omitted when describing VLANs and related L2 operations.

[0225] Virtual L2 network communication

[0226] Various forms of communication can occur within or with a virtual L2 network. These can include intra-VLAN communication. In such embodiments, a source compute instance can send communication to a destination compute instance that is in the same VLAN as the source compute instance (CI). Communication can also be sent to an endpoint outside the VLAN of the source CI. This can include, for example, communication between a source CI in a first VLAN and a destination CI in a second VLAN, communication between a source CI in a first VLAN and a destination CI in an L3 subnet, and / or communication from a source CI in a first VLAN to a destination CI outside the VCN of the VLAN containing the source CI. This communication can also include, for example, receiving communication from a source CI outside the VLAN of the destination CI at the destination CI. This source CI can be located in another VLAN, in an L3 subnet, or outside the VCN of the VLAN containing the source CI.

[0227] Each CI within a VLAN can play an active role in traffic flow. This includes learning interface identifiers to MAC addresses (also referred to herein as interface-to-MAC address mapping), mapping instances within the VLAN to maintain the L2 forwarding table within the VLAN, and sending and / or receiving communications (e.g., frames in the case of L2 communication). VSRS can play an active role in communication within the VLAN as well as in communication with source or destination CIs outside the VLAN. VSRS can maintain presence in both L2 and L3 networks to enable egress and ingress communication.

[0228] Now for reference Figure 12 The diagram illustrates a flowchart of one embodiment of a process 1200 for communication within a VLAN. In some embodiments, process 1200 may be executed by a computing instance within a shared VLAN. This process may be specifically executed when a source CI sends communication to a destination CI within the VLAN, but does not know the IP-to-MAC address mapping of that destination CI. For example, this occurs when a source CI sends a packet to a destination CI that has an IP address in the VLAN, but the source CI does not know the MAC address of that IP address. In this case, an ARP procedure may be performed to learn the destination MAC address and the IP-to-MAC address mapping.

[0229] If the source CI knows the IP-to-MAC address mapping, it can send packets directly to the destination CI without performing an ARP procedure. In some embodiments, this packet can be intercepted by the source VNIC, which is an L2VNIC in intraVLAN communication. If the source VNIC knows the interface-to-MAC address mapping used for the destination MAC address, it can encapsulate the packet, for example, in L2 encapsulation, and forward the corresponding frame to the destination VNIC, which is an L2VNIC used for the destination MAC address in intraVLAN communication.

[0230] If the source VNIC does not know the interface-to-MAC address mapping used for MAC addresses, it can perform one aspect of the interface-to-MAC address learning process. This can include the source VNIC sending a frame to all interfaces in the VLAN. In some embodiments, this frame can be broadcast to all interfaces within the VLAN. In some embodiments, this broadcast can be implemented at the physical network level as a serial unicast. The frame can include the destination MAC and IP addresses, the interface identifier, and the source VNIC's MAC and IP addresses. Each VNIC in the VLAN can receive this frame and learn the source VNIC's interface-to-MAC address mapping.

[0231] Each receiving VNIC can also decapsulate frames and forward the decapsulated frames (e.g., the corresponding packets) to their associated CI. Each CI may include a network interface that can evaluate the forwarded packets. If the network interface determines that the CI of the received forwarded packet does not match the destination MAC and / or IP address, then the packet is discarded. If the network interface determines that the CI of the received forwarded frame matches the destination MAC and / or IP address, then the packet is received by the CI. In some embodiments, a CI having a MAC and / or IP address that matches the destination MAC and / or IP address of the packet can send a response to the source CI, whereby the source VNIC can learn the interface-to-MAC address mapping of the destination CI, and whereby the source CI can learn the IP-to-MAC address mapping of the destination CI.

[0232] Procedure 1200 can be performed when the source CI is unaware of the IP-to-MAC address mapping, or when the IP-to-MAC address mapping of the destination CI of the source CI is stale. Therefore, when the IP-to-MAC address mapping is known, the source CI can send packets. When the IP-to-MAC address mapping is unknown, procedure 1200 can be performed. When the interface-to-MAC address mapping is unknown, the interface-to-MAC address learning procedure outlined above can be performed. When the interface-to-MAC address mapping is known, the source VNIC can send the corresponding frame to the destination VNIC. Procedure 1200 begins at block 1202, where the source CI determines that the IP-to-MAC address mapping of the destination CI is unknown to the source CI. In some embodiments, this may include the source CI determining the destination IP address for the packet and determining that the destination IP address is not associated with a MAC address stored in the mapping table of the source CI. Alternatively, the source CI may determine that the IP-to-MAC address mapping for the destination CI is stale. In some embodiments, the mapping may be stale if it has not been updated and / or verified within a certain time limit. After determining that the IP-to-MAC address mapping of the destination CI is unknown and / or outdated for the source CI, the source CI initiates an ARP request for the destination IP address and sends the ARP request for Ethernet broadcast.

[0233] At box 1204, the source VNIC (also referred to herein as the source interface) receives ARP requests from the source CI. The source interface identifies all interfaces on the VLAN and sends ARP requests to all interfaces on the VLAN broadcast domain. As mentioned earlier, since the control plane knows all interfaces on the VLAN and provides this information to the interfaces connected to the VLAN, the source interface also knows all interfaces in the VLAN and is able to send ARP requests to each interface in the VLAN. To do this, the source interface copies the ARP requests and encapsulates one of the ARP requests for each interface on the VLAN. Each encapsulated ARP request includes the source CI interface identifier and the source CI MAC and IP address, the destination IP address, and the destination CI interface identifier. The source CI interface replicates Ethernet broadcast by sending the copied and encapsulated ARP requests (e.g., ARP messages) as serial unicast to each interface in the VLAN.

[0234] At box 1206, each interface in the VLAN broadcast domain receives and decapsulates an ARP message. Each interface receiving the ARP message in the VLAN broadcast domain learns the interface-to-MAC address mapping of the source VNIC for the source CI (e.g., the interface identifier to MAC address of the source interface of the source CI), because this message identifies the source CI's MAC and IP addresses as well as the source CI's interface identifier. As part of learning the interface-to-MAC address mapping for the source CI, each interface can update its mapping table (e.g., its L2 forwarding table) and can provide the updated mapping to its associated switches and / or CIs. Except for VSRS, each receiving interface can forward the decapsulated packet to its associated CI. The CI receiving the forwarded decapsulated packet, and specifically the network interface of that CI, can determine whether the destination IP address matches the CI's IP address. If the IP address of the CI associated with that interface does not match the destination CI IP address, then in some embodiments, the packet is dropped by the CI without further action. In the case of VSRS, the VSRS can determine whether the destination IP address matches the VSRS's IP address. If the IP address of the VSRS does not match the destination IP address specified in the received packet, in some embodiments the packet is discarded by the VSRS without further action.

[0235] If it is determined that the destination CI IP address specified in the received packet matches the IP address of the CI (destination CI) associated with the receiving interface, then, as indicated in box 1208, the destination CI sends a response, which may be a unicast ARP response to the source interface. This response includes the destination CI MAC address and destination CI IP address, as well as the source CI IP address and MAC address. This response is received by the destination interface encapsulating the unicast ARP response, as indicated in box 1210. In some embodiments, this encapsulation may include a GENEVE encapsulation. The destination interface may forward the encapsulated ARP response to the source interface via a destination switch. The encapsulated ARP response includes the destination CI MAC and IP addresses and the destination CI interface identifier, as well as the source CI MAC and IP addresses and the source CI interface identifier.

[0236] At box 1212, the source interface receives and decapsulates the ARP response. The source interface may further learn the interface-to-MAC address mapping of the destination CI based on the encapsulation and / or information contained in the encapsulated frame. In some embodiments, the source interface may forward the ARP response to the source CI.

[0237] At block 1214, the source CI receives an ARP response. In some embodiments, the source CI may update its mapping table based on the information contained in the ARP response, and specifically, based on the MAC and IP addresses of the destination CI to reflect the IP-to-MAC address mapping. The source CI may then send a packet to the destination CI based on this MAC address. This packet may include the source CI's MAC address and interface identifier as the source MAC address and source interface, and the destination CI's MAC address and interface identifier as the destination MAC address and destination interface.

[0238] At box 1216, the source interface can receive packets from the source CI. The source interface can encapsulate the packets, and in some embodiments, this encapsulation uses GENEVE encapsulation. The source interface can forward the corresponding frame to the destination CI, specifically to the destination interface. The encapsulated frame can include the MAC address and interface identifier of the source CI as the source MAC address and source interface identifier, and the MAC address and interface identifier of the destination CI as the destination MAC address and destination interface.

[0239] At box 1218, the destination interface receives frames from the source interface. The destination interface can decapsulate the frames and then forward the corresponding packets to the destination CI. At box 1220, the destination CI receives packets from the destination interface.

[0240] ACL

[0241] Access Control Lists (ACLs) contain information about permissions and / or restrictions associated with access to and / or use of computing resources. In a Physical Layer 2 network, customers can define ACLs at the switch level to filter frames processed by the network's switches. In such physical L2 networks, a single hardware switch is typically used, so the definition, deployment, and enforcement of ACLs are performed locally on that switch. In contrast, in a virtual L2 network (e.g., any VLANs described above), a single hardware switch is not used. Instead, while customers perceive the use of a single switch (considered a vswitch in the above context), a distributed L2 virtual switch architecture is used, comprising multiple L2 switches that are local to multiple NVDs and hosted on multiple NVDs (the actual hardware providing the hosting functionality). This distributed architecture influences how ACLs can be deployed in a virtual L2 network. This influence can span multiple dimensions. In the first dimension, to support the customer's understanding of a single switch, a translation can be performed between the customer's L2 virtual network and the customer configuration of ACLs, and the actual distribution of local L2 switches and associated L2 VNICs on NVDs. In the second dimension, to support port-specific ACLs (each ACL can be specific to one or more ports emulated by the corresponding L2 VNIC(s),) distribution can be performed to identify the relevant NVD (e.g., the NVD hosting one or more L2 VNICs) and send the ACL to it. In the third dimension, to support the enforcement of permissions and / or restrictions indicated by ACL information, the local L2 switch (rather than the entire distributed L2 switch) can provide the relevant enforcement. The first dimension can be achieved through the operation of the control plane of the virtual network hosting the virtual L2 network. The third dimension can be achieved by the NVD.

[0242] Figure 13 An example environment suitable for defining ACLs for an L2 virtual network is illustrated. In an embodiment, the environment includes a computer system 1310 that communicates with client device 1320 via one or more networks (not shown). Computer system 1310 may include a collection of hardware computing resources hosting VCN 1312. A control plane hosted by one or more of these hardware computing resources can receive and process input from client device 1320 to deploy an L2 virtual network (shown as...) within VCN 1312. Figure 13 (L2 VLAN 1314 in the middle).

[0243] In the example, input from client device 1320 can include various types of information. This information can be specified via the console or API calls and can include L2 VLAN configuration 1322 and ACL configuration 1324, as well as other client-specified configurations.

[0244] L2 VLAN configuration 1322 can indicate, for example, the number, type(s), and configuration(s) of L2 compute instances to be included in L2 VLAN 1314. Furthermore, L2 VLAN configuration 1322 can indicate the customer-specified names of ports on the customer-aware vswitch, the MAC addresses of the L2 compute instances, and the associations between ports and MAC addresses (or more generally, L2 compute instances). For example, a customer can specify that L2 VLAN 1314 needs to include two L2 compute instances, the first with MAC address M.1 and associated with a first port named P1, and the other with MAC address M.2 and associated with a second port named P2.

[0245] ACL configuration 1324 can indicate restrictions and / or permissions on, for example, traffic (including frames) entering and / or leaving VLAN 1314 within L2 VLAN 1314. For example, referring back to the examples in the two computational instances above, a customer can specify that frames received from port P2 at port P1 should be dropped. Of course, this is for illustrative purposes only, and other types of traffic flow control can be indicated in ACL configuration 1324, as further described in the following diagrams. Furthermore, ACL configuration 1324 can indicate whether to enforce ACLs on inbound and / or outbound traffic. For example, referring back to the flow restriction of frames from P2 to P1 above, ACL configuration 1324 can indicate whether frames should be dropped at the outbound port P2 or the inbound port P1.

[0246] The control plane receives various information, then deploys and manages the different resources of L2 VLAN 1314, generating relevant ACL information and distributing it to these resources. For example, L2 VLAN 1314 is configured according to L2 VLAN configuration 1322 and includes the requested L2 compute instance hosted on the host machine and the L2 VNIC-L2 virtual switch pair hosted on the NVD. To generate ACL information, the control plane translates the client-defined ACL configuration into the actual topology of L2 VLAN 1314. For example, each L2 VNIC emulates a port, and the control plane associates the L2 VNIC (e.g., its interface identifier, its IP address, and / or the IP address of the NVD hosting the L2 VNIC) with the port name and a specified MAC address. Instead of using port names, the ACL information indicates restrictions and / or permissions by identifying the associated L2 VNICs (e.g., their interface identifiers, their IP addresses, and / or the IP address of the NVD hosting the L2 VNIC). The NVD hosting the L2 VNIC to which the ACL information is applied receives this ACL information so that the NVD can enforce traffic flow.

[0247] Figure 14 The illustration shows an example ACL technique in a VLAN. Figure 14 The top of the diagram illustrates a VLAN implementation view 1410. Figure 14 The bottom diagram shows the client representation of VLAN 1420. VLAN configuration and... Figure 11 The two diagrams are identical or similar. The commonalities between them will not be repeated here. Generally, the control plane maintains the client definitions of VLANs (e.g., compute instance 1 connected to port 1, etc.), the mapping of VLANs to actual network implementations (e.g., {client 1, M.1 → IP.1, VLAN A}), and the association between client definitions and mappings (e.g., port 1 corresponds to VNIC 1). The control plane can also receive input from clients indicating information about ACLs (e.g., via API calls and / or the console).

[0248] In the example, a client can specify an ACL for the destination port (e.g., filtering frames sent to port 1), an ACL for the source port (e.g., filtering frames sent from port 2), and / or a combination of both (e.g., filtering frames sent from port 2 to port 1). Similarly, a client can specify an ACL for the destination overlay MAC address (e.g., filtering frames with M.1 as the destination MAC address), an ACL for the source overlay MAC address (e.g., filtering frames with M.2 as the source MAC address), an ACL for the destination overlay IP address (e.g., filtering frames that use the overlay IP address of computed instance 1 as the destination IP address), an ACL for the source overlay IP address (e.g., filtering frames that use the overlay IP address of computed instance 2 as the source IP address), and / or a combination of any of these (e.g., filtering frames that use M.1 as the destination MAC address and M.2 as the source MAC address). ACLs can be additionally or alternatively defined based on one or more TCP or UDP source port numbers, one or more TCP or UDP destination port numbers, a range of TCP or UDP source or destination port numbers, or a combination of these. ACLs can be defined additionally or alternatively based on Ethernet type (e.g., IPv4, IPv6, etc.). ACLs can also be defined additionally or alternatively based on the type of destination MAC address of the traffic, such as unicast traffic for a specific MAC address, MAC address ranges, multicast traffic, or broadcast traffic. ACLs can also be specified based on byte offsets within the frame header. Customers can also indicate whether to enforce ACLs on ingress and / or egress points.

[0249] Based on this input, mapping, and client-defined mapping association, the control plane generates an ACL and distributes it to the NVDs for enforcement. To illustrate, consider two examples. In the first example, the client's input indicates that frames from port 2 to port 1 will be dropped at the ingress. In this example, the control plane determines that ports 1 and 2 correspond to VNIC 1 and VNIC 2 respectively, and VNIC 1 and VNIC 2 are hosted on NVD 1 and NVD 2 respectively, and associated with M.1 and M.2 respectively. Therefore, the control plane generates an ACL and sends it to NVD 1 (but not to NVD 2) indicating that frames with M.2 as the source MAC address and M.1 as the destination MAC address will be dropped. Next, NVD 1 receives the encapsulated frame from NVD 2. At this point, NVD 1 cannot drop the encapsulated frame based on the ACL. Instead, NVD 1 processes it (e.g., decapsulates it) to extract the frame. After determining that its destination is M.1, the frame is passed to VNIC 1. VNIC 1 then applies an ACL and discards the frame after determining that the source MAC address is M.2.

[0250] In the second example, the client's input also indicates that frames from port 2 to port 1 will be dropped, but this filtering will be performed on the egress. In this example, the control plane determines that ports 1 and 2 correspond to VNIC 1 and VNIC 2, respectively, which are hosted on NVD 1 and NVD 2, and associated with M.1 and M.2, respectively. Therefore, the control plane generates an ACL and sends it to NVD 2 (but not to NVD 1), indicating that frames with M.1 as the destination MAC address and M.2 as the source MAC address will be dropped. Next, NVD 2 receives the encapsulated frame from host 2. After decapsulation, the frame is passed to VNIC 2. VNIC 2 then applies the ACL and drops the frame after determining that the destination MAC address is M.1.

[0251] Now for reference Figure 15 The diagram illustrates a flowchart of one embodiment of a process 1500 for distributing ACL information in a Layer 2 virtual network. In some embodiments, process 1500 may be executed by a control plane deployed on the underlying physical network of the Layer 2 virtual network.

[0252] Process 1500 begins at block 1502, where the control plane receives client input indicating Layer 2 VLAN configuration and ACL configuration. In some embodiments, the client input is received from the client device via API calls and / or a console.

[0253] At box 1504, the control plane generates ACL information based on client input. In some embodiments, the control plane deploys L2 VLANs according to the L2 VLAN configuration indicated in the client input. Based on this deployment, the control plane determines and tracks the topology of the L2 VLANs by determining and tracking the placement of L2 compute instances on host machines and the placement of L2 VNICs and L2 virtual switches on NVDs, where the L2 VLANs include L2 compute instances, L2 VNICs, and L2 virtual switches. For each L2 compute instance, the topology information may indicate the associated L2 VNIC (and its corresponding interface identifier), the associated MAC address (which may be indicated in the client input), the associated IP address of the L2 VNIC (which may also be indicated in the client input), and the associated IP address of the NVD hosting the L2 VNIC, as well as other attributes of the L2 VLAN topology. Additionally, the control plane can translate client-specified names of ports into topology information about the corresponding L2 VNICs emulating those ports, and can then generate ACL information by updating the client-specified ACL configuration to use the topology information instead of the client-defined one.

[0254] At box 1506, the control plane determines the subset of L2 VNICs for which ACL information applies. In some embodiments, a client may indicate ACLs for some, but not all, ports of a client-aware vswitch. Furthermore, the ACLs for these ports may differ. In this case, the control plane can determine the L2 VNICs corresponding to the indicated ports and can generate associated ACL information for each such L2 VNIC. Additionally, the control plane can maintain a mapping of specific ACL information applicable to each L2 VNIC.

[0255] At block 1508, the control plane determines the set of ACL information to be sent to the NVDs. In some embodiments, the control plane uses the mapping mentioned at block 1506 to determine a subset of L2 VNICs that conform to the ACL. Based on topology information, the control plane determines the NVDs hosting such L2 VNICs. Therefore, the control plane determines the specific ACL information that each NVD should receive. For example, if a first NVD hosts two L2 VNICs, each associated with its own specific ACL information, then this first NVD will receive a first ACL information specific to the first L2 VNIC and a second ACL information specific to the second L2 VNIC. Similarly, if a second NVD hosts an L2 VNIC associated with its own specific ACL information, then the second NVD will receive that ACL information. However, if a third NVD hosts one or more L2 VNICs not associated with any ACL information, then the third NVD will not receive any ACL information.

[0256] At box 1510, the control plane sends ACL information to the collection of NVDs. In the example, each ACL is sent to the relevant NVD during and / or after L2 VLAN deployment via push or pull mechanisms.

[0257] Now for reference Figure 16 The illustration shows a flowchart of one embodiment of a process 1600 for determining the applicability of ACL information to an L2 VNIC. In some embodiments, process 1600 may be performed by a control plane deployed on the underlying physical network by the management layer 2 virtual network.

[0258] Process 1600 begins at block 1602, where the control plane stores interface identifiers that associate L2 compute instances with L2 VNICs. In some embodiments, the interface identifiers may uniquely identify the L2 VNIC and may be included in the header information of frames sent from or to the L2 compute instance.

[0259] At box 1604, the control plane receives client input indicating the name of the L2 port. In some embodiments, client input can be received from the client device via API calls and / or a console. The L2 VNIC can emulate an L2 port, but the client may not be aware of this L2 VNIC.

[0260] At box 1606, the control plane determines the correspondence between L2 ports and L2 VNICs. In some embodiments, the control plane determines this correspondence based on topology information maintained for L2 VLANs that include L2 compute instances and L2 VNICs.

[0261] At box 1608, the control plane stores the association between the L2 VNIC and the ACL information applicable to the L2 port. In some embodiments, client input may indicate the ACL configuration applicable to the L2 port. The control plane remaps the ACL configuration to be applicable to the L2 VNIC based on the mapping between the L2 port and the L2 VNIC to generate ACL information. Additionally, the control plane may store an indication that the ACL information is applicable to the L2 VNIC. This indication may include the interface identifier, the MAC address of the L2 VNIC, and / or the IP address of the L2 VNIC. Furthermore, this indication may be stored in the ACL information itself or in a data structure that also references the ACL information.

[0262] At box 1610, the control plane generates ACL information for the NVD hosting the L2 VNIC, indicating that the ACL information is applicable to the L2 VNIC. In some embodiments, the control plane also stores this indication in ACL information or a data structure, for example, by including the IP address of the NVD.

[0263] Now for reference Figure 17 The illustration shows a flowchart of one embodiment of a process 1700 for enforcing an ACL. In some embodiments, process 1700 may be performed by the NVD of an L2 VNIC to which the managed ACL may be applied.

[0264] Process 1700 begins at block 1702, where the NVD hosts an L2 VNIC and an L2 virtual switch belonging to the customer's L2 virtual network (e.g., a VLAN). In some embodiments, the L2 VNIC and L2 virtual switch are associated with an L2 compute instance of the L2 virtual network. This L2 compute instance may be hosted on a host machine communicatively coupled to the NVD.

[0265] At block 1704, the NVD receives and stores ACL information associated with the L2 VNIC. In some embodiments, the NVD receives the ACL information from the control plane. The ACL information can control the flow of frames from the L2 VNIC (corresponding to the exit enforcement at the NVD) and / or to the L2 VNIC (corresponding to the ingress enforcement at the NVD).

[0266] At block 1706, NVD receives a frame with header information. In some embodiments, the frame may be received from an L2 compute instance via a host machine (in which case, the frame is an egress frame sent from the L2 compute instance). In some embodiments, the frame may be received from another L2 compute instance via another host machine (in which case, the frame is an ingress frame sent to the L2 compute instance).

[0267] At box 1708, the NVD determines that the frame is associated with an L2 VNIC based on header information. In various embodiments, the header information includes an interface identifier and / or MAC address associated with the L2 VNIC. The NVD can use the interface identifier and / or MAC address to determine that the frame will be processed by the L2 VNIC.

[0268] At box 1710, the NVD determines that the ACL information is applicable. In some embodiments, the ACL information indicates restrictions and / or permissions based on MAC addresses. In this case, the NVD determines that the ACL information controls the flow of frames based on this MAC address. For example, at the egress, if the source MAC address corresponds to the L2 VNIC and another restriction criterion (e.g., if the frame has a specific destination MAC address), then the ACL information may indicate that the frame cannot be sent. At the ingress, if the destination MAC address corresponds to the L2 VNIC and another restriction criterion (e.g., if the frame has a specific source MAC address), then the ACL information may indicate that the frame cannot be sent to the L2 compute instance. In some embodiments, the ACL information may not be bound to a MAC address. Instead, the ACL information may relate to IP addresses, communication protocols, etc. (e.g., the ACL information may restrict the use of ingress frames with a specific encapsulation). In this case, instead of using the L2 VNIC's interface identifier to determine the ACL information, the NVD can determine how the ACL information controls the frame flow based on one or more relevant parts of the header information (e.g., fields regarding the encapsulation type).

[0269] At box 1712, NVD controls frame flow based on ACL information. In some embodiments, the ACL information may indicate that a frame will be dropped. In this case, the frame is not forwarded. In some embodiments, the ACL information may indicate that a frame should be forwarded. In this case, the frame is forwarded. For example, at the egress, the L2 virtual switch associated with the L2 compute instance sends the frame to the next associated L2 VNIC. At the ingress, the L2 VNIC associated with the L2 compute instance receives the frame and forwards it to the L2 compute instance. In some embodiments, the ACL information may indicate that the frame should be processed in a specific way before being forwarded (e.g., to be encapsulated with a specific encapsulation type, or to have specific header or payload information edited). In this case, the frame is processed first (e.g., by the L2 VNIC) before being forwarded.

[0270] Now for reference Figure 18 The illustration shows a flowchart of one embodiment of a process 1800 for enforcing an ACL. In some embodiments, process 1800 may be performed by the NVD of an L2 VNIC to which the managed ACL may be applied. Compared to process 1700, process 1800 may include adding ACL information (or a portion thereof) to a frame, such that the enforcement of the ACL information can be performed at various nodes between the source compute instance and the destination compute instance.

[0271] Process 1800 begins at block 1802, where the NVD receives a frame with header information. In some embodiments, the frame may be received from an L2 compute instance via a host machine (in which case, the frame is an egress frame sent from the L2 compute instance). In some embodiments, the frame may be received from another L2 compute instance via another host machine (in which case, the frame is an ingress frame sent to the L2 compute instance).

[0272] At box 1804, NVD determines whether a frame can be sent. This determination can be similar to the operation described in conjunction with boxes 1708-1712 above. If the frame cannot be sent (e.g., to an L2 compute instance on the ingress), then box 1806 follows box 1806, where the frame is discarded. Otherwise, box 1808 follows box 1804.

[0273] At box 1808, the NVD determines whether the ACL information applies to the next(s) network hops(s) to the frame's destination. In some embodiments, a frame can be sent from an L2 compute instance to a destination compute instance. This destination compute instance can be in the same VLAN as the L2 compute instance, in another VLAN, in an L2 network within the same VCN of the VLAN, in another VCN, or in some other network. The ACL information can restrict the destination to any of these network boundaries (e.g., it can restrict the frame from being sent outside the VLAN of the L2 compute instance). In this case, the NVD can determine that the ACL information applies to the next(s) network hop(s), and box 1810 follows box 1808. Otherwise, box 1812 follows box 1808.

[0274] At box 1810, the NVD adds the relevant portion of the ACL information to the frame. For example, the NVD identifies the network boundary where the destination may be located. This network boundary can be a network identifier, such as a VLAN tag for VLANs, a network name, etc. In this way, when the frame arrives at the next network hop (e.g., another NVD or compute resource), the frame header can be checked to determine whether the frame can still be sent forward, thereby enforcing the ACL information as indicated in the header information at this next network hop.

[0275] At box 1812, the NVD sends a frame. In some embodiments, after adding the relevant portion of the ACL information, the L2 virtual switch associated with the L2 compute instance, if applicable, sends the frame to the relevant next L2 VNIC.

[0276] C- Example Infrastructure as a Service architecture

[0277] As noted above, Infrastructure as a Service (IaaS) is a specific type of cloud computing. IaaS can be configured to provide virtualized computing resources over a public network (e.g., the Internet). In the IaaS model, cloud providers can host infrastructure components (e.g., servers, storage devices, network nodes (e.g., hardware), deployment software, platform virtualization (e.g., hypervisor layer), etc.). In some cases, IaaS providers can also offer various services accompanying these infrastructure components (e.g., billing, monitoring, logging, security, load balancing, and clustering, etc.). Therefore, because these services may be policy-driven, IaaS users can implement policies to drive load balancing to maintain application availability and performance.

[0278] In some cases, IaaS customers can access resources and services over a wide area network (WAN) such as the Internet and can use the cloud provider's services to install the remaining elements of the application stack. For example, a user can log in to the IaaS platform to create virtual machines (VMs), install an operating system (OS) on each VM, deploy middleware such as databases, create buckets for workloads and backups, and even install enterprise software into that VM. The customer can then use the provider's services to perform various functions, including balancing network traffic, troubleshooting application issues, monitoring performance, and managing disaster recovery.

[0279] In most cases, cloud computing models will require the involvement of cloud providers. Cloud providers can, but are not necessarily, third-party providers specializing in (e.g., provisioning, renting, selling) IaaS services. Entities may also choose to deploy private clouds, thus becoming their own infrastructure service providers.

[0280] In some examples, IaaS deployment is the process of placing a new application or a new version of an application onto a prepared application server, etc. It may also include the processing of server preparation (e.g., installation libraries, daemons, etc.). This is typically managed by the cloud provider, below the hypervisor layer (e.g., servers, storage devices, network hardware, and virtualization). Therefore, the customer can be responsible for processing (OS), middleware, and / or application deployment (e.g., on self-service virtual machines, etc., which can be started on demand).

[0281] In some examples, IaaS provisioning can refer to acquiring computers or virtual hosts for use, or even installing necessary libraries or services on them. In most cases, deployment does not include provisioning, and provisioning may need to be performed first.

[0282] In some cases, IaaS provisioning presents two distinct challenges. First, there's the initial challenge of provisioning the initial infrastructure set before anything is operational. Second, once everything is provisioned, there's the challenge of evolving the existing infrastructure (e.g., adding new services, changing services, removing services, etc.). In some cases, both challenges can be addressed by enabling configuration that declaratively defines the infrastructure. In other words, the infrastructure (e.g., which components are needed and how they interact) can be defined by one or more configuration files. Therefore, the overall topology of the infrastructure (e.g., which resources depend on which resources and how they work together) can be described declaratively. In some cases, once the topology is defined, workflows for creating and / or managing the different components described in the configuration files can be generated.

[0283] In some examples, the infrastructure can have many interconnected components. For example, there may be one or more Virtual Private Clouds (VPCs) (e.g., potential on-demand pools of configurable and / or shared computing resources), also known as the core network. In some examples, one or more security group rules may also be provided to define how the network's security is configured, as well as one or more virtual machines (VMs). Other infrastructure elements, such as load balancers, databases, etc., may also be provided. The infrastructure can evolve incrementally as more and / or more infrastructure elements are expected and added.

[0284] In some cases, continuous deployment techniques can be used to enable the deployment of infrastructure code across various virtual computing environments. Furthermore, the described techniques enable infrastructure management within these environments. In some examples, service teams may write code that they expect to deploy to one or more, but often many, different production environments (e.g., across various geographical locations, sometimes spanning the entire world). However, in some examples, the infrastructure on which the code will be deployed must first be set up. In some cases, provisioning can be done manually, resources can be provisioned using provisioning tools, and / or once the infrastructure is provisioned, the code can be deployed using deployment tools.

[0285] Figure 19 This is a block diagram 1900 illustrating an example pattern of an IaaS architecture according to at least one embodiment. Service provider 1902 may communicatively couple to secure host lease 1904, which may include a virtual cloud network (VCN) 1906 and a secure host subnet 1908. In some examples, service provider 1902 may use one or more client computing devices, which may be portable handheld devices (e.g., Cellular phone Computing tablets, personal digital assistants (PDAs), or wearable devices (e.g., Google) Head-mounted displays), running software (such as Microsoft Windows) ) and / or various mobile operating systems (such as iOS, Windows Phone, Android, BlackBerry 8, Palm OS, etc.), and support the Internet, email, and short message service (SMS). Or other communication protocols. Alternatively, the client computing device can be a general-purpose personal computer, including, for example, those running various versions of Microsoft... Apple Personal computers and / or laptops running Linux operating systems. Client computing devices can be running various commercially available operating systems. Workstation computers operating systems, including but not limited to any of the various GNU / Linux operating systems (such as, for example, Google Chrome OS), or UNIX-like operating systems. Alternatively or additionally, the client computing device can be any other electronic device, such as a thin client computer, an internet-enabled gaming system (e.g., with or without...). A Microsoft Xbox game console with gesture input devices, and / or a personal messaging device capable of communicating via a network that can access VCN 1906 and / or the Internet.

[0286] VCN 1906 may include a local peering gateway (LPG) 1910, which may be communicatively coupled to a secure shell (SSH) VCN 1912 via LPG 1910 contained in SSH VCN 1912. SSH VCN 1912 may include an SSH subnet 1914, and SSH VCN 1912 may be communicatively coupled to a control plane VCN 1916 via LPG 1910 contained in control plane VCN 1916. Furthermore, SSH VCN 1912 may be communicatively coupled to a data plane VCN 1918 via LPG 1910. Control plane VCN 1916 and data plane VCN 1918 may be contained within a service lease 1919 that may be owned and / or operated by an IaaS provider.

[0287] The control plane VCN 1916 may include a control plane demilitarized zone (DMZ) layer 1920 that acts as a peripheral network (e.g., part of a corporate network between a corporate intranet and an external network). DMZ-based servers can assume limited liability and help control security vulnerabilities. Furthermore, the DMZ layer 1920 may include one or more load balancer (LB) subnets 1922, a control plane application layer 1924 that may include one or more application subnets 1926, and a control plane data layer 1928 that may include one or more database (DB) subnets 1930 (e.g., one or more front-end DB subnets and / or one or more back-end DB subnets). One or more LB subnets 1922 contained in the control plane DMZ layer 1920 may be communicatively coupled to one or more application subnets 1926 contained in the control plane application layer 1924 and an Internet gateway 1934 that may be contained in the control plane VCN 1916. The application subnets 1926 may be communicatively coupled to one or more DB subnets 1930 contained in the control plane data layer 1928, as well as a service gateway 1936 and a Network Address Translation (NAT) gateway 1938. The control plane VCN 1916 may include the service gateway 1936 and the NAT gateway 1938.

[0288] The control plane VCN 1916 may include a data plane mirror application layer 1940, which may include one or more application subnets 1926. The one or more application subnets 1926 included in the data plane mirror application layer 1940 may include a Virtual Network Interface Controller (VNIC) 1942 capable of executing a compute instance 1944. The compute instance 1944 may communicatively couple the one or more application subnets 1926 of the data plane mirror application layer 1940 to the one or more application subnets 1926 that may be included in the data plane application layer 1946.

[0289] Data plane VCN 1918 may include data plane application layer 1946, data plane DMZ layer 1948, and data plane data layer 1950. Data plane DMZ layer 1948 may include one or more LB subnets 1922 communicatively coupled to one or more application subnets 1926 of data plane application layer 1946 and Internet gateway 1934 of data plane VCN 1918. One or more application subnets 1926 may be communicatively coupled to service gateway 1936 and NAT gateway 1938 of data plane VCN 1918. Data plane data layer 1950 may also include one or more DB subnets 1930 communicatively coupled to one or more application subnets 1926 of data plane application layer 1946.

[0290] The Internet gateway 1934 of the control plane VCN 1916 and data plane VCN 1918 can communicatively couple to the metadata management service 1952, which in turn can communicatively couple to the public Internet 1954. The public Internet 1954 can communicatively couple to the NAT gateway 1938 of the control plane VCN 1916 and data plane VCN 1918. The service gateway 1936 of the control plane VCN 1916 and data plane VCN 1918 can communicatively couple to the cloud service 1956.

[0291] In some examples, the service gateway 1936 of the control plane VCN 1916 or data plane VCN 1918 can make application programming interface (API) calls to the cloud service 1956 without traversing the public internet 1954. API calls from the service gateway 1936 to the cloud service 1956 can be unidirectional: the service gateway 1936 can make API calls to the cloud service 1956, and the cloud service 1956 can send requested data to the service gateway 1936. However, the cloud service 1956 may not initiate API calls to the service gateway 1936.

[0292] In some examples, secure host lease 1904 can be directly connected to service lease 1919, which would otherwise be isolated. Secure host subnet 1908 can communicate with SSH subnet 1914 via LPG 1910, which enables bidirectional communication between otherwise isolated systems. Connecting secure host subnet 1908 to SSH subnet 1914 allows secure host subnet 1908 to access other entities within service lease 1919.

[0293] Control plane VCN 1916 allows users of service lease 1919 to configure or otherwise provision desired resources. Desired resources provisioned in control plane VCN 1916 can be deployed or otherwise used in data plane VCN 1918. In some examples, control plane VCN 1916 can be isolated from data plane VCN 1918, and the data plane mirror application layer 1940 of control plane VCN 1916 can communicate with the data plane application layer 1946 of data plane VCN 1918 via VNIC 1942, which can be included in both the data plane mirror application layer 1940 and the data plane application layer 1946.

[0294] In some examples, users or clients of the system can make requests, such as create, read, update, or delete (CRUD) operations, via the public internet 1954, which can transmit requests to the metadata management service 1952. The metadata management service 1952 can transmit requests to the control plane VCN 1916 via internet gateway 1934. Requests can be received by one or more LB subnets 1922 contained in the control plane DMZ layer 1920. The LB subnets 1922 can determine that the request is valid, and in response to this determination, they can transmit the request to one or more application subnets 1926 contained in the control plane application layer 1924. If the request is validated and requires a call to the public internet 1954, the call to the public internet 1954 can be transmitted to a NAT gateway 1938 that can make calls to the public internet 1954. The request may expect the storage to be located in one or more DB subnets 1930.

[0295] In some examples, the data plane mirroring application layer 1940 can facilitate direct communication between the control plane VCN 1916 and the data plane VCN 1918. For example, it might be desirable to apply configuration changes, updates, or other appropriate modifications to resources contained in the data plane VCN 1918. Through VNIC 1942, the control plane VCN 1916 can communicate directly with the resources contained in the data plane VCN 1918, and thus can perform configuration changes, updates, or other appropriate modifications.

[0296] In some embodiments, the control plane VCN 1916 and data plane VCN 1918 may be included in service lease 1919. In this case, the system's users or clients may not own or operate the control plane VCN 1916 or data plane VCN 1918. Alternatively, the IaaS provider may own or operate both the control plane VCN 1916 and data plane VCN 1918, both of which may be included in service lease 1919. This embodiment can enable the isolation of networks that might prevent users or clients from interacting with resources of other users or clients. Furthermore, this embodiment can allow users or clients of the system to privately store databases without relying on the public Internet 1954, which may not have the desired level of security for storage.

[0297] In other embodiments, one or more LB subnets 1922 included in the control plane VCN 1916 may be configured to receive signals from the service gateway 1936. In this embodiment, the control plane VCN 1916 and the data plane VCN 1918 may be configured to be invoked by the IaaS provider's customers without invoking the public Internet 1954. The IaaS provider's customers may expect this embodiment because the database(s) used by the customer can be controlled by the IaaS provider and can be stored on service lease 1919, which may be isolated from the public Internet 1954.

[0298] Figure 20 This is a block diagram 2000 illustrating another example pattern of an IaaS architecture according to at least one embodiment. Service operator 2002 (e.g., Figure 19 Service providers (1902) can communicatively couple to secure host rental (2004) (e.g., Figure 19 Secure Host Rental 2004), which may include Virtual Cloud Network (VCN) 2006 (e.g., Figure 19 VCN 1906) and Secure Host Subnet 2008 (e.g., Figure 19 The secure host subnet 1908). VCN 2006 can include Local Peer Gateway (LPG) 2010 (e.g., Figure 19 The LPG 1910, which can be communicatively coupled to the Secure Shell (SSH) VCN 2012 (e.g., via the LPG 1910 included in SSH VCN 2012) Figure 19 SSH VCN 1912). SSH VCN 2012 can include SSH subnets 2014 (e.g., Figure 19 SSH subnet 1914), and SSH VCN 2012 can be communicatively coupled to control plane VCN 2016 via LPG 2010 included in control plane VCN 2016 (e.g., Figure 19 Control plane VCN 2016. Control plane VCN 2016 may be included in service lease 2019 (e.g., Figure 19 In the service lease 1919), and the data plane VCN 2018 (e.g., Figure 19 The data plane (VCN 1918) can be included in customer leases that may be owned or operated by users or customers of the system in 2021.

[0299] Control plane VCN 2016 may include control plane DMZ layer 2020 (e.g., Figure 19 The control plane DMZ layer 1920 may include one or more LB subnets 2022 (e.g., Figure 19 (one or more) LB subnets 1922), may include (one or more) application subnets 2026 (e.g., Figure 19 The control plane application layer 2024 of (one or more) application subnets 1926 (e.g., Figure 19 The control plane application layer 1924) may include one or more database (DB) subnets 2030 (e.g., similar to...). Figure 19 The control plane data layer of (one or more) DB subnets 1930) 2028 (e.g., Figure 19 The control plane data layer 1928). One or more LB subnets 2022 contained in the control plane DMZ layer 2020 can be communicatively coupled to one or more application subnets 2026 contained in the control plane application layer 2024 and an Internet gateway 2034 that can be contained in the control plane VCN 2016 (e.g., Figure 19 Internet gateway 1934), and application subnet(s) 2026 can communicatively couple to DB subnet(s) 2030 contained in control plane data layer 2028 and service gateway 2036 (e.g., Figure 19 The service gateway) and Network Address Translation (NAT) gateway 2038 (e.g., Figure 19 (NAT gateway 1938). The control plane VCN 2016 may include service gateway 2036 and NAT gateway 2038.

[0300] The control plane VCN 2016 may include a data plane mirror of the application layer 2040 (e.g., Figure 19 The data plane mirror application layer 2040 may include one or more application subnets 2026. The application subnets 2026 included in the data plane mirror application layer 2040 may include instances 2044 capable of performing computations (e.g., similar to...). Figure 19 The virtual network interface controller (VNIC) 2042 (e.g., the VNIC of 1942) of the computing instance 1944. The computing instance 2044 may facilitate the mirroring of the data plane application layer 2040 with one or more application subnets 2026 and may be included in the data plane application layer 2046 (e.g., Figure 19 Communication between one or more application subnets 2026 in the data plane application layer 1946 via VNIC 2042 contained in the data plane mirror application layer 2040 and VNIC 2042 contained in the data plane application layer 2046.

[0301] The Internet gateway 2034 included in the control plane VCN 2016 can be communicatively coupled to the metadata management service 2052 (e.g., Figure 19 Metadata management service 2052), which can communicatively couple to the public Internet 2054 (e.g., Figure 19 The public internet 2054 can communicatively couple to a NAT gateway 2038 contained in the control plane VCN 2016. The service gateway 2036 contained in the control plane VCN 2016 can communicatively couple to a cloud service 2056 (e.g., ...). Figure 19 Cloud services (1956).

[0302] In some examples, the data plane VCN 2018 may be included in customer lease 2021. In this case, the IaaS provider may provide a control plane VCN 2016 for each customer, and the IaaS provider may set up a unique compute instance 2044 for each customer included in service lease 2019. Each compute instance 2044 may allow communication between the control plane VCN 2016 included in service lease 2019 and the data plane VCN 2018 included in customer lease 2021. The compute instance 2044 may allow resources provisioned in the control plane VCN 2016 included in service lease 2019 to be deployed or otherwise used in the data plane VCN 2018 included in customer lease 2021.

[0303] In other examples, an IaaS provider's customer may have a database residing in customer lease 2021. In this example, control plane VCN 2016 may include data plane mirror application layer 2040, which may include one or more application subnets 2026. Data plane mirror application layer 2040 may reside in data plane VCN 2018, but may not reside in data plane VCN 2018. That is, data plane mirror application layer 2040 may have access to customer lease 2021, but may not reside in data plane VCN 2018 or be owned or operated by an IaaS provider's customer. Data plane mirror application layer 2040 may be configured to invoke data plane VCN 2018, but may not be configured to invoke any entity contained in control plane VCN 2016. Customers may expect to deploy or otherwise use resources provisioned in the control plane VCN 2016 in the data plane VCN 2018, and the data plane mirror application layer 2040 can facilitate customers' desired deployments or other uses of resources.

[0304] In some embodiments, IaaS provider customers can apply filters to data plane VCN 2018. In this embodiment, customers can determine what data plane VCN 2018 can access, and customers can restrict access from data plane VCN 2018 to the public Internet 2054. The IaaS provider may not be able to apply filters or otherwise control data plane VCN 2018's access to any external networks or databases. Applying filters and controls to data plane VCN 2018 included in customer lease 2021 helps isolate data plane VCN 2018 from other customers and the public Internet 2054.

[0305] In some embodiments, cloud service 2056 may be invoked by service gateway 2036 to access services that may not exist on public internet 2054, control plane VCN 2016, or data plane VCN 2018. The connection between cloud service 2056 and control plane VCN 2016 or data plane VCN 2018 may not be real-time or continuous. Cloud service 2056 may reside on different networks owned or operated by an IaaS provider. Cloud service 2056 may be configured to receive calls from service gateway 2036 and may be configured not to receive calls from public internet 2054. Some cloud services 2056 may be isolated from other cloud services 2056, and control plane VCN 2016 may be isolated from cloud services 2056 that may not be in the same region as control plane VCN 2016. For example, control plane VCN 2016 may be located in "Region 1," and cloud service "Deployment 19" may be located in both "Region 1" and "Region 2." If the service gateway 2036, contained in the control plane VCN 2016 located in region 1, makes a call to deployment 19, then that call can be transmitted to deployment 19 in region 1. In this example, the control plane VCN 2016 or deployment 19 in region 1 may not be communicatively coupled to or otherwise communicate with deployment 19 in region 2.

[0306] Figure 21 This is a block diagram 2100 illustrating another example pattern of an IaaS architecture according to at least one embodiment. Service operator 2102 (e.g., Figure 19 The service provider 1902) can communicatively couple to the secure host lease 2104 (e.g., Figure 19 Secure hosting lease 1904), the secure hosting lease 2104 may include a virtual cloud network (VCN) 2106 (e.g., Figure 19 VCN 1906) and Secure Host Subnet 2108 (e.g., Figure 19 The secure host subnet 1908). VCN 2106 may include LPG 2110 (e.g., Figure 19 The LPG 1910), which can be communicatively coupled to the SSH VCN 2112 via the LPG 2110 included in the SSH VCN 2112 (e.g., Figure 19 SSH VCN 2112 can include SSH subnet 2114 (e.g., SSH VCN 2112). Figure 19 SSH subnet 1914), and SSH VCN 2112 can be communicatively coupled to control plane VCN 2116 via LPG 2110 included in control plane VCN 2116 (e.g., Figure 19 The control plane VCN 1916) and coupled to the data plane VCN 2118 via the LPG 2110 contained in the data plane VCN 2118 (e.g., Figure 19 Data plane 2116 and data plane 2118 can be contained in service lease 2119 (e.g., data plane 2118). Figure 19 In the service rental of 1919.

[0307] The control plane VCN 2116 may include a load balancer (LB) subnet 2122 (e.g., Figure 19 The control plane DMZ layer 2120 of (one or more) LB subnets 1922) (e.g., Figure 19 The control plane DMZ layer 1920 may include one or more application subnets 2126 (e.g., similar to...). Figure 19 The control plane application layer 2124 of (one or more) application subnets 1926 (e.g., Figure 19 The control plane application layer 1924) may include a control plane data layer 2128 (e.g., DB subnet 2130) that may include one or more DB subnets 2130. Figure 19 The control plane data layer 1928). One or more LB subnets 2122 contained in the control plane DMZ layer 2120 can be communicatively coupled to one or more application subnets 2126 contained in the control plane application layer 2124 and an Internet gateway 2134 that can be contained in the control plane VCN 2116 (e.g., Figure 19 Internet gateway 1934), and application subnet(s) 2126 can communicatively couple to DB subnet(s) 2130 contained in control plane data layer 2128 and service gateway 2136 (e.g., Figure 19 The service gateway) and Network Address Translation (NAT) gateway 2138 (e.g., Figure 19 (NAT gateway 1938). Control plane VCN 2116 may include service gateway 2136 and NAT gateway 2138.

[0308] Data plane VCN 2118 may include data plane application layer 2146 (e.g., Figure 19 Data plane application layer 1946), data plane DMZ layer 2148 (e.g., Figure 19 Data plane DMZ layer 1948), and data plane data layer 2150 (e.g., Figure 19 The data plane data layer 1950). The data plane DMZ layer 2148 may include one or more trusted application subnets 2160 and one or more untrusted application subnets 2162 that can be communicatively coupled to the data plane application layer 2146, and one or more LB subnets 2122 of the Internet gateway 2134 contained in the data plane VCN 2118. The one or more trusted application subnets 2160 may be communicatively coupled to the service gateway 2136 contained in the data plane VCN 2118, the NAT gateway 2138 contained in the data plane VCN 2118, and one or more DB subnets 2130 contained in the data plane data layer 2150. The one or more untrusted application subnets 2162 may be communicatively coupled to the service gateway 2136 contained in the data plane VCN 2118 and the one or more DB subnets 2130 contained in the data plane data layer 2150. The data plane data layer 2150 may include one or more DB subnets 2130 that can be communicatively coupled to the service gateway 2136 contained in the data plane VCN 2118.

[0309] One or more untrusted application subnets 2162 may include one or more primary VNICs 2164(1)-(N) that can be communicatively coupled to tenant virtual machines (VMs) 2166(1)-(N). Each tenant VM 2166(1)-(N) may be communicatively coupled to a corresponding application subnet 2167(1)-(N) that may be contained in a corresponding container egress VCN 2168(1)-(N), which may be contained in a corresponding customer lease 2170(1)-(N). A corresponding secondary VNIC 2172(1)-(N) may facilitate communication between one or more untrusted application subnets 2162 contained in data plane VCN 2118 and application subnets contained in container egress VCN 2168(1)-(N). Each container exit VCN 2168(1)-(N) may include a NAT gateway 2138, which can communicatively couple to the public Internet 2154 (e.g., Figure 19 The public internet (1954).

[0310] The Internet gateway 2134, contained in the control plane VCN 2116 and the data plane VCN 2118, can be communicatively coupled to the metadata management service 2152 (e.g., Figure 19 A metadata management system 2152 (1952) is communicatively coupled to a public internet 2154. The public internet 2154 is communicatively coupled to a NAT gateway 2138 contained in a control plane VCN 2116 and a data plane VCN 2118. A service gateway 2136 contained in both the control plane VCN 2116 and the data plane VCN 2118 is communicatively coupled to a cloud service 2156.

[0311] In some embodiments, the data plane VCN 2118 may be integrated with the customer lease 2170. Such integration may be useful or desired by the IaaS provider's customers in certain situations, such as when support may be expected during code execution. Customers may provide code that could be destructive, might communicate with other customer resources, or might otherwise cause undesirable effects. In response, the IaaS provider may determine whether to run the code provided by the customer to the IaaS provider.

[0312] In some examples, an IaaS provider's customer can grant temporary network access to the IaaS provider and request functionality attached to data plane layer application 2146. The code running this functionality can execute within VMs 2166(1)-(N), and this code may not be configured to run anywhere else on data plane VCN 2118. Each VM 2166(1)-(N) can be connected to a customer lease 2170. A corresponding container 2171(1)-(N) contained within VMs 2166(1)-(N) can be configured to run the code. In this scenario, dual isolation can exist (e.g., container 2171(1)-(N) runs the code, where container 2171(1)-(N) may be contained within at least one VM 2166(1)-(N) contained in untrusted application subnet 2162), which can help prevent incorrect or otherwise unintended code from corrupting the IaaS provider's network or the networks of different customers. Containers 2171(1)-(N) may be communicatively coupled to customer lease 2170 and may be configured to transmit or receive data from customer lease 2170. Containers 2171(1)-(N) may not be configured to transmit or receive data from any other entity in the data plane VCN 2118. After the code execution is complete, the IaaS provider may terminate or otherwise dispose of containers 2171(1)-(N).

[0313] In some embodiments, one or more trusted application subnets 2160 may run code that can be owned or operated by an IaaS provider. In this embodiment, one or more trusted application subnets 2160 may be communicatively coupled to one or more database subnets 2130 and configured to perform CRUD operations in one or more database subnets 2130. One or more untrusted application subnets 2162 may be communicatively coupled to one or more database subnets 2130, but in this embodiment, one or more untrusted application subnets may be configured to perform read operations in one or more database subnets 2130. Containers 2171(1)-(N) that may be contained in each customer's VM 2166(1)-(N) and may run code from the customer may not be communicatively coupled to one or more database subnets 2130.

[0314] In other embodiments, the control plane VCN 2116 and the data plane VCN 2118 may be coupled without direct communication. In this embodiment, there may be no direct communication between the control plane VCN 2116 and the data plane VCN 2118. However, communication can occur indirectly through at least one method. The LPG 2110 may be established by an IaaS provider, which can facilitate communication between the control plane VCN 2116 and the data plane VCN 2118. In another example, either the control plane VCN 2116 or the data plane VCN 2118 may invoke the cloud service 2156 via the service gateway 2136. For example, an invocation of the cloud service 2156 from the control plane VCN 2116 may include a request for a service that can communicate with the data plane VCN 2118.

[0315] Figure 22 This is a block diagram 2200 illustrating another example pattern of an IaaS architecture according to at least one embodiment. Service operator 2202 (e.g., Figure 19 The service provider 1902) can communicatively couple to the secure host lease 2204 (e.g., Figure 19 The secure hosting lease 2204 may include a virtual cloud network (VCN) 2206 (e.g., Figure 19 VCN 1906) and Secure Host Subnet 2208 (e.g., Figure 19 The secure host subnet 1908). VCN 2206 can include LPG 2210 (e.g., Figure 19 The LPG 1910), the LPG 2210 can be contained in SSH VCN 2212 (e.g., Figure 19The LPG 2210 in SSH VCN 2212 is communicatively coupled to SSH VCN 2212. SSH VCN 2212 may include SSH subnet 2214 (e.g., Figure 19 The SSH subnet 1914), and the SSH VCN 2212 can be communicatively coupled to the control plane VCN 2216 via the LPG 2210 included in the control plane VCN 2216 (e.g., Figure 19 The control plane VCN 1916) and coupled to the data plane VCN 2218 via the LPG 2210 contained in the data plane VCN 2218 (e.g., Figure 19 Data plane 2218). Control plane VCN 2216 and data plane VCN 2218 may be contained in service lease 2219 (e.g., Figure 19 In the service rental of 1919.

[0316] The control plane VCN 2216 may include one or more LB subnets 2222 (e.g., Figure 19 The control plane DMZ layer 2220 of (one or more) LB subnets 1922) (e.g., Figure 19 The control plane DMZ layer 1920 may include (one or more) application subnets 2226 (e.g., Figure 19 The control plane application layer 2224 of (one or more) application subnets 1926 (e.g., Figure 19 The control plane application layer 1924) may include (one or more) DB subnets 2230 (e.g., Figure 21 The control plane data layer 2228 of (one or more) DB subnets 2130 (e.g., Figure 19 The control plane data layer 1928). One or more LB subnets 2222 contained in the control plane DMZ layer 2220 can be communicatively coupled to one or more application subnets 2226 contained in the control plane application layer 2224 and an Internet gateway 2234 that can be contained in the control plane VCN 2216 (e.g., Figure 19 Internet gateway 1934), and application subnet(s) 2226 can communicatively couple to DB subnet(s) 2230 contained in control plane data layer 2228 and service gateway 2236 (e.g., Figure 19 The service gateway) and Network Address Translation (NAT) gateway 2238 (e.g., Figure 19 (NAT gateway 1938). The control plane VCN 2216 may include the service gateway 2236 and the NAT gateway 2238.

[0317] The data plane VCN 2218 may include the data plane application layer 2246 (e.g., Figure 19 Data plane application layer 1946), data plane DMZ layer 2248 (e.g., Figure 19 Data plane DMZ layer 1948), and data plane data layer 2250 (e.g., Figure 19 The data plane data layer 1950). The data plane DMZ layer 2248 may include one or more trusted application subnets 2260 that can be communicatively coupled to the data plane application layer 2246 (e.g., Figure 21 (one or more) trusted application subnets 2160) and (one or more) untrusted application subnets 2262 (e.g., Figure 21 The data plane VCN 2218 may include one or more untrusted application subnets 2162 and one or more LB subnets 2222. One or more trusted application subnets 2260 may be communicatively coupled to a service gateway 2236, a NAT gateway 2238, and a DB subnet 2230, all contained in the data plane VCN 2218. One or more untrusted application subnets 2262 may be communicatively coupled to a service gateway 2236 and a DB subnet 2230, all contained in the data plane VCN 2218 and the data plane data layer 2250. The data plane data layer 2250 may include one or more DB subnets 2230 that may be communicatively coupled to a service gateway 2236, all contained in the data plane VCN 2218.

[0318] One or more untrusted application subnets 2262 may include a primary VNIC 2264(1)-(N) communicatively coupled to tenant virtual machines (VMs) 2266(1)-(N) residing within one or more untrusted application subnets 2262. Each tenant VM 2266(1)-(N) may run code in a corresponding container 2267(1)-(N) and is communicatively coupled to an application subnet 2226 that may be contained in a data plane application layer 2246 contained in a container egress VCN 2268. A corresponding secondary VNIC 2272(1)-(N) may facilitate communication between one or more untrusted application subnets 2262 contained in a data plane VCN 2218 and the application subnets contained in a container egress VCN 2268. The container egress VCN may include a public internet 2254 (e.g., Figure 19 NAT gateway 2238 for the public Internet (1954).

[0319] Internet gateway 2234, contained in control plane VCN 2216 and data plane VCN 2218, can be communicatively coupled to metadata management service 2252 (e.g., Figure 19 A metadata management system 2252 (1952) is communicatively coupled to a public internet 2254. The public internet 2254 is communicatively coupled to a NAT gateway 2238 contained in a control plane VCN 2216 and a data plane VCN 2218. A service gateway 2236 contained in both the control plane VCN 2216 and the data plane VCN 2218 is communicatively coupled to a cloud service 2256.

[0320] In some examples, Figure 22 The architecture shown in block diagram 2200 can be considered as... Figure 21 This is an exception to the pattern shown in the architecture of block diagram 2100, and this pattern may be what the IaaS provider's customers would expect if the IaaS provider cannot communicate directly with the customer (e.g., in a disconnected region). The customer can access in real time the corresponding container 2267(1)-(N) contained in each customer's VM 2266(1)-(N). Container 2267(1)-(N) can be configured to invoke a corresponding auxiliary VNIC 2272(1)-(N) contained in one or more application subnets 2226 of the data plane application layer 2246, which may be contained in the container egress VCN 2268. The auxiliary VNIC 2272(1)-(N) can transmit the call to a NAT gateway 2238, which can then transmit the call to the public internet 2254. In this example, containers 2267(1)-(N), which can be accessed by clients in real time, can be isolated from the control plane VCN 2216 and from other entities contained in the data plane VCN 2218. Containers 2267(1)-(N) can also be isolated from resources from other clients.

[0321] In other examples, a client can use container 2267(1)-(N) to invoke cloud service 2256. In this example, the client can run code within container 2267(1)-(N) requesting a service from cloud service 2256. Container 2267(1)-(N) can then transmit the request to auxiliary VNIC 2272(1)-(N), which can then transmit the request to a NAT gateway, which can then transmit the request to the public internet 2254. The public internet 2254 can then transmit the request via internet gateway 2234 to one or more LB subnets 2222 contained in control plane VCN 2216. In response to determining that the request is valid, the one or more LB subnets can then transmit the request to one or more application subnets 2226, which can then transmit the request to cloud service 2256 via service gateway 2236.

[0322] It should be recognized that the IaaS architectures 1900, 2000, 2100, and 2200 depicted in the figures may have other components besides those depicted. Furthermore, the embodiments shown in the figures are merely some examples of cloud infrastructure systems that can be incorporated into embodiments of this disclosure. In some other embodiments, the IaaS system may have more or fewer components than shown in the figures, may combine two or more components, or may have different configurations or component arrangements.

[0323] In some embodiments, the IaaS system described herein may include application suites, middleware, and database service offerings delivered to customers in a self-service, subscription-based, elastically scalable, reliable, highly available, and secure manner. An example of such an IaaS system is the Oracle Cloud Infrastructure (OCI) provided by this assignee.

[0324] Figure 23 An example computer system 2300, in which various embodiments can be implemented, is illustrated. System 2300 can be used to implement any of the computer systems described above. As shown, computer system 2300 includes a processing unit 2304 that communicates with a plurality of peripheral subsystems via a bus subsystem 2302. These peripheral subsystems may include a processing acceleration unit 2306, an I / O subsystem 2308, a storage subsystem 2318, and a communication subsystem 2324. Storage subsystem 2318 includes a tangible computer-readable storage medium 2322 and system memory 2310.

[0325] Bus subsystem 2302 provides a mechanism for allowing various components and subsystems of computer system 2300 to communicate with each other as intended. While bus subsystem 2302 is schematically shown as a single bus, alternative embodiments of the bus subsystem may utilize multiple buses. Bus subsystem 2302 can be any of several types of bus architectures, including memory buses or memory controllers, peripheral buses, and local buses using any of the various bus architectures. For example, such architectures may include Industry Standard Architecture (ISA) buses, Microchannel Architecture (MCA) buses, Enhanced ISA (EISA) buses, Video Electronics Standards Association (VESA) local buses, and Peripheral Component Interconnect (PCI) buses, which may be implemented as Mezzanine buses manufactured according to the IEEE P1386.1 standard.

[0326] A processing unit 2304, which may be implemented as one or more integrated circuits (e.g., a conventional microprocessor or microcontroller), controls the operation of the computer system 2300. One or more processors may be included in the processing unit 2304. These processors may include single-core or multi-core processors. In some embodiments, the processing unit 2304 may be implemented as one or more independent processing units 2332 and / or 2334, each including a single-core or multi-core processor. In other embodiments, the processing unit 2304 may also be implemented as a quad-core processing unit formed by integrating two dual-core processors into a single chip.

[0327] In various embodiments, processing unit 2304 can execute various programs in response to program code and can maintain multiple concurrently executing programs or processes. At any given time, some or all of the program code to be executed can reside in processor(s) 2304 and / or storage subsystem 2318. With appropriate programming, processor(s) 2304 can provide the various functions described above. Computer system 2300 may additionally include processing acceleration unit 2306, which may include digital signal processor (DSP), dedicated processor, etc.

[0328] The I / O subsystem 2308 may include user interface input devices and user interface output devices. User interface input devices may include keyboards, pointing devices such as mice or trackballs, touchpads or touchscreens integrated into a display, scroll wheels, click wheels, dials, buttons, switches, audio input devices with voice command recognition systems, microphones, and other types of input devices. User interface input devices may include, for example, motion sensing and / or gesture recognition devices, such as those from Microsoft… Motion sensors enable users to control devices such as Microsoft products via a natural user interface using gestures and voice commands. The 360 ​​game controller's input device interacts with it. The user interface input device may also include eye gesture recognition devices, such as detecting eye movements from the user (e.g., "blinking" when taking a photo and / or making a menu selection) and translating the eye gestures into the input device (e.g., Google). Google input in ) Blink detector. Additionally, the user interface input device may include enabling the user to interact with a voice recognition system (e.g., ...) via voice commands. Voice recognition sensing devices for interaction with navigators.

[0329] User interface input devices may also include, but are not limited to, 3D mice, joysticks or pointing sticks, game panels and drawing tablets, as well as audio / video devices such as speakers, digital cameras, digital camcorders, portable media players, webcams, image scanners, fingerprint scanners, barcode readers, 3D scanners, 3D printers, laser rangefinders, and eye-tracking devices. Furthermore, user interface input devices may include, for example, medical imaging input devices such as computed tomography, magnetic resonance imaging, positron emission tomography, or medical ultrasound equipment. User interface input devices may also include, for example, audio input devices such as MIDI keyboards, digital musical instruments, etc.

[0330] User interface output devices may include display subsystems, indicator lights, or non-visual displays such as audio output devices, etc. Display subsystems may be cathode ray tubes (CRTs), flat panel devices such as those using liquid crystal displays (LCDs) or plasma displays, projection devices, touchscreens, etc. Generally, the term "output device" is intended to include all possible types of devices and mechanisms for outputting information from computer system 2300 to the user or other computers. For example, user interface output devices may include, but are not limited to, various display devices that visually convey text, graphics, and audio / video information, such as monitors, printers, speakers, headphones, car navigation systems, plotters, voice output devices, and modems.

[0331] Computer system 2300 may include a storage subsystem 2318 containing software elements, shown as currently located in system memory 2310. System memory 2310 may store program instructions that can be loaded and executed on processing unit 2304, as well as data generated during the execution of these programs.

[0332] Depending on the configuration and type of computer system 2300, system memory 2310 may be volatile (such as random access memory (RAM)) and / or non-volatile (such as read-only memory (ROM), flash memory, etc.). RAM typically contains data and / or program modules that can be immediately accessed by processing unit 2304 and / or are currently being operated and executed by processing unit 2304. In some implementations, system memory 2310 may include various different types of memory, such as static random access memory (SRAM) or dynamic random access memory (DRAM). In some implementations, a basic input / output system (BIOS), containing basic routines that facilitate the transfer of information between elements of computer system 2300 during startup, may typically be stored in ROM. As an example, but not a limitation, system memory 2310 also includes application programs 2312, program data 2314, and an operating system 2316, which may include client applications, web browsers, middleware applications, relational database management systems (RDBMS), etc. As an example, operating system 2316 may include various versions of Microsoft... Apple and / or Linux operating system, and various commercially available... Or a UNIX-like operating system (including but not limited to various GNU / Linux operating systems, Google...) OS, etc.) and / or such as iOS, Phone OS 23OS and A mobile operating system based on the OS operating system.

[0333] Storage subsystem 2318 may also provide a tangible computer-readable storage medium for storing basic programming and data structures that provide the functionality of some embodiments. Software (programs, code modules, instructions) that provides the above-described functionality when executed by a processor may be stored in storage subsystem 2318. These software modules or instructions may be executed by processing unit 2304. Storage subsystem 2318 may also provide a repository for storing data used according to this disclosure.

[0334] Storage subsystem 2300 may also include a computer-readable storage medium reader 2320 that can be further connected to computer-readable storage medium 2322. Together with and optionally in conjunction with system memory 2310, computer-readable storage medium 2322 can comprehensively represent a remote, local, fixed, and / or removable storage device plus storage medium for temporarily and / or more persistently containing, storing, transmitting, and retrieving computer-readable information.

[0335] The computer-readable storage medium 2322 containing code or portions thereof may also include any suitable medium known or used in the art, including storage and communication media, such as, but not limited to, volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing and / or transmitting information. This may include tangible computer-readable storage media such as RAM, ROM, electrically erasable programmable ROM (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic tape cassettes, magnetic tape, disk storage or other magnetic storage devices, or other tangible computer-readable media. This may also include non-tangible computer-readable media such as data signals, data transmissions, or any other medium that can be used to transmit desired information and can be accessed by the computing system 2300.

[0336] For example, computer-readable storage medium 2322 may include a hard disk drive that reads from or writes to a non-removable non-volatile magnetic medium, a disk drive that reads from or writes to a removable non-volatile magnetic disk, and a removable non-volatile optical disc (such as a CD-ROM, DVD, etc.). An optical disc drive that reads from or writes to a disk or other optical medium. Computer-readable storage media 2322 may include, but is not limited to, Disk drives, flash memory cards, Universal Serial Bus (USB) flash drives, Secure Digital (SD) cards, DVDs, digital audio tapes, and so on. Computer-readable storage media 2322 may also include solid-state drives (SSDs) based on non-volatile memory (such as flash memory-based SSDs, enterprise flash drives, solid-state ROMs, etc.), volatile memory-based SSDs (such as solid-state RAM, dynamic RAM, static RAM), DRAM-based SSDs, magnetoresistive RAM (MRAM) SSDs, and hybrid SSDs using a combination of DRAM-based and flash memory-based SSDs. Disk drives and their associated computer-readable media can provide non-volatile storage of computer-readable instructions, data structures, program modules, and other data for computer system 2300.

[0337] The communication subsystem 2324 provides an interface to other computer systems and networks. The communication subsystem 2324 serves as an interface for receiving data from other systems and sending data from computer system 2300 to other systems. For example, the communication subsystem 2324 enables computer system 2300 to connect to one or more devices via the Internet. In some embodiments, the communication subsystem 2324 may include radio frequency (RF) transceiver components (e.g., advanced data network technologies using cellular telephone technologies, such as 3G, 4G, or EDGE (Enhanced Data Rates for Global Evolution), WiFi (IEEE 802.11 series standards), or other mobile communication technologies, or any combination thereof), GPS receiver components, and / or other components for accessing wireless voice and / or data networks. In some embodiments, as an addition to or alternative to the wireless interface, the communication subsystem 2324 may provide a wired network connection (e.g., Ethernet).

[0338] In some embodiments, the communication subsystem 2324 may also represent one or more users who can use the computer system 2300 to receive input communications in the form of structured and / or unstructured data feeds 2326, event streams 2328, event updates 2330, etc.

[0339] For example, the communication subsystem 2324 can be configured to receive data feeds 2326 in real time from users of social networks and / or other communication services, such as feed, Updates, web feeds such as rich site summary (RSS) feeds, and / or real-time updates from one or more third-party information sources.

[0340] Furthermore, the communication subsystem 2324 can also be configured to receive data in the form of a continuous data stream, which may include event streams 2328 and / or event updates 2330 that are essentially continuous or unbounded real-time events without a clearly defined termination. Examples of applications that generate continuous data may include, for example, sensor data applications, financial quotation machines, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, vehicle traffic monitoring, and so on.

[0341] The communication subsystem 2324 can also be configured to output structured and / or unstructured data feeds 2326, event streams 2328, event updates 2330, etc. to one or more databases, which can communicate with one or more streaming data source computers coupled to the computer system 2300.

[0342] Computer system 2300 can be one of various types, including handheld portable devices (e.g., Cellular phone Computing tablets, PDAs), and wearable devices (e.g., Glass head-mounted displays, PCs, workstations, mainframes, information stations, server racks, or any other data processing systems.

[0343] In the foregoing description, specific details have been set forth for purposes of explanation to provide a thorough understanding of the examples of this disclosure. However, it will be apparent that various examples can be practiced without these specific details. The following description is illustrative only and is not intended to limit the scope, applicability, or configuration of this disclosure. Rather, the subsequent description of the examples will provide an enabling description for those skilled in the art to implement the examples. It should be understood that various changes can be made to the function and arrangement of elements without departing from the spirit and scope of this disclosure as set forth in the appended claims. The drawings and descriptions are not intended to be restrictive. Circuits, systems, networks, processes, and other components may be shown as components in block diagram form to avoid obscuring the examples with unnecessary details. In other cases, well-known circuits, processes, algorithms, structures, and techniques may be shown without unnecessary details to avoid obscuring the examples. The teachings disclosed herein can also be applied to various types of applications, such as mobile applications, non-mobile applications, desktop applications, web applications, enterprise applications, etc. Furthermore, the teachings of this disclosure are not limited to a particular operating environment (e.g., operating system, device, platform, etc.), but can instead be applied to multiple different operating environments.

[0344] Furthermore, it should be noted that individual examples can be described as processes, which may be represented as flowcharts, flow diagrams, data flow diagrams, structure diagrams, or block diagrams. While flowcharts can describe operations as sequential processes, many operations can be executed in parallel or concurrently. Additionally, the order of operations can be rearranged. A process terminates when its operations are completed, but a process can have additional steps not included in the diagram. Processes can correspond to methods, functions, procedures, subroutines, subroutines, etc. When a process corresponds to a function, its termination can correspond to the function returning to the calling function or the main function.

[0345] The terms “example” and “exemplary” are used herein to mean “served as an example, instance, or illustration.” Any embodiment or design described herein as “exemplary” or “illustration” is not necessarily to be construed as preferred or advantageous over other embodiments or designs.

[0346] The terms "machine-readable storage medium" or "computer-readable storage medium" include, but are not limited to, portable or non-portable storage devices, optical storage devices, and various other media capable of storing, containing, or carrying one or more instructions and / or data. Machine-readable or computer-readable storage media may include non-transitory media in which data can be stored but do not include carrier waves and / or transient electronic signals propagated wirelessly or via a wired connection. Examples of non-transitory media may include, but are not limited to, disks or tapes, optical storage media such as CDs or DVDs, flash memory, or memory or storage devices thereof. Computer program products may include code and / or machine-executable instructions that may represent procedures, functions, subroutines, programs, routines, subroutines, modules, software packages, classes, or any combination of instructions, data structures, or program statements. Code segments may be coupled to another code segment or hardware circuitry by passing and / or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc., may be passed, forwarded, or transmitted via any suitable means, including memory sharing, message passing, token passing, network transmission, etc.

[0347] Furthermore, the examples can be implemented using hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented as software, firmware, middleware, or microcode, program code or code segments (e.g., computer program products) that perform the necessary tasks can be stored on a machine-readable medium. One or more processors can perform the necessary tasks. Some of the systems depicted in these figures can be provided in various configurations. In some examples, the system can be configured as a distributed system, where one or more components of the system are distributed across one or more networks in a cloud computing system. Where a component is described as being “configured” to perform certain operations, this configuration can be accomplished, for example, by designing electronic circuits or other hardware to perform the operations, by programming or controlling electronic circuits (e.g., microprocessors or other suitable electronic circuits) to perform the operations, or any combination thereof.

[0348] While specific embodiments of this disclosure have been described, various modifications, alterations, alternative constructions, and equivalents are also included within the scope of this disclosure. The embodiments of this disclosure are not limited to operation within certain specific data processing environments, but can be freely operated within multiple data processing environments. Furthermore, although embodiments of this disclosure have been described using a specific series of transactions and steps, those skilled in the art will understand that the scope of this disclosure is not limited to the described series of transactions and steps. Various features and aspects of the above embodiments can be used individually or in combination.

[0349] Furthermore, while embodiments of this disclosure have been described using specific combinations of hardware and software, it should be recognized that other combinations of hardware and software are also within the scope of this disclosure. Embodiments of this disclosure may be implemented using only hardware, or only software, or a combination thereof. The various processes described herein may be implemented in any combination on the same processor or on different processors. Thus, where a component or module is described as being configured to perform certain operations, such configuration may be accomplished, for example, by designing electronic circuitry to perform the operations, by programming programmable electronic circuitry (such as a microprocessor), or any combination thereof. Processes may communicate using a variety of technologies, including but not limited to conventional technologies for inter-process communication, and different pairs of processes may use different technologies, or the same pair of processes may use different technologies at different times.

[0350] Therefore, the specification and drawings are to be considered illustrative rather than restrictive. However, it will be apparent that additions, omissions, deletions, and other modifications and changes can be made thereto without departing from the broader spirit and scope set forth in the claims. Thus, while specific disclosed embodiments have been described, they are not intended to be limiting. Various modifications and equivalents are within the scope of the following claims.

[0351] Examples of embodiments of this disclosure may be described by considering the following terms:

[0352] Clause 1. A method comprising: generating access control list (ACL) information for traffic flows in a client's Layer 2 virtual network based on client input, wherein the Layer 2 virtual network is hosted on a physical network and includes multiple Layer 2 compute instances, multiple Layer 2 virtual network interfaces, and multiple Layer 2 virtual switches; determining that the ACL information applies to a subset of the multiple Layer 2 virtual network interfaces, the subset including the Layer 2 virtual network interfaces; determining that the ACL information will be sent to a network virtualization device (NVD) of the physical network, wherein: the NVD hosts the Layer 2 virtual network interfaces and Layer 2 virtual switches among the multiple Layer 2 virtual switches, the Layer 2 virtual network interfaces and Layer 2 virtual switches being associated with Layer 2 compute instances among the multiple Layer 2 compute instances, and the Layer 2 compute instances being hosted on a host machine on the physical network, the host machine being communicatively coupled to the NVD; and sending the ACL information to the NVD.

[0353] Clause 2. The method of Clause 1, wherein the ACL information, NVD, Layer 2 virtual network interface, and Layer 2 virtual switch are first ACL information, first NVD, first Layer 2 virtual network interface, and first Layer 2 virtual switch, and wherein the method further comprises: generating second ACL information applicable to the second Layer 2 virtual network interface based on client input; determining that the second ACL information will be sent to a second NVD of the physical network, wherein the second NVD hosts the second Layer 2 virtual network interface and a plurality of Layer 2 virtual switches; and sending the second ACL information to the second NVD.

[0354] Clause 3. The method of any of Clauses 1-2, wherein the flow of frames via the Layer 2 Virtual Network Interface to and / or to the Layer 2 Compute Instance is controlled by the NVD based on the stored association between the ACL information and the Layer 2 Virtual Network Interface.

[0355] Clause 4. The method of any one of Clauses 1-3 further includes: storing configuration information of the Layer 2 virtual network, wherein the configuration information indicates a client-specified name for the Layer 2 port; determining that the Layer 2 port is emulated by the Layer 2 virtual network interface; and storing the association between the client-specified name and the Layer 2 virtual network interface.

[0356] Clause 5. The method of Clause 4, wherein the client input indicates ACL information and a client-specified name of the Layer 2 port, and wherein the method further includes: identifying the Layer 2 virtual network interface based on the association between the client-specified name and the Layer 2 virtual network interface; and determining, based on the identified Layer 2 virtual network interface, that the ACL information applies to the Layer 2 virtual network interface.

[0357] Clause 6.4's method, wherein the client's input indicates ACL information and a client-specified name for the Layer 2 port, and wherein the method further includes: identifying the Layer 2 virtual network interface based on the association between the client-specified name and the Layer 2 virtual network interface; determining the media access control (MAC) address associated with the Layer 2 virtual network interface; and indicating to the NVD that the ACL information applies to the MAC address.

[0358] Clause 7. The method of Clause 6, wherein the client's input indicates ACL information and a client-specified name for the Layer 2 port, and wherein the method further includes: determining the Internet Protocol (IP) address associated with the NVD; and associating the ACL information with the MAC address and the IP address.

[0359] Clause 8. The method of any one of Clauses 1-7, wherein the client's input indicates that the ACL information is applicable to at least one of the following: destination port, source port, destination media access control (MAC) address, source MAC address, destination Internet protocol (IP) address, source IP address, communication protocol type, traffic broadcast, or traffic unicast.

[0360] Clause 9. A network virtualization device, comprising: one or more processors; and one or more computer-readable storage media storing instructions that, when executed by the one or more processors, configure the network virtualization device to: host a Layer 2 virtual network interface and a Layer 2 virtual switch belonging to a client's Layer 2 virtual network, wherein: the Layer 2 virtual network interface and the Layer 2 virtual switch are associated with a Layer 2 compute instance belonging to the Layer 2 virtual network, the Layer 2 compute instance being hosted on a host machine on a physical network including the network virtualization device, the host machine and the network virtualization device being communicatively coupled, and the Layer 2 virtual network being hosted on the physical network and including a plurality of Layer 2 compute instances, a plurality of Layer 2 virtual network interfaces, and a plurality of Layer 2 virtual switches; storing access control list (ACL) information associated with the Layer 2 virtual network interface; receiving frames with header information; determining, based on the header information, that a frame is associated with a Layer 2 virtual network interface; and controlling the flow of frames based on the ACL information.

[0361] Clause 10. Clause 9's network virtualization device, wherein ACL information controls inbound traffic to a Layer 2 compute instance, wherein header information includes a destination media access (MAC) address associated with a Layer 2 virtual network interface, and wherein the flow of frames via the Layer 2 virtual network interface to the Layer 2 compute instance is controlled based on a stored association between the destination MAC address and the ACL information.

[0362] Clause 11.9 The network virtualization device, wherein ACL information controls inbound traffic to a Layer 2 compute instance, wherein header information includes a source media access (MAC) address associated with another Layer 2 virtual network interface among a plurality of Layer 2 virtual network interfaces, and wherein the flow of frames via the Layer 2 virtual network interface to the Layer 2 compute instance is controlled based on a stored association between the source MAC address and the ACL information.

[0363] Clause 12. A network virtualization device of any of Clauses 9-11, wherein ACL information controls outbound traffic from a Layer 2 compute instance, wherein header information includes a source media access (MAC) address associated with a Layer 2 virtual network interface, and wherein the flow of frames from the Layer 2 compute instance via the Layer 2 virtual network interface is controlled based on a stored association between the source MAC address and the ACL information.

[0364] Clause 13. A network virtualization device of any of Clauses 9-11, wherein ACL information controls outbound traffic from a Layer 2 compute instance, wherein header information includes a destination media access (MAC) address associated with another Layer 2 virtual network interface among a plurality of Layer 2 virtual network interfaces, and wherein the flow of frames from the Layer 2 compute instance via the Layer 2 virtual network interface is controlled based on a stored association between the destination MAC address and the ACL information.

[0365] Clause 14. A network virtualization device of any of Clauses 9-13, wherein the client input indication ACL information is applicable to at least one of the following: destination port, source port, destination media access control (MAC) address, source MAC address, destination Internet protocol (IP) address, source IP address, communication protocol type, traffic broadcast or traffic unicast.

[0366] Clause 15. A network virtualization device of any of Clauses 9-14, wherein a Layer 2 compute instance, a Layer 2 virtual network interface, a Layer 2 virtual switch, and ACL information are respectively a first Layer 2 compute instance, a first Layer 2 virtual network interface, a first Layer 2 virtual switch, and first ACL information, and wherein one or more computer-readable storage media store additional instructions that, when executed by one or more processors, configure the network virtualization device to: host a second Layer 2 virtual network interface and a second Layer 2 virtual switch belonging to a Layer 2 virtual network, wherein the second Layer 2 virtual network interface and the second Layer 2 virtual switch are associated with a second Layer 2 compute instance belonging to a Layer 2 virtual network; and store second ACL information associated with the second Layer 2 virtual network interface.

[0367] Clause 16. A network virtualization device of Clause 15, wherein the frame and header information are a first frame and a first header information, respectively, and wherein one or more computer-readable storage media store further instructions that, when executed by one or more processors, configure the network virtualization device to: receive a second frame having a second header information; determine, based on the second header information, that the second frame is associated with a Layer 2 virtual network interface; and control the flow of the second frame based on second ACL information.

[0368] Clause 17. The network virtualization device of Clause 16, wherein a first ACL information controls inbound traffic to a first Layer 2 compute instance, wherein a second ACL information controls outbound traffic from the Layer 2 compute instance, wherein the flow of a first frame through a first Layer 2 virtual network interface to the first Layer 2 compute instance is controlled based on a first storage association between the first header information and the first ACL information, and wherein the flow of a second frame through a second Layer 2 virtual network interface from the first Layer 2 compute instance is controlled based on a second storage association between the second header information and the second ACL information.

[0369] Clause 18. A system comprising: one or more processors; and one or more computer-readable storage media storing instructions that, when executed by the one or more processors, configure the system to: generate Access Control List (ACL) information for traffic flows in a client's Layer 2 Virtual Network based on client input, wherein the Layer 2 Virtual Network is hosted on a physical network and includes multiple Layer 2 compute instances, multiple Layer 2 Virtual Network interfaces, and multiple Layer 2 Virtual Switches; determine that the ACL information applies to a subset of the multiple Layer 2 Virtual Network interfaces, the subset including the Layer 2 Virtual Network interfaces; determine that the ACL information will be sent to a Network Virtualization Device (NVD) of the physical network, wherein: the NVD hosts the Layer 2 Virtual Network interfaces and Layer 2 Virtual Switches among the multiple Layer 2 Virtual Switches, the Layer 2 Virtual Network interfaces and Layer 2 Virtual Switches being associated with Layer 2 compute instances among the multiple Layer 2 compute instances, and the Layer 2 compute instances being hosted on a host machine on the physical network, the host machine being communicatively coupled to the NVD; and send the ACL information to the NVD.

[0370] Clause 19. A system of Clause 18, wherein one or more computer-readable storage media store additional instructions that, when executed by one or more processors, configure the system to: store ACL information in association with information having a Layer 2 virtual network interface; receive frames with header information; determine, based on the header information, that a frame is associated with a Layer 2 virtual network interface; and control the flow of frames based on the ACL information.

[0371] Clause 20. A system of any of Clauses 18-19, wherein the flow of frames via a Layer 2 Virtual Network Interface to and / or to a Layer 2 Compute Instance is controlled by the NVD based on ACL information and the stored association between the ACL information and the Layer 2 Virtual Network Interface.

[0372] In the context of describing the disclosed embodiments (particularly in the context of the following claims), the terms “a,” “an,” and “the,” and similar designations, are to be interpreted as encompassing both singular and plural, unless otherwise indicated herein or clearly contradicted by the context. Unless otherwise stated, the terms “comprising,” “having,” “including,” and “containing” are to be interpreted as open-ended terms (i.e., meaning “including but not limited to”). The term “connected” should be interpreted as partially or wholly contained in, attached to, or joined together, even if something exists in between. Unless otherwise indicated herein, the enumeration of value ranges herein is intended only as a shorthand method for individually referencing each individual value falling within that range, and each individual value is incorporated into the specification as if it were individually enumerated herein. Unless otherwise indicated herein or clearly contradicted by the context, all methods described herein can be performed in any suitable order. The use of any and all examples or exemplary language (e.g., “such as”) provided herein is intended only to better illustrate embodiments of this disclosure and does not constitute a limitation on the scope of this disclosure, unless otherwise stated. Nothing in the specification should be construed as indicating that any unclaimed element is essential to the practice of this disclosure.

[0373] Exclusion language, such as the phrase “at least one of X, Y, or Z”, unless otherwise explicitly stated, is intended to be understood in the context generally used to represent items, terms, etc., and may be X, Y, or Z, or any combination thereof (e.g., X, Y, and / or Z). Therefore, such exclusion language is generally not intended to, and should not, imply that some embodiments require the presence of at least one of X, at least one of Y, or at least one of Z.

[0374] This document describes preferred embodiments of the present disclosure, including the best modes known for carrying out the present disclosure. Variations of those preferred embodiments will become apparent to those skilled in the art upon reading the foregoing description. Those skilled in the art should be able to suitably employ such variations and may practice the present disclosure in ways other than those specifically described herein. Thus, the present disclosure includes all modifications and equivalents to the subject matter recited in the appended claims, where permitted by applicable law. Furthermore, unless otherwise indicated herein, the present disclosure includes any combination of the foregoing elements in all its possible variations.

[0375] All references cited in this article, including publications, patent applications and patents, are incorporated into this article by reference to the same extent as if each reference individually and specifically indicated to be incorporated by reference and elaborated in full in this article.

[0376] In the foregoing specification, various aspects of this disclosure have been described with reference to specific embodiments thereof; however, those skilled in the art will recognize that this disclosure is not limited thereto. The various features and aspects of the foregoing disclosure may be used individually or in combination. Furthermore, embodiments may be used in any number of settings and applications other than those described herein without departing from the broader spirit and scope of this specification. Therefore, this specification and the accompanying drawings should be considered illustrative rather than restrictive.

Claims

1. A method for providing Layer 2 networking, comprising: Based on the client's input, access control list (ACL) information is generated for the traffic flow in the client's Layer 2 virtual network, which is hosted on the physical network and includes multiple Layer 2 compute instances, multiple Layer 2 virtual network interfaces, and multiple Layer 2 virtual switches; Determine that the ACL information applies to a subset of the plurality of Layer 2 virtual network interfaces, the subset including Layer 2 virtual network interfaces; It is determined that the ACL information will be sent to the network virtualization device (NVD) of the physical network, where: NVD-hosted Layer 2 virtual network interface and the Layer 2 virtual switches among the plurality of Layer 2 virtual switches. The Layer 2 virtual switch corresponds to a Layer 2 switch with Layer 2 ports in the physical network. The Layer 2 virtual network interface and the Layer 2 virtual switch are associated with the Layer 2 computing instance among the plurality of Layer 2 computing instances. The Layer 2 virtual network interface emulates the Layer 2 port to indicate the Layer 2 port on the Layer 2 virtual switch, and The Layer 2 computing instance is hosted on a host machine in a physical network, which is communicatively coupled to NVD; as well as Send ACL information to NVD.

2. The method of claim 1, wherein the ACL information, NVD, Layer 2 virtual network interface, and Layer 2 virtual switch are first ACL information, first NVD, first Layer 2 virtual network interface, and first Layer 2 virtual switch, and wherein the method further comprises: Based on the customer's input, generate second ACL information suitable for the Layer 2 virtual network interface; It is determined that the second ACL information will be sent to the second NVD of the physical network, wherein the second NVD hosts the Layer 2 virtual network interface and the Layer 2 virtual switches among the plurality of Layer 2 virtual switches; as well as Send the second ACL information to the second NVD.

3. The method of any one of claims 1-2, wherein the flow of frames from and / or to the Layer 2 compute instance via the Layer 2 virtual network interface is controlled by the NVD based on the stored association between the ACL information and the Layer 2 virtual network interface according to ACL information.

4. The method according to any one of claims 1-2, further comprising: Configuration information for the storage layer 2 virtual network, wherein the configuration information indicates the client-specified name of the layer 2 port; It is determined that the Layer 2 port is simulated by a Layer 2 virtual network interface; and Stores the association between the client-specified name and the Layer 2 virtual network interface.

5. The method of claim 4, wherein the client's input indicates ACL information and a client-specified name for the Layer 2 port, and wherein the method further comprises: Identify the Layer 2 virtual network interface based on the association between the customer-specified name and the Layer 2 virtual network interface; as well as The ACL information is determined to be applicable to the Layer 2 virtual network interface based on the identified Layer 2 virtual network interface.

6. The method of claim 4, wherein the client's input indicates ACL information and a client-specified name for the Layer 2 port, and wherein the method further comprises: Identify the Layer 2 virtual network interface based on the association between the customer-specified name and the Layer 2 virtual network interface; Determine the Media Access Control (MAC) address associated with the Layer 2 virtual network interface; as well as Indicate to NVD that the ACL information applies to the MAC address.

7. The method of claim 6, wherein the client's input indicates ACL information and a client-specified name for the Layer 2 port, and wherein the method further comprises: Determine the Internet Protocol (IP) address associated with the NVD; as well as Associate ACL information with MAC addresses and IP addresses.

8. The method of any one of claims 1-2, wherein the input indication ACL information of the client is applicable to at least one of the following: destination port, source port, destination media access control (MAC) address, source MAC address, destination Internet protocol (IP) address, source IP address, communication protocol type, traffic broadcast or traffic unicast.

9. A network virtualization device, comprising: One or more processors; as well as One or more computer-readable storage media storing instructions that, when executed by said one or more processors, configure the network virtualization device to: Hosting Layer 2 virtual network interfaces and Layer 2 virtual switches belonging to the customer's Layer 2 virtual network, including: The Layer 2 virtual switch corresponds to a Layer 2 switch with Layer 2 ports in the physical network. Layer 2 virtual network interfaces and Layer 2 virtual switches are associated with Layer 2 compute instances belonging to the Layer 2 virtual network. The Layer 2 virtual network interface emulates the Layer 2 port to indicate the Layer 2 port on the Layer 2 virtual switch. Layer 2 compute instances are hosted on host machines within a physical network that includes network virtualization devices. The host machines and network virtualization devices are communicatively coupled. The Layer 2 virtual network is hosted on the physical network and includes multiple Layer 2 compute instances, multiple Layer 2 virtual network interfaces, and multiple Layer 2 virtual switches; Stores access control list (ACL) information associated with the Layer 2 virtual network interface; Receive frames with header information; The header information is used to determine the association between the frame and the Layer 2 virtual network interface; as well as Stream control based on ACL information frames.

10. The network virtualization device of claim 9, wherein the ACL information controls the ingress traffic to the Layer 2 compute instance, wherein the header information includes a destination media access (MAC) address associated with the Layer 2 virtual network interface, and wherein the flow of frames through the Layer 2 virtual network interface to the Layer 2 compute instance is controlled based on a stored association between the destination MAC address and the ACL information.

11. The network virtualization device of claim 9, wherein the ACL information controls the ingress traffic to the Layer 2 compute instance, wherein the header information includes a source media access (MAC) address associated with another Layer 2 virtual network interface among the plurality of Layer 2 virtual network interfaces, and wherein the flow of frames via the Layer 2 virtual network interface to the Layer 2 compute instance is controlled based on a stored association between the source MAC address and the ACL information.

12. The network virtualization device of any one of claims 9-11, wherein the ACL information controls the outgoing traffic from the Layer 2 compute instance, wherein the header information includes a source media access (MAC) address associated with the Layer 2 virtual network interface, and wherein the flow of frames from the Layer 2 compute instance via the Layer 2 virtual network interface is controlled based on the stored association between the source MAC address and the ACL information.

13. The network virtualization device of any one of claims 9-11, wherein ACL information controls outbound traffic from the Layer 2 compute instance, wherein header information includes a destination media access (MAC) address associated with another Layer 2 virtual network interface among the plurality of Layer 2 virtual network interfaces, and wherein the flow of frames from the Layer 2 compute instance via the Layer 2 virtual network interface is controlled based on a stored association between the destination MAC address and the ACL information.

14. The network virtualization device as claimed in any one of claims 9-11, wherein the client input indication ACL information is applicable to at least one of the following: destination port, source port, destination media access control (MAC) address, source MAC address, destination Internet protocol (IP) address, source IP address, communication protocol type, traffic broadcast or traffic unicast.

15. The network virtualization device of any one of claims 9-11, wherein the Layer 2 compute instance, the Layer 2 virtual network interface, the Layer 2 virtual switch, and the ACL information are respectively a first Layer 2 compute instance, a first Layer 2 virtual network interface, a first Layer 2 virtual switch, and first ACL information, and wherein the one or more computer-readable storage media store additional instructions that, when executed by the one or more processors, configure the network virtualization device to: Hosting a Layer 2 virtual network interface and a Layer 2 virtual switch belonging to a Layer 2 virtual network, wherein the Layer 2 virtual network interface and the Layer 2 virtual switch are associated with a Layer 2 compute instance belonging to a Layer 2 virtual network; and Stores the second ACL information associated with the Layer 2 virtual network interface.

16. The network virtualization device of claim 15, wherein the frame and header information are respectively a first frame and first header information, and wherein the one or more computer-readable storage media store further instructions, which, when executed by the one or more processors, configure the network virtualization device to: Receive the second frame containing the second header information; Based on the second header information, it is determined that the second frame is associated with the Layer 2 2 virtual network interface; and The second frame's flow is controlled based on the second ACL information.

17. The network virtualization device of claim 16, wherein the first ACL information controls the inbound traffic to the first Layer 2 compute instance, wherein the second ACL information controls the outbound traffic from the Layer 2 compute instance, wherein the flow of a first frame through the first Layer 2 virtual network interface to the first Layer 2 compute instance is controlled based on a first storage association between the first header information and the first ACL information, and wherein the flow of a second frame through the second Layer 2 virtual network interface from the first Layer 2 compute instance is controlled based on a second storage association between the second header information and the second ACL information.

18. A system for providing Layer 2 networking, comprising: One or more processors; as well as One or more computer-readable storage media storing instructions that, when executed by said one or more processors, configure the system to: Based on the client's input, access control list (ACL) information is generated for the traffic flow in the client's Layer 2 virtual network, which is hosted on the physical network and includes multiple Layer 2 compute instances, multiple Layer 2 virtual network interfaces, and multiple Layer 2 virtual switches; Determine that the ACL information applies to a subset of the plurality of Layer 2 virtual network interfaces, the subset including the Layer 2 virtual network interfaces; It is determined that the ACL information will be sent to the network virtualization device (NVD) of the physical network, where: NVD-hosted Layer 2 virtual network interface and the Layer 2 virtual switches among the plurality of Layer 2 virtual switches. The Layer 2 virtual switch corresponds to a Layer 2 switch with Layer 2 ports in the physical network. The Layer 2 virtual network interface and the Layer 2 virtual switch are associated with the Layer 2 computing instance among the plurality of Layer 2 computing instances. The Layer 2 virtual network interface emulates the Layer 2 port to indicate the Layer 2 port on the Layer 2 virtual switch, and Layer 2 compute instances are hosted on host machines in a physical network, which are communicatively coupled to NVD; as well as Send ACL information to NVD.

19. The system of claim 18, wherein the one or more computer-readable storage media store additional instructions that, when executed by the one or more processors, configure the system to: ACL information is stored in association with information that has a Layer 2 virtual network interface; Receive frames with header information; The association between the frame and the Layer 2 virtual network interface is determined based on the header information; and Stream control based on ACL information frames.

20. The system of any one of claims 18-19, wherein the flow of frames from and / or to a Layer 2 compute instance via a Layer 2 virtual network interface is controlled by the NVD based on ACL information and the stored association between the ACL information and the Layer 2 virtual network interface.

Images