Cloud infrastructure resources for connecting service provider private networks to customer private networks

Abstract

It provides access to resources on a customer's private network to resources on a service provider's private network. [Solution] Service to Customer (S2C) resources are deployed on a cloud infrastructure to facilitate access. IP address ranges may overlap between private networks, and private IP addresses may be used for one or more private networks. The S2C resources enable data exchange between private networks. For example, the S2C resources translate between IP addresses so that data within each private network uses IP addresses that are appropriately handled by the private network.

Description

[Technical Field]

[0001] This international patent application claims priority to U.S. Patent Application No. 17 / 347,061, filed June 14, 2021, entitled "Cloud Infrastructure Resources for Connecting a Service Provider Private Network to a Customer Private Network," which claims the benefit of U.S. Provisional Application No. 63 / 149,273, filed February 13, 2021, the entire contents of which are incorporated by reference into this disclosure for all purposes. [Background technology]

[0002] background Cloud infrastructures such as Oracle® Cloud Infrastructure (OCI) can offer a suite of complementary cloud services that enable enterprises to build and run a wide range of applications and services in a highly available, cloud-hosted environment. Enterprises need a cloud environment that replicates their on-premises data center environment while providing all the benefits of a public cloud. Cloud infrastructure can provide high-performance compute, storage, and network functions in a flexible overlay virtual network that runs on a physical underlay network and is securely accessible from the enterprise's on-premises network. Cloud infrastructures such as OCI enable enterprises to manage their cloud-based workloads in the same way they manage on-premises workloads, enabling them to rapidly migrate many existing on-premises applications without making changes to the applications themselves. Enterprises can thus obtain all the benefits of the cloud with the same control, isolation, security, and predictable performance as their on-premises networks.

[0003] Virtual networking is the foundation of cloud infrastructure and applications because it enables access, connection, security, and modification of cloud resources. Virtual networking enables communication between multiple computers, virtual machines (VMs), virtual servers, or other devices in different physical locations. While physical networking connects computer systems through cables and other hardware, virtual networking uses software management to connect computers and servers in different physical locations over the Internet. Virtual networks use virtualized versions of traditional network elements, such as network switches, routers, and adapters, to enable more efficient routing and easier network configuration / reconfiguration.

[0004] Entities such as customers and service providers can configure private networks deployed on cloud infrastructure. Such private networks are also called virtual private networks or virtual cloud networks. Typically, private networks are isolated such that access to each private network is strictly controlled. For example, one entity cannot access another entity's private network, and data cannot be exchanged between entities' private networks. Summary of the Invention [Means for solving the problem]

[0005] overview FIELD OF THE DISCLOSURE This disclosure relates generally to virtual networking. More particularly, Techniques are described for providing access to resources on a service provider's private network to resources on a service provider's private network. This access may be provided via a set of network resources on a cloud infrastructure that virtualizes at least a portion of each of the two private networks. Herein, this access may be referred to as service provider-to-customer access or service-to-consumer access, and may be abbreviated as "S2C" access for brevity.

[0006] According to some embodiments of the present disclosure, two private networks may have overlapping Internet Protocol (IP) address ranges (e.g., Classless Inter-Domain Routing (CIDR) ranges). Additionally or alternatively, multiple customer private networks may have overlapping IP address ranges. Furthermore, even if the IP address ranges do not overlap, each private network may have its own set of private IP addresses.

[0007] Network resources of the cloud infrastructure are configured to provide S2C access by implementing different network functions. For brevity, these network resources may be referred to herein as "S2C resources." The network functions include translating IP addresses of two private networks that support inbound packet flow from the service provider's resources to the customer's resources and outbound packet flow from the customer's resources to the service provider's resources.

[0008] In a Domain Name System (DNS) environment, DNS queries from a service provider's resources (e.g., instances in the service provider's virtual cloud network) are resolved by the S2C resource to a reserved IP address that is mapped to an IP address of a customer's private network (e.g., a private endpoint in the customer's virtual cloud network). Inbound packet flows from the service provider's private network use the reserved IP address as their destination. The S2C resource updates its destination address to the customer's IP address to enter the customer's private network. Conversely, outbound packet flows from the customer's private network use the customer's IP address as their source. The S2C resource updates its source address to the reserved IP address to enter the service provider's private network.

[0009] In an actual application cluster (RAC) environment, an S2C resource receives RAC traffic from a service provider's resource (e.g., an instance in the service provider's virtual cloud network). This RAC traffic has a source IP address and a destination IP address of the service provider's private network. These IP addresses are updated to IP addresses of the customer's private network, and the RAC traffic is sent to the IP addresses of the customer's private network. Conversely, an S2C resource receives a RAC response from a customer's resource (e.g., a database service in the customer's virtual cloud network). This RAC response has a source IP address and a destination IP address of the customer's private network. These IP addresses are updated to IP addresses of the service provider's private network, and the RAC response is sent to the IP addresses of the service provider's private network.

[0010] For each service provider-customer pair, a specific set of resources can be deployed. The cloud infrastructure is configured with virtual network interface cards (VNIs). The S2C resource includes a group of virtualization devices that provide the S2C resource. The VNICs are connected to the S2C resources. The VNICs are also connected to two private clouds, but these VNICs may be of a different type than the VNICs of the S2C resources. Data exchanged between the two private networks via the S2C resources flows through the VNICs.

[0011] This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to be used alone to determine the scope of the claimed subject matter, which subject matter should be understood by reference to appropriate portions of the entire specification of this patent application, any and all drawings, and each claim.

[0012] These and other features and embodiments will become more apparent with reference to the following specification, claims and accompanying drawings.

[0013] Exemplary embodiments will now be described in detail with reference to the drawings. [Brief explanation of the drawings]

[0014] [Figure 1] FIG. 1 is a high-level diagram of a distributed environment illustrating a virtual or overlay cloud network hosted by a cloud service provider infrastructure, according to certain embodiments. [Figure 2] 1 is an architectural schematic diagram illustrating physical elements of a physical network within a CSPI, according to certain embodiments. [Figure 3] FIG. 1 illustrates an exemplary arrangement of CSPI in which a host machine is connected to multiple network virtualization devices (NVDs), according to certain embodiments. [Figure 4]FIG. 1 illustrates a connection between a host machine and an NVD that provides I / O virtualization to support multi-tenancy, according to certain embodiments. [Figure 5] 1 is a schematic block diagram illustrating a physical network provided by CSPI in accordance with certain embodiments. [Figure 6] FIG. 1 illustrates an example of an S2C service between private networks, according to at least one embodiment. [Figure 7] FIG. 1 illustrates an example of a service provider private network connected to a customer private network via a set of S2C resources supporting DNS traffic and proxy-less S2 traffic, in accordance with at least one embodiment. [Figure 8] FIG. 1 illustrates an example of a service provider private network connected to a customer private network via a set of S2C resources supporting RAC traffic, in accordance with at least one embodiment. [Figure 9] FIG. 1 illustrates an example of components of a service provider private network, in accordance with at least one embodiment. [Figure 10] FIG. 1 illustrates an example of components of a customer private network, in accordance with at least one embodiment. [Figure 11] FIG. 1 illustrates an example of components of a cloud infrastructure that implements S2C resources, according to at least one embodiment. [Figure 12] FIG. 1 illustrates an example of DNS traffic, according to at least one embodiment. [Figure 13] FIG. 1 illustrates an example of proxy-free traffic, according to at least one embodiment. [Figure 14] FIG. 10 illustrates another example of DNS traffic in accordance with at least one embodiment. [Figure 15] FIG. 2 illustrates an example of RAC traffic, according to at least one embodiment. [Figure 16]FIG. 1 illustrates an example flow for providing S2C services, according to at least one embodiment. [Figure 17] FIG. 1 illustrates an example flow for a service provider private network to support DNS, non-proxy, and RAC traffic, in accordance with at least one embodiment. [Figure 18] FIG. 1 illustrates an example flow for a customer private network to support DNS traffic, no-proxy traffic, and RAC traffic, in accordance with at least one embodiment. [Figure 19] FIG. 1 illustrates an example flow for S2C resources to support DNS services, according to at least one embodiment. [Figure 20] FIG. 1 illustrates an example flow for supporting proxy-less S2C traffic, in accordance with at least one embodiment. [Figure 21] FIG. 1 illustrates an example flow for S2C resources to support RAC services, in accordance with at least one embodiment. [Figure 22] FIG. 1 is a block diagram illustrating one pattern for implementing a cloud infrastructure as a service system in accordance with at least one embodiment. [Figure 23] FIG. 1 is a block diagram illustrating another pattern for implementing a cloud infrastructure as a service system in accordance with at least one embodiment. [Figure 24] FIG. 1 is a block diagram illustrating another pattern for implementing a cloud infrastructure as a service system in accordance with at least one embodiment. [Figure 25] FIG. 1 is a block diagram illustrating another pattern for implementing a cloud infrastructure as a service system in accordance with at least one embodiment. [Figure 26] FIG. 1 is a block diagram illustrating an exemplary computer system in accordance with at least one embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0015] Detailed Description The present disclosure relates generally to virtual networking, and more particularly to providing resources on a service provider's private network with access to resources on a customer's private network. S2C resources are deployed on a cloud infrastructure to facilitate access. Although IP address ranges may overlap between private networks and / or private IP addresses may be used for more than one private network, the S2C resources enable data exchange between the private networks. For example, the S2C resources translate between IP addresses so that data within each private network uses IP addresses that can be properly processed by the private network. The cloud infrastructure includes virtualization devices that provide virtual network interface cards (VNICs). For each service provider-customer pair, a set of S2C resources is deployed, and the VNICs are connected to these resources and the private network. Data exchange occurs via the VNICs and the S2C resources.

[0016] Example Virtual Networking Architecture The term cloud services generally refers to services that are made available on demand (e.g., via a subscription model) to users or customers by a cloud service provider (CSP) using systems and infrastructure (cloud infrastructure). Typically, the servers and systems that make up the CSP's infrastructure are separate from the customer's own on-premise servers and systems. Thus, customers can use cloud services provided by the CSP without having to separately purchase hardware and software resources for the services. Cloud services are: It is designed to provide subscribing customers with easy and scalable access to application and computing resources without requiring the customer to invest in procuring the infrastructure used to deliver the service.

[0017] There are several cloud service providers that offer different types of cloud services. Cloud services come in a variety of different types or types, such as Software-as-a-Service (SaaS), Platform-as-a-Service (PaaS), and Infrastructure-as-a-Service (IaaS). Includes models.

[0018] A customer can subscribe to one or more cloud services offered by a CSP. A customer can be any entity, such as an individual, an organization, or a business. When a customer subscribes or registers for a service offered by a CSP, a tenant or account is created for the customer. The customer can then access one or more subscribed cloud resources associated with the account through this account.

[0019] As mentioned above, IaaS (Infrastructure as a Service) is a service that provides services to a specific type of client. Cloud computing services. In the IaaS model, a CSP provides infrastructure (called cloud service provider infrastructure or CSPI) that customers can use to build their own customizable networks and deploy customer resources. Therefore, customer resources and networks are hosted in a distributed environment by the infrastructure provided by the CSP. This differs from traditional computing, where the customer's infrastructure hosts the customer's resources and networks.

[0020] CSPI may include interconnected high-performance computing resources, including various host machines, memory resources, and network resources, forming a physical network, also known as a substrate network or underlay network. CSPI resources may be distributed across one or more data centers, geographically dispersed across one or more geographic regions. Virtualization software can run on these physical resources to provide a virtualized distributed environment. Virtualization creates an overlay network (also known as a software-based network, software-defined network, or virtual network) on the physical network. The CSPI physical network provides the basis for creating one or more overlay or virtual networks on top of the physical network. A virtual or overlay network can include one or more virtual cloud networks (VCNs). A virtual network is implemented using software virtualization technologies (e.g., hypervisors, functions performed by network virtualization devices (NVDs) (e.g., smart NICs), top-of-rack (TOR) switches, smart TORs that implement one or more functions performed by the NVDs, and other mechanisms) to create a network abstraction layer that can run on top of the physical network. Virtual networks can take various forms, such as peer-to-peer networks and IP networks. A virtual network is typically either a Layer 3 IP network or a Layer 2 VLAN. Such a virtual network or overlay network is often called a virtual Layer 3 network or an overlay Layer 3 network. Examples of protocols developed for virtual networks are IP-in-IP (or Generic Routing Encapsulation (GRE)), Virtual Extensible LAN (VXLAN), and so on. -IETF RFC7348), virtual private networks (VPNs) (e.g., MPLS Layer 3 Virtual Private Networks (RFC4364)), VMware NSX, Generic Network Virtualization Encapsulation (GENEVE), etc.

[0021] In the case of IaaS, the infrastructure provided by the CSP (CSPI) is In the IaaS model, cloud computing service providers may host infrastructure elements (e.g., servers, storage devices, network nodes (e.g., hardware), deployment software, platform virtualization (e.g., hypervisor layer), etc.). In some cases, IaaS providers may offer various services (e.g., billing, monitoring, logging, security, load balancing, clustering, etc.) associated with those infrastructure elements. Because these services are policy-driven, IaaS users can maintain application availability and performance by implementing policies to drive load balancing. CSPI provides infrastructure and a set of complementary cloud services, enabling customers to build and run a wide range of applications and services in a highly available, hosted, distributed environment. CSPI provides high-performance computing resources and power, as well as storage capacity, over a flexible virtual network that can be securely accessed from various network locations, such as customer on-premises networks. When a customer subscribes or registers for an IaaS service offered by a CSP, the tenancy created for that customer is a secure, isolated partition from the CSP where the customer can create, organize, and manage their cloud resources.

[0022] Customers can build their own virtual networks using the compute, memory, and networking resources provided by CSPI. On these virtual networks, they can connect one or more customer resources or workloads, such as compute instances. For example, a customer can use resources provided by CSPI to build one or more customizable private virtual networks called virtual cloud networks (VCNs). The customer can then deploy one or more customer resources, such as compute instances, on the customer VCN. The compute instances may be virtual machines, bare metal instances, etc. Thus, CSPI provides infrastructure and a set of complementary cloud services that enable customers to build and run various applications and services in a highly available, virtualized host environment. While customers do not manage or control the underlying physical resources provided by CSPI, they do control the operating systems, storage, and deployed applications, and in some cases have limited control over some networking components (e.g., firewalls).

[0023] The CSP may provide a console that enables customers and network administrators to configure, access, and manage resources deployed in the cloud using CSPI resources. In certain embodiments, the console provides a web-based user interface that can be used to utilize and manage CSPI. In some embodiments, the console is a web-based application provided by the CSP.

[0024] CSPI can support single-tenancy or multi-tenancy architectures. In a single-tenancy architecture, software (e.g., applications, databases) or hardware elements (e.g., host machines or servers) serve a single customer or tenant. In a multi-tenancy architecture, software or hardware elements serve multiple customers or tenants. Thus, in a multi-tenancy architecture, CSPI resources are shared among multiple customers or tenants. In a multi-tenancy environment, CSPI employs precautions and safeguards to ensure that each tenant's data is isolated and not visible to other tenants.

[0025] In a physical network, a network endpoint (endpoint) refers to a computing device or system that is connected to the physical network and communicates bidirectionally with the connected network. A network endpoint of a physical network may be connected to a local area network (LAN), a wide area network (WAN), or other types of physical networks. Examples of traditional endpoints of a physical network include modems, hubs, bridges, switches, routers, and other networking devices, physical computers (or host machines), etc. Each physical device of a physical network has a fixed network address that can be used to communicate with the device. This fixed network address may be a Layer 2 address (e.g., a MAC address), a fixed Layer 3 address (e.g., an IP address), etc. In a virtualized environment or virtual network, endpoints may include various virtual endpoints, such as virtual machines hosted by elements of the physical network (e.g., hosted by a physical host machine). These endpoints of the virtual network are addressed by overlay addresses, such as overlay Layer 2 addresses (e.g., an overlay MAC address) and overlay Layer 3 addresses (e.g., an overlay IP address). Network overlays achieve flexibility by allowing network administrators to move overlay addresses associated with network endpoints using software management (e.g., via software implementing the virtual network's control plane). Thus, unlike physical networks, in virtual networks, overlay addresses (e.g., overlay IP addresses) can be moved from one endpoint to another using network management software. Because virtual networks are built on physical networks, both the virtual network and the underlying physical network are involved in communications between elements of the virtual network.To facilitate such communications, each element of the CSPI is configured to learn and store mappings that map overlay addresses in the virtual network to real physical addresses in the substrate network, or real physical addresses in the substrate network to overlay addresses in the virtual network. These mappings are used to facilitate communications. To facilitate virtual network routing, customer traffic is encapsulated.

[0026] Thus, physical addresses (e.g., physical IP addresses) are associated with elements of the physical network, and overlay addresses (e.g., overlay IP addresses) are associated with entities of the virtual network. Both physical and overlay IP addresses are real IP addresses. They are distinct from virtual IP addresses, which are mapped to multiple real IP addresses. Virtual IP addresses provide a one-to-many mapping between virtual IP addresses and multiple real IP addresses.

[0027] A cloud infrastructure or CSPI is physically hosted in one or more data centers in one or more regions around the world. A CSPI may include elements of a physical or substrate network and virtualized elements (e.g., virtual networks, compute instances, virtual machines) of a virtual network built on the physical network elements. In certain embodiments, a CSPI may be organized into realms, regions, and available services. The CSPI resources are organized and hosted in distinct domains. A region is a local geographic area that typically contains one or more data centers. The regions are generally independent of each other and may be separated by vast distances, e.g., across countries or continents. For example, a first region may be in Australia, another region may be in Japan, and yet another region may be in India. CSPI resources are divided among these regions such that each region has an independent subset of CSPI resources. Each region provides a set of core infrastructure services and resources, such as compute resources (e.g., bare metal servers, virtual machines, containers and related infrastructure), storage resources (e.g., , block volume storage, file storage, object storage, archive storage), networking resources (e.g., virtual cloud networks (VCNs), load balancing resources, connectivity to on-premises networks), database resources, edge networking resources (e.g., DNS), access management and monitoring resources, etc. Each region typically has multiple routes connecting it to other regions within the realm.

[0028] Typically, applications are deployed in the region where they will be used most (i.e., on infrastructure associated with that region) because using nearby resources is faster than using resources that are farther away. Applications may also be deployed in different regions for a variety of reasons, such as redundancy to mitigate the risk of region-wide events such as large weather systems or earthquakes, or to meet various requirements for legal jurisdictions, tax domains, and other business or societal criteria.

[0029] Data centers within a region may be further organized and subdivided into availability domains (ADs). An availability domain is one or more data centers located in a region. A region may correspond to a data center on the cloud. A region may consist of one or more availability domains. In such a distributed environment, CSPI resources may be region-specific, such as a virtual cloud network (VCN), or availability domain-specific, such as a compute instance.

[0030] ADs within a region are isolated from each other to be fault-tolerant, with the likelihood of simultaneous failures extremely low. This is achieved by configuring ADs so that they do not share critical infrastructure resources, such as networking, physical cables, cable routes, and cable entrances, so that a failure of one AD in a region rarely impacts the availability of other ADs in the same region. Connecting ADs within the same region to each other via low-latency, high-bandwidth networks provides highly available connections to other networks (e.g., the Internet, customer on-premises networks), allowing multiple ADs to be replicated for both high availability and disaster recovery. Crowdsense utilizes multiple ADs to ensure high availability and protect against resource failures. As the infrastructure provided by an IaaS provider grows, more regions and ADs may be added along with additional capacity. Traffic between available domains is typically encrypted.

[0031] In certain embodiments, regions are grouped into realms. A realm is a logical collection of regions. Realms are isolated from each other and do not share any data. Regions within the same realm can communicate with each other, but regions within different realms cannot. A CSP's customer tenancy or account exists in a single realm and can span one or more regions within that single realm. Typically, when a customer subscribes to an IaaS service, their tenancy or account is created in a customer-specified region (called their "home" region) within a realm. The customer can extend their tenancy to one or more other regions within the realm. The customer cannot access regions that do not exist within the realm in which the customer's tenancy resides.

[0032] An IaaS provider may offer multiple realms, each corresponding to a particular set of customers or users. For example, a commercial realm may be offered for commercial customers. As another example, a realm may be offered for a particular country or for customers in that country. As yet another example, a government realm may be offered for, for example, a government. For example, a government realm may be created for a particular government and may have a higher level of security than a commercial realm. For example, Oracle® Cloud Infrastructure The OCI cloud infrastructure currently offers one realm for the commercial sector and two realms for the government cloud sector (e.g., FedRAMP authorization and IL5 authorization).

[0033] In certain embodiments, an AD can be subdivided into one or more fault domains. A fault domain can be A failure domain is a grouping of infrastructure resources within an AD. A failure domain allows for the distribution of compute instances so that they are not located on the same physical hardware within a single AD. This is known as anti-affinity. A failure domain refers to a set of hardware elements (computers, switches, etc.) that share a single point of failure. A compute pool is logically divided into failure domains. Thus, a hardware failure or compute hardware maintenance event that affects one failure domain does not affect instances in other failure domains. Depending on the embodiment, the number of failure domains in each AD may vary. For example, in a particular embodiment, each AD contains three failure domains. Fault domains function as logical data centers within an AD.

[0034] When a customer subscribes to an IaaS service, resources from CSPI are provisioned to the customer and associated with the customer's tenancy. Customers can use these provisioned resources to build private networks and deploy resources on these networks. A customer network hosted on the cloud by CSPI is called a virtual cloud network (VCN). Customers can configure one or more virtual cloud networks (VCNs) using the CSPI resources allocated for the customer. A VCN is a virtual or software-defined private network. Customer resources deployed in a customer's VCN can include compute instances (e.g., virtual machines, bare metal instances) and other resources. These compute instances may represent various customer workloads, such as applications, load balancers, and databases. Compute instances deployed on a VCN can communicate with publicly accessible endpoints (public endpoints) over a public network, such as the Internet; with other instances in the same VCN or other VCNs (e.g., other VCNs of the customer or VCNs not belonging to the customer); with customer on-premises data centers or networks; with sender endpoints; and with other types of endpoints.

[0035] CSPs can offer a variety of services using CSPI. In some cases, customers of a CSPI themselves can act as service providers and provide services using CSPI resources. Service providers can expose service endpoints characterized by identifying information (e.g., IP addresses, DNS names, and ports). Customer resources (e.g., compute instances) can consume a particular service by accessing the service endpoint for that particular service exposed by the service. These service endpoints are generally publicly accessible over a public communications network, such as the Internet, with users using the public IP address associated with the endpoint. Publicly accessible network endpoints are sometimes referred to as public endpoints.

[0036] In certain embodiments, a service provider may expose a service through a service endpoint (sometimes referred to as a service endpoint). A customer of the service may access the service using the service endpoint. In certain embodiments, a service endpoint provided for a service may be accessed by multiple customers wishing to consume the service. In some implementations, a dedicated service endpoint may be provided to a customer, so that only that customer can access the service using that dedicated sendy endpoint.

[0037] In certain embodiments, when a VCN is created, it is associated with a private overlay Classless Inter-Domain Routing (CIDR) address space, which is a private overlay IP address range (e.g., 10.0 / 16) assigned to the VCN. A VCN includes associated subnets, route tables, and gateways. A VCN exists within a single region but can extend to one or more or all available domains in the region. A gateway is a virtual interface configured for a VCN that enables traffic communication between the VCN and one or more endpoints outside the VCN. One or more different types of gateways can be configured for a VCN to enable communication between different types of endpoints.

[0038] A VCN may be subdivided into one or more sub-networks, such as one or more subnetworks. A subnet is thus a unit or division that can be created within a VCN. A VCN can have one or more subnets. Each subnet in a VCN is associated with a contiguous range of overlay IP addresses (e.g., 10.0.0.0 / 24 and 10.0.1.0 / 24) that does not overlap with other subnets in the VCN and represents a subset of the VCN's address space.

[0039] Each compute instance is associated with a virtual network interface card (VNIC), which allows it to participate in a subnet of a VCN. A VNIC is a logical representation of a physical network interface card (NIC). Generally, a VNIC is an interface between an entity (e.g., a compute instance, a service) and a virtual network. A VNIC resides in a subnet and has one or more associated IP addresses and associated security rules or policies. A VNIC corresponds to a Layer 2 port on a switch. A VNIC connects a compute instance to a subnet within a VCN. A VNIC associated with a compute instance allows the compute instance to be part of a subnet of a VCN and enables the compute instance to communicate (e.g., send and receive packets) with endpoints on the same subnet as the compute instance, endpoints in a different subnet within the VCN, or endpoints outside the VCN. Thus, the VNIC associated with a compute instance determines how the compute instance connects with endpoints inside and outside the VCN. A VNIC for a compute instance is created and associated with the compute instance when the compute instance is created and added to a subnet within the VCN. If a subnet consists of a set of compute instances, it includes VNICs corresponding to the set of compute instances, each VNIC being connected to a compute instance in the set of compute instances.

[0040] Each compute instance is assigned a private overlay IP address via the VNIC associated with the compute instance. This private overlay IP address is assigned to the VNIC associated with the compute instance when the compute instance is created and is used to route traffic for the compute instance. All VNICs in a particular subnet use the same route table, security lists, and DHCP options. As mentioned above, each subnet in a VCN has overlay IP addresses that do not overlap with other subnets in that VCN and represent an address space subset of the VCN's address space (e.g., 10.0.0.0 / 24 and 10.0.1.0 / 2). 4) Associated with a contiguous range of overlay IP addresses. For a VNIC on a particular subnet of a VCN, the overlay IP address assigned to the VNIC is an address from the contiguous range of overlay IP addresses assigned to the subnet.

[0041] In certain embodiments, if desired, a compute instance can be assigned additional overlay IP addresses in addition to the private overlay IP address, for example, one or more public IP addresses in the case of a public subnet. These multiple addresses are assigned to the same VNIC or multiple VNICs associated with the compute instance. However, each instance has a primary VNIC associated with the overlay private IP address that is created and assigned to the instance at instance launch. This primary VNIC cannot be deleted. Additional VNICs, called secondary VNICs, can be added to an existing instance within the same availability domain as the primary VNIC. All VNICs are in the same availability domain as the instance. Secondary VNICs can be in the same subnetwork in the same VCN as the primary VNIC, or in different subnetworks in the same or a different VCN.

[0042] Compute instances can optionally be assigned public IP addresses if they are in a public subnet. When creating a subnet, you can specify that the subnet is either a public or private subnet. A private subnet means that resources (e.g., compute instances) and associated VNICs within the subnet cannot have public overlay IP addresses. A public subnet means that resources and associated VNICs within the subnet can have public IP addresses. Customers can specify subnets that exist across a single available domain or multiple available domains within a region or realm.

[0043] As described above, a VCN may be subdivided into one or more subnets. In certain embodiments, a virtual router (referred to as a VCN VR or simply VR) configured for a VCN enables communication between subnets of the VCN. For a subnet within a VCN, the VR represents the logical gateway for that subnet, enabling communication between the subnet (i.e., the compute instances on that subnet) and endpoints on other subnets within the VCN and other endpoints outside the VCN. A VCN VR is a logical entity configured to route traffic between VNICs in a VCN and a virtual gateway (gateway) associated with the VCN. Gateways are further described below with respect to FIG. 1. A VCN VR is a Layer 3 / IP layer concept. In one embodiment, there is one VCN VR per VCN. This VCN VR potentially has an unlimited number of ports addressed by IP addresses, one port for each subnet of the VCN. In this way, a VCN VR has a different IP address for each subnet of the VCN to which the VCN VR is connected. VRs are also connected to various gateways configured for the VCN. In certain embodiments, a particular overlay IP address from a subnet's overlay IP address range is reserved in a port of the subnet's VCN VR. For example, consider a VCN with two subnets, each with associated address ranges 10.0 / 16 and 10.1 / 16. For a first subnet in the VCN with address range 10.0 / 16, addresses from this range are reserved in a port of the subnet's VCN VR. In some cases, a first IP address from this range may be reserved in a VCN VR. For example, for a subnet with overlay IP address range 10.0 / 16, IP address 10.0.0.1 may be reserved in a port of the subnet's VCN VR. A second subnet in the same VCN with address range 10.1 / 16 may be reserved in a port of the subnet's VCN VR. In this case, the VCN VR may have a port in the second subnet with IP address 10.1.0.1. The VCN VR has a different IP address for each subnet in the VCN.

[0044] In some other embodiments, each subnet within a VCN may have its own associated VR that is addressable by the subnet using a reserved or default IP address associated with the VR. The reserved or default IP address may, for example, be the first IP address from an IP address range associated with the subnet. VNICs within a subnet can use this default or reserved IP address to communicate (e.g., send and receive packets) with the VR associated with the subnet. In such embodiments, a VR is the ingress / egress point for that subnet. VRs associated with subnets within a VCN can communicate with other VRs associated with other subnets within the VCN. VRs can also communicate with gateways associated with the VCN. The VR functions for a subnet are performed on or by one or more NVDs that perform the VNIC functions for VNICs within the subnet.

[0045] Route tables, security rules, and DHCP options may be configured for a VCN. A route table is a virtual route table for a VCN and contains rules for routing traffic from subnets inside the VCN to destinations outside the VCN through gateways or specially configured instances. You can customize a VCN's route table to control the forwarding / routing of packets into and out of the VCN. DHCP options refer to configuration information that is automatically provided to an instance when it is launched.

[0046] Security rules configured for a VCN represent the VCN's overlay firewall rules. Security rules can include inbound and outbound rules and can specify the type of traffic allowed in and out of instances in the VCN (e.g., based on protocol and port). Customers can choose whether certain rules are stateful or stateless. For example, a customer can allow incoming SSH traffic from anywhere to a set of instances by configuring a stateful inbound rule with source CIDR 0.0.0.0 / 0 and destination TCP port 22. Security rules may be implemented using network security groups or security lists. A network security group consists of a set of security rules that apply only to resources within that group. On the other hand, a security list contains rules that apply to all resources in subnets that use that security list. A VCN may also include default security rules and a default security list. DHCP options configured for a VCN provide configuration information that is automatically provided when instances in the VCN are launched.

[0047] In particular embodiments, configuration information for a VCN is determined and stored by a VCN control plane. The configuration information for a VCN may include, for example, address ranges associated with the VCN, subnets and associated information within the VCN, one or more VRs associated with the VCN, compute instances and associated VNICs within the VCN, NVDs that perform various virtualized network functions (e.g., VNICs, VRs, gateways) associated with the VCN, VCN state information, and other VCN-related information. In particular embodiments, a VCN distribution service publishes the configuration information stored by the VCN control plane, or portions thereof, to the NVD. The distributed information is used to update information stored and used by the NVD (e.g., forwarding tables, routing tables, etc.) to allow compute instances within the VCN to access the VCN. It can transfer packets between

[0048] In certain embodiments, VCN and subnet creation is handled by a VCN control plane (CP), and compute instance launch is handled by the compute control plane. The compute control plane is configured to allocate physical resources for the compute instance and then invoke the VCN control plane to create and attach VNICs to the compute instance. The VCN CP also sends VCN data mappings to a VCN data plane configured to perform packet forwarding and routing functions. In certain embodiments, the VCN CP provides a distribution service configured to provide updates to the VCN data plane. Examples of the VCN control plane are shown in Figures 21, 22, 23, and 24 (see reference numerals 2116, 2216, 2316, and 2416) and described below.

[0049] Customers can create one or more VCNs with resources hosted by CSPI. Compute instances deployed on a customer VCN can communicate with different endpoints. These endpoints can include endpoints hosted by CSPI and endpoints external to CSPL.

[0050] Various different architectures for implementing cloud-based services using CSPI are shown in Figures 1, 2, 3, 4, 5, 21, 22, 23, and 25 and described below. Figure 1 is a high-level diagram of a distributed environment 100 illustrating an overlay VCN or customer VCN hosted by CSPI, according to certain embodiments. The distributed environment shown in Figure 1 includes multiple elements in an overlay network. The distributed environment 100 shown in Figure 1 is merely an example and is not intended to unduly limit the scope of the claimed embodiments. Many variations, alternatives, and modifications are possible. For example, in some implementations, the distributed environment shown in Figure 1 may have more or fewer systems or elements than those shown in Figure 1, may combine two or more systems, or may have a different system configuration or arrangement.

[0051] As shown in the example of FIG. 1 , distributed environment 100 includes CSPI 101, which provides services and resources that customers can subscribe to and use to build a virtual cloud network (VCN). In a particular embodiment, CSPI 101 provides IaaS services to subscribing customers. Data centers within CSPI 101 may be organized into one or more regions. FIG. 1 shows an example region, "US Region" 102. A customer configures a customer VCN 104 for region 102. A customer can deploy various compute instances on VCN 104, which may include virtual machines or bare metal instances. Example instances include applications, databases, load balancers, etc.

[0052] In the embodiment shown in Figure 1, customer VCN 104 includes two subnets, "Subnet-1" and "Subnet-2," each with its own CIDR IP address range. In Figure 1, the overlay IP address range of Subnet-1 is 10.0 / 16, and the address range of Subnet-2 is 10.1 / 16. VCN virtual router 105 represents the logical gateway of the VCN, enabling communication between subnets in VCN 104 and with other endpoints outside the VCN. VCN VR105 is configured to route traffic between VNICs in VCN104 and gateways associated with VCN104. VCN VR105 provides a port for each subnet of VCN104. For example, VR105 may provide a port with IP address 10.0.0.1 for subnet-1 and a port with IP address 10.1.0.1 for subnet-2.

[0053] Multiple compute instances can be deployed on each subnet. In this case, the compute instances may be virtual machine instances and / or bare metal instances. The compute instances within a subnet may be hosted by one or more host machines within CSPI 101. A compute instance joins a subnet through a VNIC associated with the compute instance. For example, as shown in FIG. 1, compute instance C1 is part of subnet-1 through a VNIC associated with the compute instance. Similarly, compute instance C2 is part of subnet-1 through a VNIC associated with C2. Similarly, multiple compute instances, which may be virtual machine instances or bare metal instances, may be part of subnet-1. Each compute instance is assigned a private overlay IP address and MAC address through the associated VNIC. For example, in FIG. 1, compute instance C1 has an overlay IP address of 10.0.0.2 and a MAC address of M1, and compute instance C2 has a private overlay IP address of 10.0.0.3 and a MAC address of M2. Each compute instance in Subnet-1, including compute instances C1 and C2, has a default route to VCN VR105 using IP address 10.0.0.1, which is the IP address of a port in VCN VR105 in Subnet-1.

[0054] Subnet-2 may have multiple compute instances deployed, including virtual machine instances and / or bare metal instances. For example, as shown in FIG. 1, compute instances D1 and D2 are part of subnet-2 via VNICs associated with the respective compute instances. In the embodiment shown in FIG. 1, compute instance D1 has an overlay IP address of 10.1.0.2 and a MAC address of MM1, and compute instance D2 has a private overlay IP address of 10.1.0.3 and a MAC address of MM2. Each compute instance in subnet-2, including compute instances D1 and D2, has a default route to VCN VR105 using IP address 10.1.0.1, which is the IP address of a port in VCN VR105 in subnet-2.

[0055] VCN A 104 may also include one or more load balancers. For example, a load balancer may be provided for a subnet and configured to load balance traffic among multiple compute instances on the subnet. A load balancer may also be provided to load balance traffic among subnets within the VCN.

[0056] A particular compute instance deployed on VCN 104 can communicate with a variety of different endpoints. These endpoints may include endpoints hosted by CSPI 200 and endpoints external to CSPI 200. Endpoints hosted by CSPI 101 may include endpoints on the same subnet as the particular compute instance (e.g., communication between two compute instances in Subnet-1), endpoints in a different subnet but within the same VCN (e.g., communication between a compute instance in Subnet-1 and a compute instance in Subnet-2), endpoints in a different VCN in the same region (e.g., communication between a compute instance in Subnet-1 and an endpoint in a VCN in the same region 106 or 110, or communication between a compute instance in Subnet-1 and an endpoint in the service network 110 in the same region), or endpoints in a VCN in a different region (e.g., communication between a compute instance in Subnet-1 and an endpoint in a VCN in a different region 108). Additionally, compute instances in a subnet hosted by CSPI 101 can communicate with endpoints not hosted by CSPI 101 (i.e., external to CSPI 101). These external endpoints are , endpoints within customer on-premises networks 116, endpoints within other remote cloud host networks 118, public endpoints 114 accessible via public networks such as the Internet, and other endpoints.

[0057] Communication between compute instances on the same subnet is facilitated using VNICs associated with the source and destination compute instances. For example, compute instance C1 in Subnet-1 may want to send a packet to compute instance C2 in Subnet-1. For a packet sent from a source compute instance whose destination is another compute instance in the same subnet, the packet is first processed by the VNIC associated with the source compute instance. The processing performed by the VNIC associated with the source compute instance may include determining the packet's destination information from the packet header, identifying any policies (e.g., security lists) configured for the VNIC associated with the source compute instance, determining the packet's next hop, performing any packet encapsulation / decapsulation functions as needed, and forwarding / routing the packet to the next hop to facilitate communication of the packet to its intended destination. If the destination compute instance is in the same subnet as the source compute instance, the VNIC associated with the source compute instance is configured to identify the VNIC associated with the destination compute instance and forward the packet for processing to that VNIC. The VNIC associated with the destination compute instance is then executed and forwards the packet to the destination compute instance.

[0058] When communicating a packet from a compute instance in a subnet to an endpoint in a different subnet of the same VCN, the communication is facilitated by the VNICs and VCN VRs associated with the source and destination compute instances. For example, if compute instance C1 in Subnet-1 in Figure 1 wants to send a packet to compute instance D1 in Subnet-2, the packet is first processed by the VNIC associated with compute instance C1. The VNIC associated with compute instance C1 is configured to route the packet to VCN VR105 using the VCN VR's default route or port 10.0.0.1. VCN VR105 is configured to route the packet to Subnet-2 using port 10.1.0.1. The packet is then received and processed by the VNIC associated with D1, which forwards the packet to compute instance D1.

[0059] To communicate packets from a compute instance within VCN 104 to an endpoint outside VCN 104, the communication is facilitated by a VNIC associated with the source compute instance, VCN VR 105, and a gateway associated with VCN 104. One or more types of gateways can be associated with VCN 104. A gateway is an interface between a VCN and another endpoint, where the other endpoint is outside the VCN. A gateway is a Layer 3 / IP layer concept that enables a VCN to communicate with endpoints outside the VCN. Thus, a gateway facilitates traffic flow between a VCN and other VCNs or networks. A variety of different types of gateways can be configured in a VCN to facilitate different types of communications with different types of endpoints. Through gateways, communications may occur over a public network (e.g., the Internet) or a private network. These communications may use various communication protocols.

[0060] For example, compute instance C1 may want to communicate with an endpoint outside of VCN 104. The packet may first be processed by the VNIC associated with the source compute instance C1. The VNIC processing determines if the packet's destination is outside of Cl's subnet-1. The VNIC associated with C1 can forward the packet to VCN VR105 of VCN104. VCN VR105 then processes the packet and, as part of the processing, determines a particular gateway associated with VCN104 as the packet's next hop based on the packet's destination. VCN VR105 can then forward the packet to the particular gateway. For example, if the destination is an endpoint within a customer's operating premises network, the packet may be forwarded by VCN VR105 to a dynamic routing gateway (DRG) 122 configured for VCN104. The packet can then be forwarded from the gateway to the next hop to facilitate communication of the packet to its intended final destination.

[0061] Various different types of gateways may be configured for a VCN. An example of a gateway that may be configured for a VCN is shown in FIG. 1 and described below. Examples of gateways associated with a VCN are also shown in FIGS. 21, 22, 23, and 24 (e.g., gateways indicated by reference numbers 2134, 2136, 2138, 2234, 2236, 2238, 2334, 2336, 2338, 2434, 2436, and 2438) and described below. As shown in the embodiment shown in FIG. 1, a dynamic routing gateway (DRG) 122 may be added to or associated with the customer VCN 104. The DRG 122 provides a path for private network traffic communication between the customer VCN 104 and another endpoint. The other endpoint may be the customer on-premises network 116, a VCN 108 in a different region of the CSPI 101, or another remote cloud network 118 not hosted by the CSPI 101. The customer on-premises network 116 may be a customer network or customer data center built using customer resources. Access to the customer on-premises network 116 is typically highly restricted. For a customer that has both the customer on-premises network 116 and one or more VCNs 104 deployed or hosted in the cloud by CSPI 101, the customer may want the on-premises network 116 and the cloud-based VCNs 104 to be able to communicate with each other. This allows the customer to build an extended hybrid environment that includes the on-premises network 116 and the customer's VCNs 104 hosted by CSPI 101. The DRG 122 enables such communication. To enable such communication, a communication channel 124 is established. In this case, one endpoint of the communication channel is located in the customer on-premises network 116, and the other endpoint is located in CSPI 101 and connected to the customer VCN 104. The communication channel 124 can traverse a public communication network, such as the Internet, or a private communication network.A variety of different communication protocols can be used, such as IPsec VPN technology over a public communication network such as the Internet, or Oracle's FastConnect technology, which uses a private network instead of a public network. The device or equipment in the customer on-premises network 116 that forms one endpoint of the communication channel 124 is called customer premises equipment (CPE), such as the CPE 126 shown in Figure 1. The endpoint on the CSPI 101 side may be a host machine running the DRG 122.

[0062] In certain embodiments, remote peering connections (RPCs) can be added to a DRG, allowing customers to peer one VCN with another VCN in another region. Using such RPCs, a customer VCN 104 can connect to a VCN 108 in another region using a DRG 122. The DRG 122 can also connect to other remote cloud networks 118 not hosted by CSPI 101, such as Microsoft® Azure Cloud, Amazon® AWS Cloud, or other cloud services. It may also be used to communicate with the cloud.

[0063] As shown in Figure 1, an Internet Gateway (IGW) 120 can be configured in a customer VCN 104 to enable compute instances on the customer VCN 104 to communicate with public endpoints 114 accessible over a public network, such as the Internet. The IGW 120 is a gateway for connecting a VCN to a public network, such as the Internet. The IGW 120 enables public subnets in a VCN, such as VCN 104 (resources in the public subnet have public overlay IP addresses) to directly access public endpoints 112 on the public network 114, such as the Internet. The IGW 120 can be used to initiate connections from subnets in the VCN 104 or from the Internet.

[0064] Customer VCN 104 can be configured with a network address translation (NAT) gateway 128. NAT gateway 128 allows cloud resources in the customer VCN that do not have dedicated public overlay IP addresses to access the Internet without exposing them to direct incoming Internet connections (e.g., L4-L7 connections). This allows private subnets in a VCN, such as private subnet-1 in VCN 104, to privately access public endpoints on the Internet. With a NAT gateway, connections can be initiated from the private subnet to the public Internet, but connections cannot be initiated from the Internet to the private subnet.

[0065] In certain embodiments, a service gateway (SGW) 126 can be configured in customer VCN 104. SGW 126 provides a pathway for private network traffic between VCN 104 and service endpoints supported by service network 110. In certain embodiments, service network 110 may be provided by a CSP and may offer a variety of services. An example of such a service network is the Oracle® Service Network, which offers a variety of services available to customers. For example, a compute instance (e.g., a database system) in a private subnet of customer VCN 104 can back up data to a service endpoint (e.g., an object store) without requiring a public IP address or access to the Internet. In some embodiments, a VCN can have only one SGW, and connections can be initiated only from subnets within the VCN, not from service network 110. When a VCN is peered with another VCN, resources in the other VCN typically cannot access the SGW. Resources in an on-premises network connected to a VCN with FastConnect or VPN Connect can also use a service gateway configured in that VCN.

[0066] In some implementations, the SGW 126 uses service classless inter-domain routing (CIDR) labels. A CIDR label is a string that represents all regional public IP address ranges for a service or group of services of interest. Customers use service CIDR labels to control traffic to services when configuring the SGW and associated routing rules. Customers can optionally use service CIDR labels when configuring security rules without having to adjust the security rules if the service's public IP addresses change in the future.

[0067] A local peering gateway (LPG) 132 is a gateway that can be added to a customer VCN 104 to enable the VCN 104 to peer with another VCN in the same region. Peering allows traffic to travel across a network without going over a public network such as the Internet or through the customer on-premises network 116. This means that VCNs communicate using private IP addresses without routing traffic. In a preferred embodiment, a VCN has a separate LPG for each peering established. Local peering, or VCN peering, is a common practice used to establish network connectivity between different applications or infrastructure management functions.

[0068] A service provider, such as a provider of a service in service network 110, can provide access to a service using different access models. According to a public access model, the service may be exposed as a public endpoint publicly accessible by a compute instance in the customer VCN over a public network such as the Internet, or may be accessed privately through SGW 126. According to a specific private access model, the service may be accessed as a private IP endpoint in a private subnet in the customer VCN. This is called private endpoint (PE) access and allows a service provider to expose its service as an instance in the customer's private network. A private endpoint resource represents a service in a customer VCN. Each PE appears as a VNIC (called a PE-VNIC, which has one or more private IPs) that the customer selects from a subnet in the customer VCN. Thus, the PE provides a way to provide a service within the customer's private VCN subnet using a VNIC. Because the endpoint is exposed as a VNIC, the PE VNIC can utilize all the functionality associated with a VNIC, such as routing rules and security lists.

[0069] Service providers register services to make them accessible through PEs. Providers can associate policies with services that regulate the visibility of the service to customer tenants. Providers can register multiple services under a single virtual IP address (VIP), especially for multi-tenant services. There can also be multiple private endpoints (in multiple VCNs) that represent the same service.

[0070] Compute instances in the private subnet can then access the service using the private IP address or service DNS name of the PE VNIC. Compute instances in the customer VCN can access the service by sending traffic to the private IP address of the PE in the customer VCN. The Private Access Gateway (PAGW) 130 is a gateway resource that can connect to a service provider VCN (e.g., a VCN in the service network 110) and serves as the ingress / egress point for all traffic from / to the customer subnet private endpoints. The PAGW 130 allows providers to scale the number of PE connections without utilizing internal IP address resources. A provider only needs to configure one PAGW for any number of services registered in a single VCN. A provider can present services as private endpoints in multiple VCNs for one or more customers. From the customer's perspective, the PE VNIC appears not to be connected to the customer's instance but to the service the customer wants to interact with. Traffic destined for the private endpoint is routed to the service through the PAGW 130. These are called customer-to-service private connections (C2S connections).

[0071] Also, the PE concept is used to ensure that traffic is routed between the FastConnect / IPsec link and the customer VCN. Extending private access of services to customer on-premises networks and data centers by allowing traffic to flow through private endpoints within the network. It also extends private access of services to customer peering VCNs by allowing traffic to flow between LPG132 and PEs in the customer VCN.

[0072] Customers can control VCN routing at the subnet level, allowing them to specify which subnets in a customer VCN, such as VCN 104, use each gateway. A VCN's route tables can be used to determine whether traffic can be routed outside the VCN through a particular gateway. For example, in a particular case, the route table for a public subnet in customer VCN 104 can send non-local traffic through IGW 120. The route table for a private subnet in the same customer VCN 104 can send traffic to CSP services through SGW 126. All remaining traffic may be sent through NAT gateway 128. Route tables only control traffic that leaves the VCN.

[0073] Security lists associated with a VCN are used to control traffic entering the VCN through inbound connections and gateways. All resources within a subnet use the same mute tables and security lists. Security lists may be used to control specific types of traffic entering and leaving instances within a VCN's subnets. Security list rules may include inbound (inbound) rules and outbound (outbound) rules. For example, inbound rules may specify allowed source address ranges, and outbound rules may specify allowed destination address ranges. Security rules may specify specific protocols (e.g., TCP, ICMP), specific ports (e.g., port 22 for SSH, port 3389 for Windows RDP), etc. In certain implementations, the instance's operating system may enforce its own firewall rules that match security list rules. Rules may be stateful (e.g., connections are tracked and responses are automatically allowed without explicit security list rules for the response traffic) or stateless.

[0074] Access from a customer VCN (i.e., resources or compute instances deployed on VCN 104) can be categorized as public access, private access, or dedicated access. Public access refers to an access model for accessing public endpoints using public IP addresses or NAT. Private access enables customer workloads in VCN 104 with private IP addresses (e.g., resources in a private subnet) to access a service without traversing a public network such as the Internet. In particular embodiments, CSPI 101 enables customer VCN workloads with private IP addresses to access the service's public service endpoint using a service gateway. Thus, the service gateway provides a private access model by establishing a virtual link between the customer VCN and the service's public endpoint, which resides outside the customer's private network.

[0075] Additionally, CSPI is working to develop dedicated public peering services using technologies such as FastConnect public peering. It can provide brick access, where a customer's on-premises instance can access the FastConnect connection without going through a public network such as the Internet. You can access one or more services in a customer VCN using FastConnect. CSPI also provides dedicated private access using FastConnect private peering. In this case, a customer on-premise instance with a private IP address can be used. The instance can access the customer VCN workloads using a FastConnect connection. FastConnect connects customers' on-premises networks using the public internet. FastConnect is a network connection used instead of connecting your network to CSPI and its services. FastConnect offers higher bandwidth options and It provides an easy, flexible and economical way to create dedicated, private connections with a reliable and consistent networking experience.

[0076] FIG. 1 and the accompanying description above illustrate various virtualized elements in an exemplary virtual network. As noted above, a virtual network is built on an underlying physical or substrate network. FIG. 2 is a simplified architecture diagram illustrating physical elements within a physical network within CSPI 200 that provides the foundation for the virtual network, according to certain embodiments. As shown, CSPI 200 provides a distributed environment including elements and resources (e.g., compute, memory, and networking resources) provided by a cloud service provider (CSP). These elements and resources are used to provide cloud services (e.g., IaaS services) to subscribing customers, i.e., customers who subscribe to one or more services offered by the CSP. Based on the services to which the customer subscribes, CSPI 200 provides some resources (e.g., compute, memory, and networking resources) to the customer. The customer can then build their own cloud-based (i.e., CSPI-hosted), customizable private virtual network using the physical compute, memory, and networking resources provided by CSPI 200. As noted above, these customer networks are referred to as virtual cloud networks (VCNs). Customers can deploy one or more customer resources, such as compute instances, into these Customer VCNs. The compute instances may be virtual machines, bare metal instances, etc. CSPI200 provides infrastructure and a set of complementary cloud services that enable customers to build and run a wide range of applications and services in a highly available hosted environment.

[0077] In the exemplary embodiment shown in FIG. 2, the physical elements of CSPI 200 include one or more physical host machines or physical servers (e.g., 202, 206, 208), network virtualization devices (NVDs) (e.g., 210, 212), top-of-rack (TOR) switches (e.g., 214, 216), a physical network (e.g., 218), and switches within physical network 218. The physical host machines or servers can host and execute various compute instances participating in one or more subnets of a VCN. The compute instances may include virtual machine instances and bare metal instances. For example, the various compute instances shown in FIG. 1 may be hosted by the physical host machines shown in FIG. 2. The virtual machine compute instances in a VCN may be executed by one host machine or by multiple different host machines. Additionally, the physical host machines may host virtual host machines, container-based hosts or functions, etc. The VICs and VCN VRs shown in FIG. 1 may be executed by the FTVDs shown in FIG. 2. The gateway shown in FIG. 1 may be implemented by the host machine and / or NVD shown in FIG.

[0078] A host machine or server may run a hypervisor (also called a virtual machine monitor or VMM) that creates and enables a virtualized environment on the host machine. Virtualization or a virtualized environment facilitates cloud-based computing. One or more computing instances may be created, executed, and managed on the host machine by the hypervisor on the host machine. The hypervisor on the host machine enables the host machine's physical computing resources (e.g., compute resources, memory resources, and networking resources) to be shared among the various computing instances running on the host machine. It can be done.

[0079] For example, as shown in FIG. 2, host machines 202 and 208 execute hypervisors 260 and 266, respectively. These hypervisors may be implemented using software, firmware, hardware, or a combination thereof. Typically, a hypervisor is a process or software layer that resides in a host machine's operating system (OS), which executes on the host machine's hardware processor. The hypervisor provides a virtualized environment that allows the host machine's physical computing resources (e.g., processing resources such as processors / cores, memory resources, and networking resources) to be shared among various virtual machine computing instances executed by the host machine. For example, in FIG. 2, hypervisor 260 resides in the OS of host machine 202 and allows the host machine's computing resources (e.g., processing resources, memory resources, and networking resources) to be shared among computing instances (e.g., virtual machines) executed by host machine 202. A virtual machine can have its own OS (called a guest OS). This guest OS may be the same as or different from the host machine's OS. The OS of a virtual machine executed by a host machine may be the same as or different from the OS of other virtual machines executed by the same host machine. Thus, the hypervisor can run multiple OSs in parallel while sharing the same computing resources of the host machine. The host machines shown in Figure 2 may have the same type of hypervisor or different types of hypervisors.

[0080] A compute instance may be a virtual machine instance or a bare metal instance. In Figure 2, compute instance 268 on host machine 202 and compute instance 274 on host machine 208 are examples of virtual machine instances. Host machine 206 is an example of a bare metal instance provided to a customer.

[0081] In certain examples, an entire host machine may be provided to a single customer, and one or more compute instances (either virtual machines or bare metal instances) hosted by that host machine may all belong to the same customer. In other examples, a host machine may be shared among multiple customers (i.e., multiple tenants). In such a multi-tenant scenario, a host machine may host virtual machine compute instances belonging to different customers. These compute instances may be members of different VCNs for different customers. In certain embodiments, bare metal compute instances are hosted by bare metal servers without a hypervisor. When bare metal compute instances are provided, a single customer or tenant maintains control of the physical CPU, memory, and network interfaces of the host machine hosting the bare metal instance, and the host machine is not shared with other customers or tenants.

[0082] As previously described, each compute instance that is part of a VCN is associated with a VNIC that enables the compute instance to be a member of a subnet of the VCN. The VNIC associated with a compute instance facilitates communication of packets or frames to and from the compute instance. A VNIC is associated with the compute instance when the compute instance is created. In particular embodiments, for a compute instance executed by a host machine, the VNIC associated with the compute instance is executed by an NVD connected to the host machine. For example, in FIG. 2 , host machine 202 executes virtual machine compute instance 268 associated with VNIC 276, which is executed by NVD 210 connected to host machine 202. As another example, bare metal instance 27 hosted by host machine 206 2 is associated with VNIC 280 executed by NVD 212 connected to host machine 206. As yet another example, VNIC 284 is associated with compute instance 274 executed by host machine 208, and VNIC 284 is executed by NVD 212 connected to host machine 208.

[0083] For a compute instance hosted by a host machine, the NVD connected to that host machine executes a VCN VR corresponding to the VCN of which the compute instance is a member. For example, in the embodiment shown in Figure 2, NVD 210 executes VCN VR 277 corresponding to the VCN of which compute instance 268 is a member. NVD 212 may also execute one or more VCN VRs 283 corresponding to the VCNs corresponding to the compute instances hosted by host machines 206 and 208.

[0084] A host machine may include one or more network interface cards (NICs) for connecting the host machine to other devices. The NICs on a host machine may provide one or more ports (or interfaces) for communicatively connecting the host machine to another device. For example, one or more ports (or interfaces) on the host machine and the NVD may be used to connect the host machine to the NVD. The host machine may also be connected to other devices, such as other host machines.

[0085] 2, host machine 202 is connected to NVD 210 using link 220 extending between port 234 provided by NIC 232 of host machine 202 and port 236 of NVD 210. Host machine 206 is connected to NVD 212 using link 224 extending between port 246 provided by NIC 244 of host machine 206 and port 248 of NVD 212. Host machine 208 is connected to NVD 212 using link 226 extending between port 252 provided by NIC 250 of host machine 208 and port 254 of NVD 212.

[0086] Similarly, the NVDs are connected via communication links to top-of-rack (TOR) switches, which are connected to a physical network 218 (also called a switch fabric). In particular embodiments, the links between the host machines and the NVDs and the links between the NVDs and the TOR switches are Ethernet links. For example, in FIG. 2, NVDs 210 and 212 are connected to TOR switches 214 and 216, respectively, via links 228 and 230. In particular embodiments, links 220, 224, 226, 228, and 230 are Ethernet links. A collection of host machines and NVDs connected to a TOR may be referred to as a rack.

[0087] The physical network 218 provides a communications fabric that enables the TOR switches to communicate with each other. The physical network 218 may be a multi-tier network. In a particular implementation, the physical network 218 is a multi-tier Clos network of switches, with the TOR switches 214 and 216 representing leaf-level nodes of the multi-tier and multi-node physical switching network 218. Different Clos network configurations are possible, including, but not limited to, 2-tier networks, 3-tier networks, 4-tier networks, 5-tier networks, and generally "n"-tier networks. An example of a Clos network is shown in FIG. 5 and described below.

[0088] There are various possible connection configurations between the host machine and the N virtual machines, such as a one-to-one configuration, a many-to-one configuration, and a one-to-many configuration. In a one-to-one configuration, each host machine Each host machine is connected to its own separate NVD. For example, in Figure 2, host machine 202 is connected to NVD 210 via NIC 232 of host machine 202. In a many-to-one configuration, multiple host machines are connected to a single NVD. For example, in Figure 2, host machines 206 and 208 are connected to the same NVD 212 via NICs 244 and 250, respectively.

[0089] In a one-to-many configuration, one host machine is connected to multiple NVDs. Figure 3 shows an example of a CSPI 300 in which a host machine is connected to multiple NVDs. As shown in Figure 3, a host machine 302 includes a network interface card (NIC) 304 including multiple ports 306 and 30S. The host machine 300 is connected to a first NVD 310 via port 306 and link 320, and to a second NVD 312 via port 308 and link 322. Ports 306 and 308 may be Ethernet ports, and links 320 and 322 between the host machine 302 and the NVDs 310 and 312 may be Ethernet links. The NVD 310 is connected to a first TOR switch 314, and the NVD 312 is connected to a second TOR switch 316. The links between the NVDs 310 and 312 and the TOR switches 314 and 316 may be Ethernet links. TOR switches 314 and 316 represent layer-0 switching devices within a multi-tier physical network 318 .

[0090] 3 provides two separate physical network paths from the physical switch network 318 to the host machine 302: a first path from the TOR switch 314 to the host machine 302 via the NVD 310, and a second path from the TOR switch 316 to the host machine 302 via the NVD 312. The separate paths provide enhanced availability (referred to as high availability) for the host machine 302. If there is a problem with one of the paths (e.g., a link on one of the paths fails) or if there is a problem with a device (e.g., a particular NVD is not functioning), the other path can be used for communications to and from the host machine 302.

[0091] In the configuration shown in Figure 3, the host machine is connected to two different NVDs using two different ports provided by the host machine's NIC. In other embodiments, the host machine may include multiple NICs allowing the host machine to connect to multiple NVDs.

[0092] Referring again to Figure 2, an NVD is a physical device or element that performs one or more network virtualization functions and / or storage virtualization functions. An NVD may be any device that has one or more processing units (e.g., a CPU, a network processing unit (NPU), an FPGA, a packet processing pipeline), memory including cache, and ports. Various virtualization functions may be performed by software / firmware executed by one or more processing units of the NVD.

[0093] The NVD may be implemented in a variety of different ways. For example, in a particular embodiment, the NVD is implemented as an interface card with an embedded processor, called a smart NIC or intelligent NIC. The smart NIC is a separate device from the NIC on the host machine. In Figure 2, the NVD 210 may be implemented as a smart NIC connected to the host machine 202, and the NVD 212 may be implemented as a smart NIC connected to the host machines 206 and 208.

[0094] However, a smart NIC is only one example of an NVD implementation. Various other implementations are possible. For example, in some other implementations, the NVD or the functions performed by the NVD may be implemented in a different way. One or more functions provided by the NVD may be incorporated into or performed by one or more host machines, one or more TOR switches, and other elements of CSPI 200. For example, the NVD may be integrated into a host machine. In this case, the functions performed by the NVD are performed by the host machine. As another example, the NVD may be part of a TOR switch, or a TOR switch may be configured to perform the functions performed by the NVD, which enables the TOR switch to perform various complex packet transformations used in public clouds. A TOR that performs the functions of an NVD may be referred to as a smart TOR. In yet other implementations that offer customers virtual machine (VM) instances rather than bare metal (BM) instances, the functions provided by the NVD may be implemented inside the hypervisor of a host machine. In some other implementations, some of the functions of the NVD may be offloaded to a centralized service running on a set of host machines.

[0095] In certain embodiments, such as when implemented as a smart NIC, as shown in FIG. 2, an NVD may include multiple physical ports that allow the NVD to connect to one or more host machines and one or more TOR switches. Ports on an NVD can be categorized as host-facing ports (also called "south ports") or network-facing or TOR-facing ports (also called "north ports"). A host-facing port of an NVD is a port used to connect the NVD to a host machine. Examples of host-facing ports in FIG. 2 include port 236 of NVD 210 and ports 248 and 254 of NVD 212. A network-facing port of an NVD is a port used to connect the NVD to a TOR switch. Examples of network-facing ports in FIG. 2 include port 256 of NVD 210 and port 258 of NVD 212. As shown in FIG. 2, the NVD 210 is connected to the TOR switch 214 via link 228 extending from port 256 of NVD 210 to the TOR switch 214. Similarly, the NVD 212 is connected to the TOR switch 216 via a link 230 that extends from a port 258 of the NVD 212 to the TOR switch 216 .

[0096] The NVD can receive packets and frames (e.g., packets and frames generated by compute instances hosted by the host machine) from the host machine via its host-facing port, perform any necessary packet processing, and then forward the packets and frames to the TOR switch via the NVD's network-facing port. The NVD can receive packets and frames from the TOR switch via the NVD's network-facing port, perform any necessary packet processing, and then forward the packets and frames to the host machine via the NVD's host-facing port.

[0097] In certain embodiments, multiple ports and associated links may be provided between the NVD and the TOR switch. These ports and links can be aggregated to form a link aggregator group (LAG) of multiple ports or links. Link aggregation allows multiple physical links between two endpoints (e.g., between the NVD and the TOR switch) to be treated as a single logical link. All physical links within a given LAG can operate at the same speed and in full-duplex mode. LAGs help increase the bandwidth and reliability of the connection between two endpoints. If one of the physical links in a LAG fails, traffic is dynamically and transparently reassigned to another physical link within the LAG. The aggregated physical link provides higher bandwidth than individual links. Multiple ports associated with a LAG are treated as a single logical port. Traffic can be load-balanced across the multiple physical links in the LAG. One or more LAGs can be configured between two endpoints. The two endpoints may be, for example, between the NVD and the TOR switch, or between a host machine and the NVD.

[0098] The NVD implements or performs network virtualization functions. These functions are performed by software / firmware executed by the NVD. Examples of network virtualization functions include, but are not limited to, packet encapsulation and decapsulation functions, functions for creating VCN networks, functions for implementing network policies such as VCN security list (firewall) functions, functions for facilitating routing and forwarding of packets to and from compute instances within the VCN, etc. In certain embodiments, upon receiving a packet, the NVD is configured to execute a packet processing pipeline that processes the packet and determines how to forward or route the packet. As part of this packet processing pipeline, the NVD executes VNICs associated with cis in the VCN, virtual routers (VRs) associated with the VCN, and the like. , encapsulating and decapsulating packets to facilitate forwarding or routing within the virtual network, enforcing specific gateways (e.g., local peering gateways), implementing security lists, network security groups, network address translation (NAT) functions (e.g., public IP to private IP translation per host), throttling functions, and other functions.

[0099] In some embodiments, the packet processing data path in the NVD may include multiple packet pipelines. Each packet pipeline consists of a series of packet transformation stages. In some implementations, upon receiving a packet, the packet is parsed and sorted into a single pipeline. The packet is then processed stage by stage in a linear fashion until it is discarded or sent out through an interface of the NVD. These stages provide packet processing building blocks of basic functions (e.g., validating headers, performing throttling, inserting new Layer 2 headers, performing L4 firewalling, VCN encapsulation / decapsulation), such that new pipelines can be constructed by assembling existing stages, and new functionality can be added by creating and inserting new stages into existing pipelines.

[0100] The NVD can perform both control plane and data plane functions corresponding to the VCN's control plane and data plane. Examples of the VCN control plane are shown in Figures 21, 22, 23, and 24 (see reference numbers 2116, 2216, 2316, and 2416) and described below. Examples of the VCN data plane are shown in Figures 21, 22, 23, and 24 (see reference numbers 2118, 2218, 2318, and 2418) and described below. Control plane functions include functions used to configure the network (e.g., setting routes and route tables, configuring VNICs) to control how data is forwarded. In certain embodiments, a VCN control plane is provided that centrally computes and exposes all overlay-to-substrate mappings to the NVD and virtual network edge devices (e.g., various gateways such as DRGs, SGWs, and IGWs). Firewall rules can also be exposed using the same mechanism. In certain embodiments, the NVD retrieves only mappings relevant to the NVD. The data plane functions include those that perform the actual routing / forwarding of packets based on the configurations established using the control plane. The VCN data plane is implemented by encapsulating customer network packets before they traverse the backbone network. The encapsulation / decapsulation functions are implemented in the NVD. In certain embodiments, the NVD is configured to intercept all network packets entering and leaving the host machine and perform network virtualization functions.

[0101] As mentioned above, the NVD performs various virtualization functions, including VNICs and VCN VRs. The NVD can run VNICs associated with compute instances hosted by one or more host machines connected to the VNICs. For example, the NVD shown in Figure 2 As such, NVD 210 executes the functions of VNIC 276 associated with compute instance 268 hosted by host machine 202 connected to NVD 210. As another example, NVD 212 executes VNIC 280 associated with bare metal compute instance 272 hosted by host machine 206 and executes VNIC 284 associated with compute instance 274 hosted by host machine 208. A host machine may host compute instances that belong to different VCNs that belong to different customers. An NVD connected to a host machine may execute the VNICs (i.e., execute the functions associated with the VNICs) corresponding to the compute instances.

[0102] NVDs also execute VCN virtual routers corresponding to the VCNs of the compute instances. For example, in the embodiment shown in FIG. 2, NVD 210 executes VCN VR 277 corresponding to the VCN to which compute instance 268 belongs. NVD 212 executes one or more VCN VRs 283 corresponding to one or more VCNs to which compute instances hosted on host machines 206 and 208 belong. In particular embodiments, a VCN VR corresponding to a VCN is executed by all NVDs connected to a host machine that hosts at least one compute instance belonging to that VCN. If a host machine hosts compute instances that belong to different VCNs, the NVDs connected to that host machine may execute VCN VRs corresponding to the different VCNs.

[0103] In addition to VNICs and VCN VRs, an NVD may include one or more hardware elements that run various software (e.g., daemons) and facilitate various network virtualization functions performed by the NVD. For simplicity, these various elements are grouped as "packet processing elements" shown in FIG. 2. For example, NVD 210 includes packet processing element 286, and NVD 212 includes packet processing element 288. For example, the packet processing element of an NVD may include a packet processor configured to monitor all packets received and communicated using the NVD by interacting with the NVD's ports and hardware working interfaces and store network information. The network information may include, for example, network flow information for identifying different network flows processed by the NVD and information about each flow (e.g., statistics for each flow). In certain embodiments, the network flow information may be stored on a per-VNIC basis. As another example, the packet processing element may include a replication agent configured to replicate information stored by the NVD to one or more different replication target stores. As yet another example, the packet processing element may include a packet processor configured to monitor all packets received and communicated using the NVD by interacting with the NVD's ports and hardware working interfaces and store network information. The network information may include, for example, network flow information for identifying different network flows processed by the NVD and information about each flow (e.g., statistics for each flow). In certain embodiments, the network flow information may be stored on a per-VNIC basis. As another example, the packet processing element may include a replication agent configured to replicate information stored by the NVD to one or more different replication target stores. The packet processing element may include a logging agent configured to perform logging functions for the NVD. The packet processing element may also include a logging agent configured to monitor the performance and It may also include software for monitoring the health and possibly the status and health of other elements connected to the NVD.

[0104] FIG. 1 illustrates elements of an exemplary virtual or overlay network, including a VCN, subnets within the VCN, compute instances deployed on the subnets, VNICs associated with the compute instances, VRs for the VCN, and a set of gateways configured for the VCN. The overlay elements illustrated in FIG. 1 may be executed or hosted by one or more of the physical elements illustrated in FIG. 2. For example, compute instances within a VCN may be executed or hosted by one or more host machines illustrated in FIG. 2. For compute instances hosted by a host machine, the VNICs associated with the compute instance are typically executed by an NVD attached to the host machine (i.e., the VNIC functionality is provided by an NVD attached to the host machine). The VCN VR functionality for the VCN is executed by all NVDs attached to the host machines that host or execute compute instances that are part of the VCN. The gateways associated with the VCN may be executed by one or more different types of NVDs. For example, some gateways may be implemented by smart NICs, while other gateways may be implemented by one or more host machines or other implementations of NVDs.

[0105] As described above, compute instances within a customer VCN can communicate with a variety of different endpoints. These endpoints may be in the same subnet as the source compute instance, in a different subnet but in the same VCN as the source compute instance, or may include endpoints outside the VCN of the source compute instance. These communications are facilitated using the VNICs associated with the compute instances, the VCN VRs, and the gateways associated with the VCN.

[0106] Communication between two compute instances on the same subnet within a VCN is facilitated using VNICs associated with the source and destination compute instances. The source and destination compute instances may be hosted by the same host machine or different host machines. A packet originating from a source compute instance may be forwarded from the host machine hosting the source compute instance to an NVD connected to that host machine. In the NVD, the packet is processed using a packet processing pipeline, which may include the execution of a VNIC associated with the source compute instance. Because the packet's destination endpoint is in the same subnet, the execution of a VNIC associated with the source compute instance forwards the packet to an NVD running a VNIC associated with the destination compute instance, which processes the packet and forwards it to the destination compute instance. The VNICs associated with the source and destination compute instances may run on the same NVD (e.g., if both the source and destination compute instances are hosted by the same host machine) or on different NVDs (e.g., if the source and destination compute instances are hosted by different host machines connected to different NVDs). The VNIC can use the routing / forwarding tables stored by the NVD to determine the next hop for a packet.

[0107] When communicating a packet from a compute instance in a subnet to an endpoint in a different subnet within the same VCN, the packet originating from the source compute instance is communicated from the host machine hosting the source compute instance to the NVD connected to that host machine. In the NVD, the packet is processed using a packet processing pipeline that may include running one or more VNICs and VRs associated with the VCN. For example, the NVD executes or invokes a function corresponding to a VNIC associated with the source compute instance (also referred to as executing a VNIC) as part of the packet processing pipeline. The function executed by the VNIC may include examining the VLAN tag on the packet. Because the packet's destination is outside the subnet, a VCN VR function is invoked and executed by the NVD. The VCN VR then routes the packet to the NVD executing the VNIC associated with the destination compute instance. The VNIC associated with the destination compute instance then processes the packet and forwards the packet to the destination compute instance. The VNICs associated with the source compute instance and the destination compute instance may run on the same NVD (e.g., if both the source compute instance and the destination compute instance are hosted by the same host machine) or may run on different NVDs (e.g., if the source compute instance and the destination compute instance are hosted by different host machines connected to different NVDs).

[0108] If the packet's destination is outside the VCN of the source compute instance, the packet originating from the source compute instance is forwarded to the host machine that hosts the source compute instance. The packet is communicated from the VCN to the NVD connected to its host machine. The NVD runs a VNIC associated with the source compute instance. Because the packet's destination endpoint is outside the VCN, the packet is processed by the VCN VR for that VCN. The NVD invokes a VCN VR function, which may result in the packet being forwarded to an NVD running the appropriate gateway associated with the VCN. For example, if the destination is an endpoint in the customer's on-premises network, the packet may be forwarded by the VCN VR to an NVD running a DRG gateway configured for the VCN. The VCN VR may run on the same NVD as the NVD running the VNIC associated with the source compute instance, or it may be run by a different NVD. The gateway may be run by an NVD that is a smart NIC, a host machine, or another NVD implementation. The packet is then processed by the gateway and forwarded to the next hop to facilitate communication of the packet to the intended destination endpoint. For example, in the embodiment shown in FIG. 2, a packet originating from compute instance 268 may be communicated from host machine 202 to NVD 210 via link 220 (using NIC 232). VNIC 276 on NVD 210 is called out because it is the VNIC associated with source compute instance 268. VNIC 276 is configured to inspect encapsulated information within a packet, determine a next hop for forwarding the packet in order to facilitate communication of the packet to an intended destination endpoint, and forward the packet to the determined next hop.

[0109] Compute instances deployed on a VCN can communicate with a variety of different endpoints. These endpoints may include endpoints hosted by CSPI 200 and endpoints external to CSPI 200. Endpoints hosted by CSPI 200 may include instances within the same VCN or other VCNs (which may be customer VCNs or VCNs not belonging to the customer). Communication between endpoints hosted by CSPI 200 may be performed over physical network 218. Compute instances can also communicate with endpoints not hosted by or external to CSPI 200. Examples of these endpoints include endpoints within a customer on-premises network or data center, or public endpoints accessible over a public network such as the Internet. Communication with endpoints external to CSPI 200 may be performed over a public network (e.g., the Internet) (not shown in FIG. 2) or a private network (not shown in FIG. 2) using various communication protocols.

[0110] The architecture of CSPI 200 shown in FIG. 2 is merely an example and is not intended to be limiting. Variations, substitutions, and modifications are possible in alternative embodiments. For example, in some implementations, CSPI 200 may have more or fewer systems or elements than those shown in FIG. 2, may combine two or more systems, or may have a different system configuration or arrangement. The systems, subsystems, and other elements shown in FIG. 2 may be implemented in software (e.g., code, instructions, programs) executed by one or more processing units (e.g., processors, cores) of the respective systems, hardware, or a combination thereof. Software may be stored in a non-transitory storage medium (e.g., a memory device).

[0111] 4 is a diagram illustrating connections between host machines and NVDs to provide I / O virtualization to support multi-tenancy, according to certain embodiments. As shown in FIG. 4, host machine 402 runs hypervisor 404, which provides a virtualized environment. Host machine 402 runs two virtual machine instances: VM1 406, which belongs to customer / tenant #1, and VM2 408, which belongs to customer / tenant #2. Host machine 402 includes a physical NIC 410 connected to an NVD 412 via link 414. Each of the compute instances is connected to a VNIC executed by NVD 412. In the embodiment of Figure 4, VM1 406 is connected to VNIC-VM1 420 and VM2 408 is connected to VNIC-VM2 422.

[0112] 4, NIC 410 includes two logical NICs: logical NIC A 416 and logical NIC B 418. Each virtual machine is connected to and configured to operate with its own logical NIC. For example, VM1 406 is connected to logical NIC A 416, and VM2 408 is connected to logical NIC B 418. The logical NICs allow each tenant's virtual machine to believe it owns its own host machine and NIC, even though host machine 402 consists of only one physical NIC 410 shared by multiple tenants.

[0113] In particular embodiments, each logical NIC is assigned its own VLAN ID. Thus, logical NIC A 416 for tenant #1 is assigned a particular VLAN ID, and logical NIC B 418 for tenant #2 is assigned a different VLAN ID. When a packet is communicated from VM1 406, the hypervisor attaches a tag assigned to tenant #1 to the packet before communicating the packet from host machine 402 to NVD 412 over link 414. Similarly, when a packet is communicated from VM2 408, the hypervisor attaches a tag assigned to tenant #2 to the packet before communicating the packet from host machine 402 to NVD 412 over link 414. Thus, a packet 424 communicated from host machine 402 to NVD 412 has an associated tag 426 that identifies the particular tenant and associated VM. When a packet 424 is received on the NVD from host machine 402, the tag 426 associated with the packet is used to determine whether the packet should be processed by VNIC-VM1 420 or VNIC-VM2 422. The packet is then processed by the corresponding VNIC. The configuration shown in Figure 4 allows each tenant's compute instance to believe it owns its own host machine and NIC. The configuration shown in Figure 4 provides I / O virtualization to support multi-tenancy.

[0114] FIG. 5 is a schematic block diagram illustrating a physical network 500 according to a particular embodiment. The embodiment illustrated in FIG. 5 is constructed as a Clos network. A Clos network is a particular type of network topology designed to provide connection redundancy while maintaining high bisection bandwidth and maximum resource utilization. A Clos network is a type of non-blocking, multi-stage or multi-layer switching network, and the number of stages or layers may be 2, 3, 4, 5, etc. The embodiment illustrated in FIG. 5 is a three-layer network, including layers 1, 2, and 3. TOR switch 504 represents a layer-0 switch in the Clos network. One or more NVDs are connected to the TOR switch. The layer-0 switch is also referred to as an edge device of the physical network. The layer-0 switch is connected to a layer-1 switch, also referred to as a leaf switch. In the embodiment illustrated in FIG. 5, “n” layer-0 TOR switches are connected to “n” layer-1 switches to form a pod. Each layer-0 switch in a pod is interconnected to all layer-1 switches in the pod, but switches between pods are not connected. In a specific implementation, the two pods are referred to as blocks. Each block is served by or connected to "n" layer-2 switches (also called spine switches). A physical network topology may include multiple blocks. The layer-2 switches are in turn connected to "n" layer-3 switches (also called super-spine switches). Communication of packets through the physical network 500 is typically performed using one or more layer-3 communication protocols. Typically, the physical network, excluding the TOR layer, All layers of the network are n-way redundant, thus achieving high availability. The physical network can be extended by specifying policies on pods and blocks to control the mutual visibility of switches in the physical network.

[0115] A characteristic of Clos networks is that the maximum hop count from one tier-0 switch to another tier-0 switch (or from an NVD connected to a tier-0 switch to another NVD connected to a tier-0 switch) is constant. For example, in a three-tier Clos network, a packet requires a maximum of seven hops to travel from one NVD to another. In this case, the source NVD and the target NVD are connected to the leaf layers of the Clos network. Similarly, in a four-tier Clos network, a packet requires a maximum of nine hops to travel from one NVD to another. In this case, the source NVD and the target NVD are connected to the leaf layers of the Clos network. Therefore, the Clos network architecture maintains constant overall network latency, which is important for intra- and inter-datacenter communications. Clos topologies are horizontally scalable and cost-effective. The network bandwidth / throughput capacity can be easily increased by adding more switches (e.g., more leaf switches and spine switches) at each tier and by increasing the number of links between switches in adjacent tiers.

[0116] In certain embodiments, each resource in the CSPI is assigned a unique identifier called a Cloud Identifier (CID). This identifier is included as part of the resource's information. This identifier can be used to manage the resource, for example, through a console or API. An exemplary syntax for a CID is as follows:

[0117] ocid1.<RESOURCE TYPE> . <realm>.[REGION].[FUTURE USE].<UNIQUE ID> is. During the ceremony, "ocid1" is a string that indicates the version of the CID.

[0118] "RESOURCE TYPE" is the type of resource (e.g., instance, volume, VCN) , subnet, user, group).

[0119] "REALM" represents the region where the resource resides. An example value is "c1" "c1" represents the government cloud region, "c2" represents the government cloud region, or "c3" represents the federal government cloud region. Each region can have its own domain name.

[0120] "REGION" represents the region that the resource belongs to. If a region does not apply to the resource, this part may be blank.

[0121] "FUTURE USE" indicates that the item is reserved for future use. "UNIQUE ID" is the unique ID part. This format is used for resources or servers. This may vary depending on the type of screw.

[0122] S2C service example 6 illustrates an example of an S2C service between private networks, in accordance with at least one embodiment. The S2C service may be provided by resources of a cloud infrastructure 650. The cloud infrastructure 650 may include a VCN deployed for the service provider and a VCN deployed for the customer. The service provider may operate a service provider private network 660 that includes the VCN 651 deployed by the cloud infrastructure 650. Optionally, the service provider private network 660 may be connected to an on-premise The cloud infrastructure 650 may also include a VCN 652 deployed by the cloud infrastructure 650. Optionally, the customer private network 670 may include an on-premises network 671. The service provider private network 660 may connect to the customer private network 670 via S2C resources 680. The VCN and on-premises network may be connected via FastConnect or over the Internet. The private network may be connected via a virtual private network (VPN) connection over a public network, such as an on-premises network. In addition to or instead of an on-premises network, the private network may include peer networks (e.g., other peer VCNs).

[0123] 6, there may be multiple service provider private networks each similar to service provider private network 660. Similarly, there may be multiple customer private networks each similar to customer private network 670. For each service provider-customer pair or each service provider private network-customer private network pair, S2C resources are deployed by cloud infrastructure 650, similar to S2C resource 680.

[0124] Referring to the pair of service provider private network 660 and customer private network 670, S2C resources 680 provide S2C services including, for example, translation services 682. Different types of translation services 682 are possible, including, for example, DNS services, RAC services, and Layer 7 (L7) application services. Generally, translation services support data addressing and address translation. Thus, data may be processed within either the service provider private network 660 or the customer private network 670, or may flow between the service provider private network 660 and the customer private network 670.

[0125] For example, a service resource 662 in a service provider private network 660 may send an access request 664 to access data at a target resource 672 in a customer private network 670. The source and destination IP addresses of the access request 664 are usable within the environment of the service provider private network 660 but are not usable outside of it. The access request 664 may be received by an S2C resource 680 over a connection supported by the cloud infrastructure 650. A translation service 682 may generate a translated request 684 by updating the source and destination IP addresses of the access request 664 with IP addresses that are usable within the environment of the customer private network 670 but are not usable outside of it. The translated request 684 may be sent to the target resource 672 over a connection supported by the cloud infrastructure 650.

[0126] Similarly, the target resource 672 can send an access response 674 containing data from the target resource 672. The source and destination IP addresses of the access response 674 are usable within the environment of the customer private network 670, but not outside of it. This access response 674 may be received by the S2C resource 680 over a connection supported by the cloud infrastructure 650. The translation service 682 updates the source and destination IP addresses of the access response 674 with IP addresses that are usable within the environment of the service provider private network 660, but not outside of it. 650. The cloud infrastructure 650 may then transmit the transformed response 686 to the service resources 662 via a connection supported by the cloud infrastructure 650.

[0127] In one example, service provider private network 660 may include a set of service resources supporting a range of services, such as analytics, data processing, security services, etc. In a VCN 651-only implementation, the set of service resources may be hosted on VCN 651 and may include, for example, compute instances (e.g., virtual machines, bare metal machines). In an implementation that also includes on-premises network 661, the set of service resources may be hosted entirely on premises network 661, entirely on VCN 651, or distributed between them.

[0128] A customer can subscribe to one or more services. When subscribing to a service, the corresponding service resources in service provider private network 660 may require S2C access to one or more target resources in customer private network 670. In a VCN 652-only implementation, the customer's set of target resources may be hosted on VCN 652 and may include, for example, compute instances (e.g., virtual machines, bare metal machines) and storage instances (e.g., databases). In an implementation that also includes on-premises network 671, the set of target resources may be hosted entirely on premises network 671, entirely on VCN 652, or distributed between them.

[0129] Generally, a VCN, such as VCN 651 or VCN 652, may be a software-defined virtual version of a traditional network, including subnets, route tables, and gateways. Various compute instances can run on a VCN. For example, a VCN is a virtual private network that an entity (e.g., a service provider or a customer) builds within cloud infrastructure 650. Cloud infrastructure 650 may be located in a specific region. Before launching compute instances, at least one cloud infrastructure must be built. Cloud infrastructure 650 may optionally include an internet gateway for handling public traffic, a virtual private network (VPN) connection, or OCI FastConnect for securely extending a customer's on-premises network. It may be configured as follows.

[0130] A VCN may be privately connected to another VCN so that traffic does not traverse the Internet. In this case, the Classless Inter-Domain Routing (CIDR) of the two VCNs do not overlap. However, if CIDRs overlap, either between a service provider's VCN and a customer's VCN or between multiple VCNs of a service provider's customers, the private connection may not be usable, at least for S2C access. For example, VCN 651 and VCN 652 may each have a contiguous range of IP version 4 (IPv4) or version 6 (IPv6) addresses that do not overlap with other subnets within the VCN. In this case, S2C resources may be deployed.

[0131] 7 illustrates an example of a service provider private network 710 connected to a customer private network 720 via a set of S2C resources 730 supporting DNS traffic and proxy-less S2 traffic, in accordance with at least one embodiment. The service provider private network 710, customer private network 720, and S2C resources 730 are connected to the service provider private network 660, customer private network 670, and S2C resources 730, respectively. 7 is an example of a DNS resource 680. DNS queries originating from the service provider private network 710 and associated with a customer's DNS can be processed by an S2C resource 730 to provide DNS resolution to the service provider private network 710. Once resolved, a network address translation (NAT) mapping is created and traffic can flow between the service provider private network 710 and the customer private network 720 through a set of S2C resources 730 based on the NAT mapping.

[0132] As shown, service provider private network 710 includes service resources 712, a service provider S2C interface 714, and a service provider DNS resolver 716. The service provider S2C interface 714 is connected to the service resources 712 and the service provider DNS resolver 716, allowing the service resources 712 and the service provider DNS resolver 716 to interface with a set of S2C resources 730. Similarly, customer private network 720 includes target resources 722, a customer S2C interface 724, and a customer DNS resolver 726. The customer S2C interface 724 is connected to the target resources 722 and the customer DNS resolver 726, allowing the target resources 722 and the customer DNS resolver 726 to interface with the set of S2C resources 730.

[0133] Service resources 712, such as compute instances, provide services to customers. The services require access to target resources 722, such as private endpoints. In one example, a customer can define a fully qualified domain name (FQDN) for accessing the target resource 722 (e.g., www.exampleFQDN1.com). When initially accessing the target resource 722, the service resource 712 can send traffic to the target resource 722 using the FQDN rather than the corresponding IP address. Thus, the service resource 712 can initiate a DNS query that includes the FQDN. The service provider S2C interface 714 receives the DNS query and sends it to a set of S2C resources 730. The set of S2C resources 730 includes a DNS proxy. The DNS proxy 732 determines a match between the FQDN and a list of FQDNs predefined by the customer. Based on the match, the DNS proxy 732 sends the DNS query to the customer S2C interface 724, which sends the DNS query to the customer DNS resolver 726. In response, customer DNS resolver 726 resolves the DNS query by determining the IP address of the target resource 722. This IP address corresponds to the FQDN and is available within customer private network 720 (e.g., defined as an address within the IP address range of customer private network 720). The DNS response, including the IP address of the target resource 722, is sent back to DNS proxy 732 via customer S2C interface 724.

[0134] Because the IP address of the target resource 722 in the DNS response is usable only in the environment of the customer private network 720 and not in the environment of the service provider private network 710, the DNS proxy generates a reserved IP address and associates it with the IP address of the target resource 722. The reserved IP address may be defined as an address outside the IP address range of the customer private network 720 and outside the IP address range of the service provider private network 710. This association may be stored in a NAT mapping 734 between the reserved IP address and the IP address of the target resource 722. This NAT mapping 734 is maintained by a set of S2C resources 730.

[0135] The DNS proxy then updates the DNS response by replacing at least the IP address of the target resource 722 with the reserved IP address and sends the updated DNS response to the service resource 712 via the service provider S2C interface. When the service resource 712 receives this DNS response and determines that the FQDN is associated with the reserved IP address, it can cache this association for subsequent use.

[0136] Traffic from the service resource 712 to the target resource 722 and responses thereto depend on the reserved IP address and the NAT mapping 734. For example, the service resource 712 can send data to the reserved IP address. The flow of data may be proxied through the DNS proxy 732. In this case, address translation is performed by the DNS proxy 732 according to the NAT mapping 734. Additionally or alternatively, the flow of data may not be proxied through the DNS proxy 732. In this case, the flow of data may go through a gateway 736 of the set of S2C resources 730. For example, the gateway 736 may be a private gateway (PAGW) built between the service provider private network 710 and the customer private network 720. In this case, the gateway 736 also performs address translation depending on the NAT mapping 734. In both cases, at least the IP destination address is updated by replacing the reserved IP address with the IP address of the target resource 722.

[0137] A reverse flow is performed for the response from the target resource 722. For example, the response data includes the IP address of the target resource 722 as the source address. The DNS proxy 732 (if the flow is proxied) or the gateway 736 (if the flow is not proxied) updates the response data by replacing at least the IP address of the target resource 722 with the reserved IP address as the source address and sends the updated data to the service resource 712.

[0138] In another example, service resource 712 generates and sends a DNS query to DNS proxy 732 via service provider S2C interface 714. This DNS query includes a different FQDN (e.g., www.exampleFQDN2.com) than those described above. In this example, DNS proxy 732 determines that no match exists between the FQDN and the customer-defined FQDN list. The mismatch may be assumed to indicate that the FQDN corresponds to a resource in service provider private network 710 and therefore can be resolved by service provider DNS resolver 716. Thus, DNS proxy 732 sends the DNS query to service provider DNS resolver 716 via service provider S2C interface 714. Upon resolution, service provider DNS resolver 716 determines an IP address that corresponds to the FQDN and is usable within service provider private network 710 (e.g., defined as an address within the IP address range of service provider private network 710). The service provider DNS resolver 716 sends the service provider's IP address to the service resource 712 via the service provider S2C interface 714 for subsequent use by the service resource 712 .

[0139] 8 illustrates an example of a service provider private network 810 connected to a customer private network 820 via a set of S2C resources 830 supporting RAC traffic, in accordance with at least one embodiment. The service provider private network 810, the customer private network 820, and the S2C resources 830 are connected to the service provider private network 660, the customer private network 820, and the S2C resources 830, respectively. 8 is an example of a private network 670 and S2C resources 180. RAC traffic may originate from a service resource 812 in a service provider private network 810 and be sent to a database resource 822 in a customer private network 820. Similarly, RAC responses may originate from the database resource 822 and be sent to the service resource 812. The set of S2C resources 830 may include an address translation service 832 that translates IP addresses to enable the exchange of RAC traffic and RAC responses. In this case, NAT mapping may not be required. Instead, address translation may rely on knowledge, obtained by the address translation service 832, of the IP address of the service provider S2C interface 814 in the service provider private network 810 and the IP address of the customer S2C interface 824 in the customer private network 820. This knowledge may be part of the configuration used to deploy the set of S2C resources 830 for the service provider private network 810 and customer private network 820 pair.

[0140] In one example, the IP addresses of the service provider S2C interface 814 and the customer S2C interface 824 are defined in a configuration file for a set of S2C resources, and the IP addresses of the service resources 812 and the database resources 822 may be defined in a configuration file or may be learned over time based on traffic routing through the set of S2C resources 830.

[0141] The service resource 812 can send RAC data to the service provider S2C interface 814. This data includes the IP address of the service resource 812 as the source address and the IP address of the service provider S2C interface 814 as the destination address. The service provider S2C interface 814 then sends this data to the address translation service 832. The address translation service 832 updates the RAC data by replacing at least the source IP address and the destination IP address with the IP address of the customer S2C interface 824 and the IP address of the database resource 822, respectively. The updated RAC data is sent to the database resource 822 via the customer S2C interface 824.

[0142] The reverse flow is performed for responses returned from the database resources 822. For example, the response data includes the IP address of the database resources 822 as the source address and the IP address of the customer S2C interface 824 as the destination address. In this case, the address translation service 832 updates the data by replacing at least the source IP address and the destination IP address with the IP address of the service provider S2C interface 814 and the IP address of the service resources 812, respectively. The updated data is sent to the service resources 812 via the service provider S2C interface 814.

[0143] 9-11 illustrate example network components of a service provider private network, a customer private network, and a cloud infrastructure and the connections therebetween, which are exemplary implementations of some of the components described above in this specification.

[0144] 9 illustrates an example of components of a service provider private network 910, in accordance with at least one embodiment. The service provider private network 910 is a network of service provider private networks 660, 210, and 810. For example, at least a portion of the service provider private network 910, e.g., a VCN, is implemented on a cloud infrastructure.

[0145] Service provider private network 910 is configured by the service provider to have a set of resources, each having an IP address within an IP address range (e.g., 10.0.0.0 / 16) of service provider private network 910. The set of resources includes, for example, service instance 912, service instance 916, and DNS resolver 918. The two service instances 912 and 916 can provide different types of services that customers can subscribe to, and these services rely on S2C access.

[0146] In one example, service instance 912 is a compute instance that has an IP address (e.g., 10.0.2.2) and supports DNS queries and a particular communication protocol (e.g., an application layer protocol such as Hypertext Transfer Protocol Secure (HTTPS)). In contrast, service instance 916 is also a compute instance, but has a different IP address (e.g., 10.0.2.3) and supports a database access protocol such as the Single Client Access Name (SCAN) protocol in a RAC environment. DNS resolver 918 has a different IP address (e.g., 10.0.55.55) and supports DNS resolution of DNS queries related to domains of service provider private network 910 (e.g., related to FQDNs with IP addresses within the IP address range of service provider private network 910). Although not shown, service instance 912, service instance 916, and DNS resolver 918 may each have one or more VNICs that communicate with resources of service provider private network 910.

[0147] When a customer subscribes to one or more services of a service provider, a corresponding set of S2C resources is activated to support the service. Furthermore, an S2C interface 914 is connected to the service provider private network 910 (e.g., a host on this network). The S2C interface 914 has many different IP addresses (FIG. 9 shows the number as a positive integer "n"). Each IP address is within the IP address range of the service provider private network 910 and corresponds to one service. For example, when performing a DNS function, the S2C interface 914 has a first IP address (shown as "IP1" in FIG. 9) and uses the first IP address to send and receive DNS traffic 901. Similarly, when performing a RAC function, the S2C interface 914 has a second IP address (shown as "IP2" in FIG. 9) and uses the second IP address to send and receive RAC traffic 905.

[0148] As further shown in the following figure, the cloud infrastructure may include dedicated virtualization appliances that provide network interface functions as a service. In such an implementation, the S2C interface 914 may be a VNIC hosted on the virtualization appliances. The IP addresses (e.g., “IP1,” “IP2,” ..., “IPn”) of the S2C interface 914 may be defined in a configuration file based on input from the service provider. The S2C interface 914 may be referred to as a floating service (FS) VNIC because it has the ability to support multiple IP addresses that belong to the IP address range of the service provider private network 910 and correspond to different types of services.

[0149] In the example of DNS traffic 901, service instance 912 sends any DNS queries to the first IP address (e.g., “IP1”) of the S2C interface. The service instance 916 is configured to send any RAC traffic to a second IP address (e.g., “IP2”) of the S2C interface and receive DNS resolutions from this IP address. Similarly, DNS queries that can be resolved via the DNS resolver 918 are sent using the first IP address. In contrast, to support RAC traffic 905, the service instance 916 is configured to send any RAC traffic to a second IP address (e.g., “IP2”) of the S2C interface and receive RAC responses from the second IP address (e.g., “IP2”) of the S2C interface.

[0150] 10 illustrates an example of components of a customer private network 1010, in accordance with at least one embodiment. Customer private network 1010 is an example of customer private networks 670, 720, and 820. At least a portion of customer private network 1010, e.g., a VCN, is implemented on a cloud infrastructure.

[0151] Customer private network 1010 is configured by a customer to have a set of resources, each having an IP address within the IP address range of customer private network 1010 (e.g., 10.0.0.0 / 16). Referring again to FIG. 9 , the IP address range of customer private network 1010 may overlap with the IP address range of service provider private network 910. The set of resources includes, for example, customer endpoint 1014, DNS resolver 1016, and database service 1018. The customer can subscribe to the service provider's services, such as those supported by service instances 912 and 916, and therefore requires S2C access to customer endpoint 1014 and database service 1018.

[0152] Customer endpoint 1014 represents a target resource, e.g., a customer VCN or customer on-premises network, associated with a service of a service provider and having an IP address (10.0.10.2) within the IP address range of customer private network 1010. In one example, customer endpoint 1014 has a private IP address and is configured to facilitate access to the service provider's service only through cloud infrastructure facilities. In this example, a private endpoint (e.g., a VNIC with a private IP address) exposes the target resource to the service. Database service 1018 represents a resource that has an IP address (10.0.10.10) within the IP address range of customer private network 1010 and provides a service for managing workloads within a customer's database. The database may be within the customer's VCN and / or on-premises network. Additionally, the DNS resolver 1016 has a further different IP address (e.g., 10.33.33) and supports DNS resolution of DNS queries associated with domains of the customer private network 1010 (e.g., associated with FQDNs with IP addresses within the IP address range of the customer private network 1010). Although not shown, the customer endpoints 1014, the DNS resolver 1016, and the database service 1018 may each have one or more VNICs that communicate with resources in the customer private network 1010.

[0153] When a customer subscribes to one or more of their services, a corresponding set of S2C resources is activated to support the services. Furthermore, the S2C interface 1012 is connected to the customer private network 1010 (e.g., hosts on this network). The S2C interface 1012 has many different IP addresses (FIG. 10 shows the number as a positive integer "n"). Each IP address is within the IP address range of the customer private network 1010 and corresponds to one service. For example, for non-proxied S2C traffic, the first IP address on the S2C interface 1012 is For DNS and RAC traffic, a second IP address (IP2) of the S2C interface 1012 is available. For Customer-to-Service Provider (C2S) traffic, yet another IP address (e.g., IPn) of the S2C interface 1012 is available.

[0154] As further shown in the following figure, the cloud infrastructure may include dedicated virtualization appliances that provide network interface functions as a service. In such an implementation, the S2C interface 1012 may be a VNIC hosted on the virtualization appliances. The IP addresses (e.g., "IP1," "IP2," ..., "IPn") of the S2C interface 1012 may be defined in a configuration file based on customer input. Because of its ability to support multiple IP addresses that belong to the IP address range of the customer private network 1010 and correspond to different types of services, the S2C interface 1012 may be referred to as a FS VNIC.

[0155] FIG. 11 illustrates an example of components of a cloud infrastructure 1100 that implements S2C resources, in accordance with at least one embodiment. Cloud infrastructure 1100 is an example of cloud infrastructure 150 of FIG. 1. As shown, the components include a gateway 1110, virtualization appliances 1120, DNS resources 1130, NAT mapping data store 1140, and RAC resources 1150. Generally, these components enable S2C access from a service provider private network to a customer private network. While virtualization appliances 1120 are located in the physical network of cloud infrastructure 1100, gateway 1110, DNS resources 1130, RAC resources 1150, and NAT mapping data store 1140 may be located in virtual networks (e.g., overlay networks) on the physical network.

[0156] In one example, the gateway 1110 facilitates traffic between a service provider private network and a customer private network. The gateway 1110 may be configured as a service gateway that allows the customer's VCN to privately access specific services offered through the service provider's VCN without exposing the data to the public Internet. In this case, the gateway 1110 may be referred to as a private gateway. As described above, S2C access may rely on reserved IP addresses (e.g., Class E IP addresses). For data from the service provider private network to the customer private network, the gateway 1110 can store a NAT mapping 1112 (e.g., destination NAT-DNAT) that maps the reserved IP address to an IP address in the customer private network. For data in the reverse direction, the NAT mapping 1112 maps the IP address of the customer's S2C interface (e.g., "IP1" of the FS VNIC shown in FIG. 10) to a service resource in the service provider computer network.

[0157] The virtualization appliance cluster 1120 includes a group of dedicated virtualization appliances that provide virtual network interfaces. VNICs may reside within subnets and be connected to compute instances. Although not shown in FIG. 11 , the virtualization appliance cluster 1120 provides FS VNICs that are connected to a service provider private network (e.g., a service provider S2C interface) and a customer private network (e.g., a customer S2C interface). Additionally, the virtualization appliance cluster 1120 provides VNICs that are connected to S2C resources, such as DNS resources 1130 and RAC resources 1150. These VNICs are subject to security list processing, A lightweight version that does not support route rule processing or peer mapping, may be called a micro VNIC (μVNIC).

[0158] DNS resources 1130 represent S2C resources that support DNS functions, including DNS reconciliation (e.g., determining whether a queried FQDN is a customer FQDN or a service provider FQDN), DNS resolution (e.g., sending a DNS query to a customer DNS resolver or a service provider DNS resolver, depending on the reconciliation), and NAT mapping of IP addresses. In one example, DNS resources 1130 are deployed when a customer subscribes to a service provider's service, and the subscription includes S2C DNS traffic from the service provider private network and the customer private network. DNS resources 1130 include a service provider-facing (SPF) S2C interface 1132, a customer-facing (CF) S2C interface 1134, and a DNS proxy 1136. Each of the SPF S2C interface 1132 and the CF S2C interface 1134 may be a μVNIC provided by the virtualization appliance cluster 1120. The SPF S2C interface 1132 is configured to communicate with the service provider's FS VNIC. In turn, CF S2C interface 1134 is configured to communicate with the customer's FS VNIC. DNS proxy 1136 receives DNS queries via SPF S2C interface 1132, performs reconciliation, and, as appropriate, sends the DNS queries to the customer's DNS resolver via CF S2C interface 1132 or to the service provider's DNS resolver via SPF S2C interface 1132. Also, upon receiving a DNS response from the customer's DNS resolver via CF S2C interface 1134, the DNS proxy maps the customer's IP address from the DNS response to a reserved IP address, stores this mapping in NAT mapping data store 1140, modifies the query response to include the reserved IP address, and sends the modified response to the service provider private network via SPF S2C interface 1132.

[0159] The NAT mapping data store 1140 can maintain different mappings (e.g., DNAT mappings) between reserved IP addresses and customer IP addresses on a regional basis, and the gateway 1110 can retrieve and store such mappings as part of the DNAT mappings of the NAT mappings 1112.

[0160] RAC resource 1150 represents an S2C resource that supports RAC functionality, including address translation services. In one example, RAC resource 1150 is deployed when a customer subscribes to a service provider's service, and the subscription includes S2C RAC traffic from the service provider private network and the customer private network. RAC resource 1150 includes an SPF S2C interface 1152, a CF S2C interface 1154, and a RAC proxy 1156. Each of SPF S2C interface 1152 and CF S2C interface 1154 may be a μVNIC provided by virtualization appliance cluster 1120. SPF S2C interface 1152 is configured to communicate with the service provider's FS VNIC. In contrast, CF S2C interface 1154 is configured to communicate with the customer's FS VNIC. RAC proxy 1156 receives RAC traffic from the service provider private network, readdresses the traffic to use IP addresses in the customer private network, and sends the updated RAC traffic via CF S2C 1154. For the reverse flow, the RAC proxy 1156 receives the RAC return from the customer private network, readdresses the return to use an IP address in the service private network, and sends the updated RAC return via SPF S2C 1152.

[0161] In one example, to achieve high availability, multiple redundant S2C resources (e.g., three sets of DNS resources 1130) are launched for each service provider-customer pair. Upon receiving a packet to be sent to an S2C resource, the virtualization cluster 1120 can select one of the multiple S2C resources to which the packet is sent. Different types of selection are possible. For example, the S2C resource can be selected randomly. In another example, load balancing is considered by tracking the number of packets processed per S2C resource. In yet another example, the selection can depend on the packet itself. For example, the packet includes protocol information, a source IP address, a source port, a destination IP address, and a destination port. This five-tuple represents the signature of the packet. The signature can be hashed. The hash may be compared to multiple hash ranges, each corresponding to one S2C resource. If the hash matches the hash range of an S2C resource, that S2C resource is selected and the packet is sent to that S2C resource.

[0162] The flow of data through the various components of cloud infrastructure 150 is further described in Figures 12-14. Examples of DNS queries, no-proxy traffic, and RAC traffic are provided for illustrative purposes. For clarity of explanation, reference is also made to the components described in Figures 9-6.

[0163] 12 illustrates an example of DNS traffic, according to at least one embodiment. A service instance 912 initiates a DNS query 1210 for a particular FQDN belonging to a customer's DNS. Thus, a DNS request 1212 is sent from the service instance 912 to the service provider's S2C interface 914. In one example, the DNS request 1212 is a packet that includes a header and a payload. The source address of the header is the service instance's IP address (e.g., 10.0.2.2). The destination address of the header is the S2C interface's IP address (e.g., "IP1" shown as 10.0.3.11). This IP address is unique to the DNS query. The payload includes the FQDN.

[0164] The DNS request 121 is then sent from the service provider private network 910 to the DNS resource 1130. At a logical level, the S2C interface 914 sends the DNS request 1212 to the SPF S2C interface 1132. At a physical level, the DNS request 1212 is routed through the virtualization appliance cluster 1120.

[0165] The DNS proxy 1136 receives the DNS request 121 and determines a match between the FQDN and the customer-defined FQDN list. This determination may include a string suffix match. The DNS proxy 1136 then updates the DNS request 121 by modifying the header information. For example, the source address is changed to the IP address of the customer's S2C interface 1012 (e.g., "IP2" shown as 10.0.2.22), which is configured to receive DNS queries. Similarly, the destination address is changed to the IP address of the customer's DNS resolver 1016 (e.g., 10.33.33). The payload remains the same and includes the FQDN. The resulting DNS request 1222 is sent to the customer private network 1010. At a logical level, the CF S2C interface 1134 sends the DNS request 1222 to the customer's S2C interface 1012. At a physical level, the DNS request 1222 is routed through the virtualization appliance cluster 1120.

[0166] The customer's S2C interface 1012 receives the DNS request 1222 and sends it to the customer's DNS resolver 1016 based on the header. The customer's DNS resolver 1016 then resolves the FQDN to a customer IP address (e.g., 10.0.10.2, which corresponds to the customer endpoint 1014) and returns a DNS response 1232. In one example, the DNS The S response 1232 is a packet that includes a header and a payload. The source address in the header is the IP address of the DNS resolver (e.g., 10.33.33). The destination address in the header is the IP address of the S2C interface 1012 (e.g., "IP2" shown as 10.0.2.22), which is specific to the DNS query. The payload includes the customer IP address (e.g., 10.0.10.2).

[0167] The customer's S2C interface 1012 receives the DNS response 1232 and sends it to the DNS proxy 1136 via the CF S2C interface 1134. The DNS proxy 1136 maps the custom IP address from the payload to a reserved IP address (254.0.2.35) and stores the resulting NAT mapping 1224 in the NAT mapping data store 1140. The DNS proxy 1136 also updates the DNS response 1232 by modifying header information. For example, the source address is changed to the IP address of the service's S2C interface 914 (e.g., "IP1," shown as 10.0.3.11), which is configured to receive DNS queries. Similarly, the destination address is changed to the IP address of the service instance 912 (e.g., 10.0.2.2). The payload includes the reserved IP address (e.g., 254.0.2.35). The resulting DNS response 1226 is sent via the SPF S2C interface 1132 to the service provider's S2C interface 914. The service provider's S2C interface 914 sends the DNS response 1226 to the service instance 912 based on the header information.

[0168] 13 illustrates an example of proxy-less traffic, in accordance with at least one embodiment. In this case, service instance 912 has already performed a DNS query to resolve the customer's FQDN to a reserved IP address (e.g., 254.0.2.35). Service instance 912 sends S2C traffic 1312 to a gateway. In one example, S2C traffic 1312 includes a packet whose header includes the service instance's IP address (10.0.2.2) as the source address and the reserved IP address as the destination address.

[0169] Next, the gateway 1110 looks up the NAT mapping 1112 and determines from the DNAT mapping that the reserved IP address is mapped to a customer IP address (e.g., 10.0.10.2) and from the SNAT mapping that the service instance IP address is mapped to the customer's S2C interface IP address (e.g., "IP1" shown as 10.0.2.21) for non-proxy traffic. The gateway 1110 updates the S2C traffic 1312 by modifying the header information. For example, the source address is changed to the customer's S2C interface IP address (e.g., 10.0.2.21). Similarly, the destination address is changed to the customer IP address (e.g., 10.0.10.2). The resulting S2C traffic 1322 is sent to the customer's S2C interface 1012. At the physical level, the S2C traffic 1322 is routed through the virtualization appliance cluster 1120.

[0170] The customer's S2C interface 1012 receives the S2C traffic 1322 and transmits it to the customer endpoint 1014 based on the header information. In response, the customer endpoint 1014 generates and transmits an S2C return 1332. In one example, the S2C return includes a packet whose header includes the customer endpoint's IP address (e.g., 10.0.10.2) as the source address and the IP address of the S2C customer's S2C interface 1012 (e.g., 10.0.2.21).

[0171] The customer's S2C interface 1012 then sends the S2C response 1332 to the gateway 1110. The gateway updates the S2C return based on the NAT mapping 1112. For example, the source address is changed to a reserved IP address (e.g., 254.0.2.35) and the destination address is changed to the IP address of the service instance 912 (e.g., 10.0.2.2). The resulting S2C return 1324 is sent to the service instance 912.

[0172] FIG. 14 illustrates another example of DNS traffic, in accordance with at least one embodiment. In this case, the DNS query is for an FQDN that belongs to the service provider's domain name, rather than the customer's. The service instance 912 initiates a DNS query 1410 for the FQDN. Thus, a DNS request 1412 is sent from the service instance 912 to the service provider's S2C interface 914. In one example, the DNS request 1412 is a packet that includes a header and a payload. The source address of the header is the service instance's IP address (e.g., 10.0.2.2). The destination address of the header is the IP address of the S2C interface 914 (e.g., "IP1" shown as 10.0.3.11), which is specific to the DNS query. The payload includes the FQDN.

[0173] The DNS request 1412 is then sent from the service provider private network 910 to the DNS resource 1130. At a logical level, the S2C interface 914 sends the DNS request 1412 to the SPF S2C interface 1132. At a physical level, the DNS request 1412 is routed through the virtualization appliance cluster 1120.

[0174] The DNS proxy 1136 receives the DNS request 1412 and determines that there is no match between the FQDN and the customer-defined FQDN list. This determination may include a string suffix match. Thus, the DNS proxy 1136 defaults to determining that the FQDN belongs to the service provider's DNS. The DNS proxy 1136 readdresses the DNS request 1412 by changing the source address to the IP address of the S2C interface 914 (e.g., 10.0.3.11) and the destination address to the service provider's DNS resolver 918 (e.g., 10.0.55.55). The resulting query request 1422 is sent back to the S2C interface 914, which sends the query request 1422 to the DNS resolver 918. The DNS resolver 918 performs DNS resolution and, once it determines the service provider IP address corresponding to the FQDN, sends the DNS response to the S2C interface 914. The DNS response 1414 may be a packet having a header and a payload. The source address of the header is the IP address of the DNS resolver 918 (e.g., 10.0.55.55). The destination address of the header is the IP address of the S2C interface 914 (e.g., 10.0.3.11). The payload includes the service provider IP address. The S2C interface 914 then readdresses the DNS response 1414 by changing the source address to the IP address of the S2C interface 914 (e.g., 10.0.3.11) and the destination address to the service instance 912 (e.g., 10.0.2.2). The resulting query response 1416 is sent to the service instance 912.

[0175] 15 illustrates an example of RAC traffic, in accordance with at least one embodiment. As shown, a service instance 916 generates and transmits RAC traffic 1512. The RAC traffic 1512 may include packets having a header and a payload. The source address of the header may be the IP address (e.g., The destination address in the header is the IP address of the S2C interface 914 (e.g., "IP2" shown as 10.0.3.12), which is configured for RAC traffic. The payload contains the database query.

[0176] The RAC traffic 1512 is then sent from the service provider private network 910 to the RAC resource 1150. At a logical level, the S2C interface 914 sends the RAC traffic 1512 to the SPF S2C interface 1152. At a physical level, the RAC traffic 1512 is routed through the virtualization appliance cluster 1120.

[0177] The RAC proxy 1156 receives the RAC traffic 1512 and readdresses it for forwarding to the customer private network. For example, the source IP address is changed to the IP address of the customer's S2C interface 1012 (e.g., "IP2" shown as 10.0.2.22), which is configured for RAC traffic. The destination IP address is changed to the IP address of the database service 1018 (e.g., 10.0.10.10). The payload is unchanged. The resulting S2C traffic 1522 is sent to the customer's S2C interface 1012 via the CF S2C interface 1154.

[0178] The customer's S2C interface 1012 receives the S2C traffic 1522 and sends it to the database service 1018 based on the header. The database service 1018 then generates and sends a RAC return 1532 based on the payload. The RAC return 1532 may include a packet with a header and a payload. The source address of the header is the IP address of the database service (e.g., 10.0.10.10). The destination address of the header is the IP address of the customer's S2C interface 1012 (e.g., 10.0.2.22). The payload is the result of the database query. The customer's S2C interface 1012 then sends the RAC return 1532 to the RAC proxy 1156 via the CF S2C interface 1154.

[0179] The RAC proxy 1156 receives the RAC return 1532 and readdresses it for forwarding to the service provider private network. For example, the source IP address is changed to the IP address of the service provider's S2C interface 914 (e.g., "IP2" shown as 10.0.3.12), which is configured for RAC traffic. The destination IP address is changed to the IP address of the service instance (e.g., 10.0.2.3). The payload is unchanged. The resulting RAC return 1524 is sent via the SPF S2C interface 1152 to the service provider's S2C interface 914. The service provider's S2C interface 914 sends the RAC return 1524 to the service instance 916 based on its header.

[0180] 16-21 illustrate an example of a flow for providing S2C access. Some or all of the instructions for performing the operations of the flow may be implemented as hardware circuits and / or stored as computer-readable instructions on a non-transitory computer-readable medium of a computer system, including, for example, a cloud infrastructure. The instructions represent modules containing circuitry or code that, when implemented, are executable by a processor of the computer system. Such instructions are used to configure the computer system to perform certain operations described herein. Each circuit or code in combination with an associated processor represents a means for performing the respective operation. The operations may be implemented as a specific Although shown in order, it should be understood that no particular order is required, and one or more operations may be omitted, skipped, performed in parallel, and / or the order of operations may be changed.

[0181] FIG. 16 illustrates an example flow for providing S2C services in accordance with at least one embodiment. The flow may begin at operation 1602. At operation 1602, service provider data is received and associated with the service provider's services. For example, the service provider may operate a service provider private network that includes a VCN. The VCN is an overlay of the cloud infrastructure. The service provider private network may support different types of services. A control plane of the cloud infrastructure may receive input from the service provider's equipment. The input may include various data for configuring the service, such as data for identifying the service, and data for setting the network configuration to provide the service, such as subnets and IP addresses to be used. At operation 1604, customer data is received and associated with the servers. For example, the customer may operate a customer private network that includes a VCN. The VCN is an overlay of the cloud infrastructure. The control plane of the cloud infrastructure may receive input from the user's equipment. The input may include various data for configuring the customer's service, such as data for identifying the service and data for configuring private endpoints through which the service can be accessed, such as the subnet, IP address, and FQDN to be used. In operation 1606, S2C resources may be deployed. For example, S2C resources are deployed specifically for a service-provider pair to support the services requested by the customer. This deployment may include launching redundant hosts that provide S2C services, such as redundant DNS resources and redundant RAC resources. S2C interfaces may also be deployed. In operation 1602, the service resource manages S2C access to the target resource. For example, the S2C resource manages S2C access based on the requested service. For DNS queries, the DNS resource may provide DNS proxy and NAT mapping functionality.For RAC queries, the RAC resource can provide address translation. For non-proxied traffic, the gateway can look up the NAT mapping generated by the DNS resource so that non-proxied traffic can be routed.

[0182] FIG. 17 illustrates an example flow for a service provider private network to support DNS traffic, non-proxy traffic, and RAC traffic, in accordance with at least one embodiment. The example flow may be performed by a service resource of the service provider private network. The flow may begin at operation 1702. At operation 1702, a service instance determines the FQDN of a customer endpoint that will receive the service provided by the service instance. At operation 1704, the service instance sends a DNS query with a first predetermined IP destination address. For example, the destination address is set to an IP address of the service provider's S2C interface (e.g., the first predetermined IP address). This IP address is configured for the DNS query. At operation 1706, the service instance receives a DNS query response. The DNS query response may be a packet that includes a payload indicating a DNS resolution. The payload may include, for example, a reserved IP address that has been mapped by the S2C DNS resource to the customer IP address of the customer endpoint. At operation 1708, the service instance sends the traffic to the customer endpoint. This traffic uses the reserved IP address as the destination address and the IP address of the service instance as the source address. The packet may include a packet that uses the reserved IP address as a source address. The packet may be sent to the gateway. At operation 1710, the service instance receives a traffic return. The traffic return is sent from the customer endpoint and received from the gateway and may include a packet. The packet uses the reserved IP address as a source address and the service instance's IP address as a destination address (e.g., a first predetermined IP address). At operation 1712, the same or a different service instance sends RAC traffic to the customer's database service. The RAC traffic may include a packet. The source address of the packet is the service instance's IP address. The destination address of the packet is a second predetermined IP address of the service provider's S2C interface. This IP address is configured for RAC traffic. At operation 1714, the service receives a RAC return. The RAC return is sent from the customer's database service and received from the S2C interface and may include a packet. The packet uses the second predetermined IP address of the S2C interface as a source address and the service instance's IP address as a destination address.

[0183] FIG. 18 illustrates an example flow for a customer private network to support DNS traffic, non-proxy traffic, and RAC traffic, in accordance with at least one embodiment. At operation 1802, the customer private network receives a DNS query with a first predetermined IP source address. For example, the DNS query includes a packet having a header and a payload. The payload includes an FQDN. The source address of the header is an IP address of the customer's S2C interface (e.g., the first predetermined IP address), which is configured for the DNS query. The destination address of the header is an IP address of the customer's DNS resolver. The DNS query may be received from an S2C DNS resource. The S2C interface sends the DNS query to the DNS resolver for resolution. At operation 1804, the customer private network sends a DNS response. The DNS response may include a packet. The payload of the packet includes the customer IP address associated with the FQDN. The source address in the header of this packet is the IP address of the DNS resolver, and the destination address in the header of this packet is the IP address of the customer's S2C interface (e.g., the first predetermined IP address). The S2C interface sends the DNS response to the S2C DNS resource. In operation 1806, the customer private network receives traffic for the customer endpoint. This traffic may include packets originating from the service instance and sent through a gateway. The destination address of this packet is the IP address of the customer endpoint. The source address of this packet is a second predetermined IP address of the customer's S2C interface, which is configured for non-proxy traffic. In operation 1808, the customer private network sends the traffic return. Similarly, the traffic return may include packets. The destination address of this packet is the second predetermined IP address. The source address of this packet is the IP address of the customer endpoint.At operation 1810, the customer private network receives RAC traffic to the database service. This traffic may include packets originating from a service instance and transmitted via an S2C RAC resource. The destination address of the packets is the IP address of the database service. The source address of the packets is a first predetermined IP address of the customer's S2C interface. At operation 1812, the customer private network transmits a RAC return. The RAC return may include a packet. The destination address of the packet is the first predetermined IP address. The source address of the packet is the IP address of the database service.

[0184] FIG. 19 illustrates an example flow for S2C resources to support DNS services, in accordance with at least one embodiment. The S2C resources may include, for example, a DNS resource. In one example, the flow may begin at operation 1902. At operation 1902, the DNS resource receives a DNS query. The DNS query may be sent from a service instance, including a packet, and may be received from a service provider's S2C interface. Thus, the destination address of the packet is a first predetermined IP address of the service provider's S2C interface, which is configured for the DNS query. The payload of the packet may include an FQDN. The source address of the packet is the IP address of the service instance. At operation 1904, the DNS resource may perform arbitration to determine whether the FQDN matches a customer-defined FQDN list. If so, operation 1910 is performed after operation 1904. If not, operation 1920 is performed. At operation 1910, the DNS resource sends a DNS query, including a payload, to a customer's DNS resolver. The DNS query may be a packet. The destination address of the packet may be the IP address of the DNS resolver. The source address of the packet may be a second predetermined IP address of the customer's S2C interface. This IP address is configured for the DNS query. The DNS query is sent to the DNS resolver via the customer's S2C interface. In operation 1912, the DNS resource receives a DNS response to the DNS query. The DNS response is sent from the customer's DNS resolver and received via the customer's S2C interface and includes a customer IP address of the customer endpoint associated with the FQDN. The DNS response may include a packet. The destination address of the packet is the second predetermined IP address of the customer's S2C interface. The source address is the IP address of the DNS resolver. In operation 1914, the DNS resource maps the customer IP address from the DNS response to a reserved IP address.At operation 1916, the DNS resource stores the NAT mapping in a NAT mapping data store. At operation 1918, the DNS resource sends a DNS response that includes the reserved IP address. For example, the DNS response is a packet. The payload of the packet includes the reserved IP address. The source address of the packet is a first predetermined IP address of the service provider's S2C interface. The destination address of the packet is the IP address of the service instance. The packet is sent to the service provider's S2C interface. At operation 1920, the DNS resource determines that the FQDN belongs to the service provider. Thus, the DNS resource sends a DNS query to the service provider's DNS resolver. The DNS query may include a packet with the FQDN in the payload. The source address of the packet is the first predetermined IP address of the service provider's S2C interface. The destination address of the packet is the IP address of the service provider's DNS resolver. The packet is sent to the service provider's S2C interface.

[0185] FIG. 20 illustrates an example flow for supporting proxy-less S2C traffic, in accordance with at least one embodiment. In one example, the flow can begin at operation 2002. In operation 2002, the gateway receives traffic with a reserved IP address. For example, the traffic includes a packet. The source address of the packet is the IP address of the service instance. The destination address of the packet is the reserved IP address. In operation 2004, the gateway determines the customer IP address of the customer endpoint. For example, the reserved IP address is used when looking up a NAT mapping that associates the reserved IP address with the customer IP address. In operation 2006, the gateway sends the traffic using the customer IP address. For example, by updating the packet header, the gateway changes the destination address from the reserved IP address to the customer IP address. The source address is also changed to the customer's S2C interface. The IP address may be updated to the IP address of the customer's S2C interface. This IP address is configured for proxy-less S2C traffic. The packet may be sent to the customer's S2C interface. In operation 2008, the gateway receives traffic return with the customer IP address. For example, the traffic return includes a packet. The source address of this packet is the customer IP address. The destination address of this packet is the IP address of the customer's S2C interface. This packet may be received from the customer's S2C interface. In operation 2010, the gateway determines the reserved IP address. For example, a reverse lookup of the NAT mapping is used. In operation 2014, the gateway sends the traffic return with the reserved IP address. For example, the traffic return includes a packet. The source address of this packet is the reserved IP address. The destination address of this packet is the IP address of the service instance. This packet is sent to the service instance.

[0186] FIG. 21 illustrates an example flow for an S2C resource to support a RAC service, according to at least one embodiment. In one example, the flow can begin at operation 2102. At operation 2102, the RAC resource receives RAC traffic. The RAC traffic may be transmitted from a service instance in a service provider private network and may include a packet. The source address of the packet is the IP address of the service instance. The destination address of the packet is a first predetermined IP address of the service provider's IP addresses. This IP address is configured for RAC traffic. The packet may be received from the service provider's S2C interface. At operation 2104, the RAC resource determines a second predetermined IP address of the customer's S2C interface. This IP address is configured for RAC traffic. At operation 2106, the RAC resource transmits the RAC traffic using the second predetermined IP address. For example, by updating the packet's header, the destination address is changed from the first predetermined IP address to the IP address of the customer's database service. The source address of the packet is changed from the IP address of the service instance to the second predetermined IP address. The packet may be sent to the customer's S2C interface. In operation 2108, the RAC resource receives a RAC return having a second predetermined IP address. For example, the RAC return includes a packet. The source address of the packet is the IP address of the database service. The destination address of the packet is the second predetermined IP address of the customer's S2C interface. The packet may be received from the customer's S2C interface. In operation 2110, the RAC service determines the first predetermined IP address as the address to which the RAC return should be sent. In operation 2114, the RAC service sends the RAC return having the first predetermined IP address. For example, the RAC return includes a packet. The source address of the packet is the first predetermined IP address.The destination address of this packet is the IP address of the service instance. The packet is sent to the service provider's S2C interface.

[0187] IaaS Architecture Examples As mentioned above, IaaS (Infrastructure as a Service) is a specific type of Cloud computing. IaaS may be configured to provide virtualized computing resources over a public network (e.g., the Internet). In the IaaS model, a cloud computing provider may host infrastructure elements (e.g., servers, storage devices, network nodes (e.g., hardware), deployment software, platform virtualization (e.g., hypervisor layer), etc.). In some cases, the IaaS provider may also provide various services (e.g., billing, monitoring, logging, security, load balancing, and other services) associated with the infrastructure elements. These services can therefore be policy-driven, allowing IaaS users to implement policies to drive load balancing to maintain application availability and performance.

[0188] In some examples, IaaS customers can access resources and services over a wide area network (WAN), such as the Internet, and use the cloud provider's services to install the remaining elements of their application stack. For example, a user can log into an IaaS platform to create virtual machines (VMs), install an operating system (OS) on each VM, deploy middleware such as databases, create storage buckets for workloads and backups, and install enterprise software on the VMs. Customers can use the provider's services to perform a variety of functions, including balancing network traffic, troubleshooting applications, monitoring performance, managing disaster recovery, and more.

[0189] In most cases, the cloud computing model requires the participation of a cloud provider, which can be, but does not have to be, a third-party service provider specializing in providing (e.g., offering, renting, or selling) IaaS. Alternatively, an enterprise can deploy a private cloud and become a provider of infrastructure services.

[0190] In some examples, IaaS deployment is the process of deploying a new application or a new version of an application onto a provisioned application server or the like. IaaS deployment may include the process of provisioning the server (e.g., installing libraries, daemons, etc.). IaaS deployment is often managed by the cloud provider below the hypervisor layer (e.g., server, storage, network hardware, and virtualization). Thus, customers can deploy the OS, middleware, and / or applications (e.g., self-service virtual machines (e.g., that can be spun up on demand)).

[0191] In some instances, IaaS provisioning may include obtaining the computer or virtual host to be used and installing the necessary libraries or services on the computer or virtual host. In most cases, deployment does not include provisioning, which must be performed first.

[0192] In some cases, IaaS provisioning presents two distinct challenges. First, there is the challenge of provisioning an initial set of infrastructure before anything can be done. Second, there is the challenge of evolving the existing infrastructure (e.g., adding new services, modifying services, removing services) after everything has been provisioned. In some cases, these two challenges can be addressed by allowing the configuration of the infrastructure to be defined declaratively. In other words, the infrastructure (e.g., which elements are needed and how these elements interact) may be defined by one or more configuration files. Thus, the overall topology of the infrastructure (e.g., which resources depend on which and how they work together) can be described declaratively. In some instances, once the topology is defined, workflows can be generated to create and / or manage the different elements described in the configuration files.

[0193] In some examples, the infrastructure may include many interconnected elements, such as one or more virtual private networks, also known as core networks. There may be a Virtual Private Cloud (VPC) (e.g., a potentially on-demand pool of configurable and / or shared computing resources). In some examples, there may be one or more security group rules and one or more virtual machines (VMs) provisioned to define how the network's security is configured. Other infrastructure elements, such as load balancers, databases, etc., may also be provisioned. The infrastructure can evolve incrementally as more and more infrastructure elements are desired and / or added.

[0194] In some examples, continuous deployment techniques may be employed to enable deployment of infrastructure code across various virtual computing environments. The described techniques may also enable infrastructure management within these environments. In some examples, a service team may write code that is desired to be deployed to one or more, typically many, different production environments (e.g., across a variety of different geographic locations, sometimes spanning the entire world). However, in some examples, the infrastructure for deploying the code must first be set up. In some examples, provisioning may be done manually, with provisioning tools used to provision resources and / or deployment tools used to deploy the code after the infrastructure has been provisioned.

[0195] 22 is a block diagram 2200 illustrating an example pattern of an IaaS architecture, according to at least one embodiment. A service operator 2202 may be communicatively connected to a secure host tenancy 2204, which may include a virtual cloud network (VCN) 2206 and a secure host subnet 2208. In some examples, the service operator 2202 may use one or more client computing devices. The one or more client computing devices may run software such as Microsoft Windows Mobile® and / or various mobile operating systems such as iOS, Windows Phone, Android®, BlackBerry 8, and Palm OS, and may be Internet, email, short message service (SMS), BlackBerry®, or other communication protocol-enabled handheld mobile devices (e.g., iPhone®, mobile phone, iPad®, tablet, personal digital assistant (PDA), or wearable device (e.g., Google Glass® head-mounted display). The client computing devices may be general-purpose personal computers, including, by way of example, personal computers and / or laptop computers running various versions of the Microsoft Windows® operating system, the Apple Macintosh® operating system, and / or the Linux® operating system. Alternatively, the client computing devices may be workstation computers running various commercially available UNIX® or UNIX-like operating systems, including, but not limited to, various GNU / Linux operating systems, e.g., Google Chrome® OS.Alternatively or additionally, the client computing device may be other electronic devices capable of communicating over VCN 2206 and / or a network with access to the Internet, such as a thin client computer, an Internet-enabled gaming system (e.g., a Microsoft Xbox® game console with or without a Kinect® gesture input device), and / or a personal messaging device.

[0196] VCN2206 is a local peering gateway that can communicate with Secure Shell (SSH) VCN2212 via LPG2210 included in SSH VCN2212. The IaaS provider may include a service tenancy 2219, which may be owned and / or operated by the IaaS provider. ...

[0197] Control plane VCN 2216 may include a control plane demilitarized zone (DMZ) tier 2220 that functions as a perimeter network (e.g., a portion of an enterprise network between the enterprise intranet and an external network). DMZ-based servers have a particular level of reliability and can contain security breaches. Additionally, DMZ tier 2220 may include one or more load balancer (LB) subnets 2222, a control plane app tier 2224 that may include an app subnet 2226, and a control plane data tier 2228 that may include a database (DB) subnet 2230 (e.g., a front-end DB subnet and / or a back-end DB subnet). LB subnet 2222 included in control plane DMZ layer 2220 may be communicatively connected to app subnet 2226 included in control plane app layer 2224 and to Internet gateway 2234, which may be included in control plane VCN 2216, and applisubli 2226 may be communicatively connected to DB subnet 2230, service gateway 2236, and network address translation (NAT) gateway 2238 included in control plane data layer 2228. Control plane VCN 2216 may include service gateway 2236 and NAT gateway 2238.

[0198] Control plane VCN 2216 can include data plane mirror app layer 2240, which can include app subnet 2226. App subnet 2226 included in data plane mirror app layer 2240 can include virtual network interface controller (VNIC) 2242 on which compute instance 2244 can run. Compute instance 2244 can communicatively connect app subnet 2226 of data plane mirror app layer 2240 to app subnet 2226, which can be included in data plane app layer 2246.

[0199] Data plane VCN 2218 may include data plane app layer 2246, data plane DMZ layer 2248, and data plane data layer 2250. Data plane DMZ layer 2248 may include LB subnet 2222, which may be communicatively connected to app subnet 2226 of data plane app layer 2246 and internet gateway 2234 of data plane VCN 2218. App subnet 2226 may be communicatively connected to service gateway 2236 of data plane VCN 2218 and NAT gateway 2238 of data plane VCN 2218. Data plane data layer 2250 may also include DB subnet 2230, which may be communicatively connected to app subnet 2226 of data plane app layer 2246.

[0200] The internet gateway 2234 of the control plane VCN 2216 and the internet gateway 2234 of the data plane VCN 2218 may be communicatively connected to a metadata management service 2252, which may be communicatively connected to the public internet 2254. The public internet 2254 may be communicatively connected to a NAT gateway 2238 of the control plane VCN 2216 and the NAT gateway 2238 of the data plane VCN 2218. The service gateway 2236 of the control plane VCN 2216 and the service gateway 2236 of the data plane VCN 2218 It may be communicatively connected to a cloud service 2256 .

[0201] In some examples, the service gateway 2236 of the control plane VCN 2216 or the data plane VCN 2218 can make application programming interface (API) calls to the cloud services 2256 without traversing the public internet 2254. The API calls from the service gateway 2236 to the cloud services 2256 can be one-way. The service gateway 2236 can make API calls to the cloud services 2256, and the cloud services 2256 can send request data to the service gateway 2236. However, the cloud services 2256 may not initiate the API calls to the service gateway 2236.

[0202] In some examples, secure host tenancy 2204 may be directly connected to service tenancy 2219, which may be an orphan. Secure host subnet 2208 can communicate with SSH subnet 2214 via LPG 2210, which allows bidirectional communication with the orphan system. By connecting secure host subnet 2208 to SSH subnet 2214, secure host subnet 2208 can access other entities in service tenancy 2219.

[0203] Control plane VCN 2216 allows users of service tenancy 2219 to configure or provision desired resources. The desired resources provisioned in control plane VCN 2216 may be deployed or used in data plane VCN 2218. In some examples, control plane VCN 2216 may be isolated from data plane VCN 2218, and data plane mirror app layer 2240 of control plane VCN 2216 can communicate with data plane app layer 2246 of data plane VCN 2218 via VNIC 2242, which may be included in data plane mirror app layer 2240 and data plane app layer 2246.

[0204] In some examples, a user or customer of the system may make a request, such as a create, read, update, or delete (CRUD) operation, via the public internet 2254, which may communicate the request to metadata management service 2252. Metadata management service 2252 may communicate the request to control plane VCN 2216 via internet gateway 2234. The request may be received by LB subnet 2222 included in control plane DMZ tier 2220. LB subnet 2222 may determine that the request is valid, and in response to this determination, LB subnet 2222 may send the request to app subnet 2226 included in control plane app tier 2224. If the request is validated and requires a call to the public internet 2254, the call to the public internet 2254 may be sent to NAT gateway 2238, which may make the call to the public internet 2254. Memory for storing the request may be stored in DB subnet 2230.

[0205] In some examples, data plane mirror app layer 2240 can facilitate direct communication between control plane VCN 2216 and data plane VCN 2218. For example, it may be desirable for changes, updates, or other suitable modifications to a configuration to be applied to resources included in data plane VCN 2218. Because control plane VCN 2216 can communicate directly with resources included in data plane VCN 2218 via VNIC 2242, changes, updates, or other suitable modifications to the configuration can be made.

[0206] In some embodiments, control plane VCN 2216 and data plane VCN 2218 may be included in service tenancy 2219. In this case, a user or customer of the system may not own or operate either control plane VCN 2216 or data plane VCN 2218. Instead, an IaaS provider may own or operate control plane VCN 2216 and data plane VCN 2218, both of which may be included in service tenancy 2219. This embodiment can prevent users or customers from interacting with other users' or customers' resources by enabling network isolation. This embodiment can also enable users or customers of the system to store databases privately without having to rely on the public internet 2254, which may not have the desired level of security for storage.

[0207] In another embodiment, LB subnet 2222 included in control plane VCN 2216 may be configured to receive signals from service gateway 2236. In this embodiment, control plane VCN 2216 and data plane VCN 2218 may be configured to be called by customers of the IaaS provider without calling the public internet 2254. Customers of the IaaS provider may desire this embodiment because databases used by the customers may be stored in service tenancy 2219, which is controlled by the IaaS provider and may be isolated from the public internet 2254.

[0208] Figure 23 is a block diagram 2300 illustrating another example parameter of an IaaS architecture, according to at least one embodiment. A service operator 2302 (e.g., service operator 2202 of Figure 22) may be communicatively connected to a secure host tenancy 2304 (e.g., secure host tenancy 2204 of Figure 22), which may include a virtual cloud network (VCN) 2306 (e.g., VCN 2206 of Figure 22) and a secure host subnet 2308 (e.g., secure host subnet 2208 of Figure 22). VCN 2306 may include a local peering gateway (LPG) 2310 (e.g., LPG 2210 of Figure 22), which may be communicatively connected to a secure shell (SSH) VCN 2312 (e.g., SSH VCN 2212 of Figure 22) via an LPG 2210 included in SSH VCN 2312. SSH VCN 2312 can include SSH subnet 2314 (e.g., SSH subnet 2214 in FIG. 22), and SSH VCN 2312 can be communicatively connected to control plane VCN 2316 (e.g., control plane VCN 2216 in FIG. 22) via LPG 2310 included in control plane VCN 2316. Control plane VCN 2316 can be included in service tenancy 2319 (e.g., service tenancy 2219 in FIG. 22), and data plane VCN 2318 (e.g., data plane VCN 2218 in FIG. 22) can be included in customer tenancy 2321, which can be owned or operated by a user or customer of the system.

[0209] The control plane VCN 2316 may include a control plane DMZ layer 2320 (e.g., control plane DMZ layer 2220 of FIG. 22) that may include a LB subnet 2322 (e.g., LB subnet 2222 of FIG. 22), a control plane app layer 2324 (e.g., control plane app layer 2224 of FIG. 22) that may include an app subnet 2326 (e.g., app subnet 2226 of FIG. 22), and a control plane data layer 2328 (e.g., control plane data layer 2228 of FIG. 22) that may include a database (DB) subnet 2330 (e.g., DB subnet 2230 of FIG. 22). The LB subnet 2322 included in the control plane DMZ layer 2320 may be communicatively connected to the app subnet 2326 included in the control plane app layer 2324 and to an Internet gateway 2334 (e.g., Internet gateway 2234 of FIG. 22), which may be included in the control plane VCN 2316. The app subnet 2326 may be connected to the control plane DMZ layer 2320. The control plane VCN 2316 may be communicatively connected to a DB subnet 2330, a service gateway 2336 (e.g., the service gateway of FIG. 22), and a network address translation (NAT) gateway 2338 (e.g., the NAT gateway 2238 of FIG. 22) included in the network data layer 2328. The control plane VCN 2316 may include the service gateway 2336 and the NAT gateway 2338.

[0210] Control plane VCN 2316 can include a data plane mirror app layer 2340 (e.g., data plane mirror app layer 2240 of FIG. 22 ), which can include app subnet 2326. App subnet 2326 included in data plane mirror app layer 2340 can include virtual network interface controller (VNIC) 2342 (e.g., VNIC 2242) on which compute instance 2344 (e.g., similar to compute instance 2244 of FIG. 22 ) can run. Compute instance 2344 can facilitate communication between app subnet 2326 of data plane mirror app layer 2340 and app subnet 2326, which can be included in data plane app layer 2346 (e.g., data plane app layer 2246 of FIG. 22 ), via VNIC 2342 included in data plane mirror app layer 2340 and VNIC 2342 included in data plane app layer 2346.

[0211] The internet gateway 2334 included in the control plane VCN 2316 may be communicatively connected to a metadata management service 2352 (e.g., metadata management service 2252 of FIG. 22), which may be communicatively connected to the public internet 2354 (e.g., public internet 2254 of FIG. 22). The public internet 2354 may be communicatively connected to a NAT gateway 2338 included in the control plane VCN 2316. The service gateway 2336 included in the control plane VCN 2316 may be communicatively connected to cloud services 2356 (e.g., cloud services 2256 of FIG. 22).

[0212] In some examples, data plane VCN 2318 may be included in customer tenancy 2321. In this case, the IaaS provider may provide a control plane VCN 2316 for each customer, and the IaaS provider may configure a unique compute instance 2344 for each customer that is included in service tenancy 2319. Each compute instance 2344 may allow communication between the control plane VCN 2316 included in service tenancy 2319 and the data plane VCN 2318 included in customer tenancy 2321. The compute instance 2344 may allow resources provisioned in the control plane VCN 2316 included in service tenancy 2319 to be deployed or used in the data plane VCN 2318 included in customer tenancy 2321.

[0213] In another example, a customer of an IaaS provider may have a database that resides in customer tenancy 2321. In this example, control plane VCN 2316 may include data plane minor app tier 2340, which may include app subnet 2326. Data plane mirror app tier 2340 may reside in data plane VCN 2318, but data plane mirror app tier 2340 may not reside in data plane VCN 2318. That is, data plane mirror app tier 2340 may have access to customer tenancy 2321, but data plane mirror app tier 2340 may not reside in data plane VCN 2318 and may not be owned or operated by the IaaS provider's customer. Data plane mirror app tier 2340 may be configured to make calls to data plane VCN 2318, but may not be configured to make calls to any entities included in control plane VCN 2316. A customer may have access to a database provisioned in control plane VCN 2316. A customer may desire to deploy or use resources within the data plane VCN 2318, and the data plane mirror application tier 2340 may facilitate the desired deployment or other use of the customer's resources.

[0214] In some embodiments, the IaaS provider's customer can apply filters to the data plane VCN 2318. In this embodiment, the customer can determine what the data plane VCN 2318 can access, and the customer can restrict access from the data plane VCN 2318 to the public internet 2354. The IaaS provider may not be able to apply filters or control access from the data plane VCN 2318 to any external network or database. Applying filters and controls to the data plane VCN 2318 included in the customer tenancy 2321 can help isolate the data plane VCN 2318 from other customers and the public internet 2354.

[0215] In some embodiments, cloud services 2356 can be called by service gateway 2336 to access services that may not reside on the public internet 2354, on the control plane VCN 2316, or on the data plane VCN 2318. The connection between cloud services 2356 and the control plane VCN 2316 or the data plane VCN 2318 may not be live or continuous. Cloud services 2356 may reside on a separate network owned or operated by the IaaS provider. Cloud services 2356 may be configured to receive calls from service gateway 2336 and may not be configured to receive calls from the public internet 2354. Some cloud services 2356 may be isolated from other cloud services 2356, and control plane VCNs 2316 may be isolated from cloud services 2356 that may not be located in the same region as the control plane VCN 2316. For example, control plane VCN 2316 may be located in “Region 1,” and cloud service “Deployment 16” may be located in “Region 1” and “Region 2.” If a call to a deployment 16 is made by a service gateway 2336 included in a control plane VCN 2316 located in Region 1, the call may be sent to the deployment 16 in Region 1. In this example, the control plane VCN 2316 or the deployment 16 in Region 1 may not be communicatively connected to the deployment 16 in Region 2.

[0216] 24 is a block diagram 2400 illustrating another example pattern of an IaaS architecture, according to at least one embodiment. A service operator 2402 (e.g., service operator 2202 of FIG. 22) may be communicatively connected to a secure host tenancy 2404 (e.g., secure host tenancy 2204 of FIG. 22), which may include a virtual cloud network (VCN) 2406 (e.g., VCN 2206 of FIG. 22) and a secure host subnet 2408 (e.g., secure host subnet 2208 of FIG. 22). VCN 2406 may include an LPG 2410 (e.g., LPG 2210 of FIG. 22), which may be communicatively connected to an SSH VCN 2412 (e.g., SSH VCN 2212 of FIG. 22) via an LPG 2410 included in SSH VCN 2412. SSH VCN 2412 can include SSH subnet 2414 (e.g., SSH subnet 2214 in FIG. 22 ), which may be communicatively connected to control plane VCN 2416 (e.g., control plane VCN 2216 in FIG. 22 ) via LPG 2410 included in control plane VCN 2416, and may be communicatively connected to data plane VCN 2418 (e.g., data plane 2224 in FIG. 22 ) via LPG 2410 included in data plane VCN 2418. Control plane VCN 2416 and data plane VCN 2418 may be included in service tenancy 2419 (e.g., service tenant 2219 in FIG. 22 ).

[0217] Control plane VCN 2416 may include a control plane DMZ layer 2420 (e.g., control plane DMZ layer 2220 in FIG. 22 ) that may include a load balancer (LB) subnet 2422 (e.g., LB subnet 2222 in FIG. 22 ), a control plane app layer 2424 (e.g., control plane app layer 2224 in FIG. 22 ) that may include an app subnet 2426 (e.g., app subnet 2226 in FIG. 22 ), and a control plane data layer 2428 (e.g., control plane data layer 2228 in FIG. 22 ) that may include a DB subnet 2430. LB subnet 2422 included in control plane DMZ layer 2420 may be communicatively connected to app subnet 2426 included in control plane app layer 2424 and to an Internet gateway 2434 (e.g., Internet gateway 2244 in FIG. 22 ), which may be included in control plane VCN 2416. The app subnet 2426 may be communicatively connected to a DB subnet 2430 included in the control plane data layer 2428, a service gateway 2436 (e.g., the service gateway in FIG. 22) and a network address translation (NAT) gateway 2438 (e.g., the NAT gateway 2248 in FIG. 22). The control plane VCN 2416 may include the service gateway 2436 and the NAT gateway 2438.

[0218] Data plane VCN 2418 may include a data plane app layer 2446 (e.g., data plane app layer 2246 in FIG. 22 ), a data plane DMZ layer 2448 (e.g., data plane DMZ layer 2248 in FIG. 22 ), and a data plane data layer 2450 (e.g., data plane data layer 2250 in FIG. 22 ). Data plane DMZ layer 2448 may include LB subnet 2422, which may be communicatively connected to trusted app subnet 2460 and untrusted app subnet 2462 of data plane app layer 2446 and internet gateway 2434 included in data plane VCN 2418. Trusted app subnet 2460 may be communicatively connected to service gateway 2436 included in data plane VCN 2418, NAT gateway 2438 included in data plane VCN 2418, and DB subnet 2430 included in data plane data layer 2450. The untrusted app subnet 2462 may be communicatively connected to a service gateway 2436 included in the data plane VCN 2418 and to a DB subnet 2430 included in the data plane data layer 2450. The data plane data layer 2450 may include a DB subnet 2430 that may be communicatively connected to a service gateway 2436 included in the data plane VCN 2418.

[0219] The untrusted app subnet 2462 may include one or more primary VNICs 2464(1)-(N), which may be communicatively connected to tenant virtual machines (VMs) 2466(1)-(N). Each tenant VM 2466(1)-(N) may be communicatively connected to a respective app subnet 2467(1)-(N), which may be included in a respective container egress VCN 2468(1)-(N), which may be included in a respective customer tenancy 2470(1)-(N). Each secondary VNIC 2472(1)-(N) may facilitate communication between the untrusted app subnet 2462 included in the data plane VCN 2418 and the app subnet included in the container egress VCN 2468(1)-(N). Each container egress VCN 2468(1)-(N) may include a NAT gateway 2438, which may be communicatively connected to the public internet 2454 (e.g., public internet 2254 in FIG. 22 ).

[0220] The internet gateway 2434 included in the control plane VCN 2416 and the internet gateway 2434 included in the data plane VCN 2418 provide a metadata management service 2452 that may be communicatively connected to the public internet 2454. (e.g., metadata management system 2252 of FIG. 22 ). Public Internet 2454 may be communicatively connected to NAT gateway 2438 included in control plane VCN 2416 and NAT gateway 2438 included in data plane VCN 2418. Service gateway 2436 included in control plane VCN 2416 and service gateway 2436 included in data plane VCN 2418 may be communicatively connected to cloud services 2456.

[0221] In some embodiments, data plane VCN 2418 may be integrated into customer tenancy 2470. This integration may be useful or desirable for an IaaS provider's customer in some cases, such as when they may want support when running their code. Customers may provide code that, when executed, may be disruptive, communicate with other customer resources, or cause undesirable effects. Thus, the IaaS provider can determine whether or not to run code that a customer has provided to the IaaS provider.

[0222] In some examples, an IaaS provider's customer can grant temporary network access to the IaaS provider and request a feature to be added to data plane app layer 2446. The code to perform the feature may run in VMs 2466(1)-(N) but cannot be configured to run elsewhere on data plane VCN 2418. Each VM 2466(1)-(N) may be connected to one customer tenancy 2470. Each container 2471(1)-(N) included in a VM 2466(1)-(N) may be configured to run code. In this case, double isolation (e.g., container 2471(1)-(N) may run code, and container 2471(1)-(N) may be included in at least one VM 2466(1)-(N) included in untrusted app subnet 2462) may exist, which can help prevent erroneous or unwanted code from damaging the IaaS provider's network or from damaging a different customer's network. Containers 2471(1)-(N) may be communicatively connected to customer tenancy 2470 and may be configured to send or receive data from customer tenancy 2470. Containers 2471(1)-(N) may not be configured to send or receive data from any other entity in data plane VCN 2418. Once code execution is complete, the IaaS provider can kill or discard containers 2471(I)-(N).

[0223] In some embodiments, trusted app subnet 2460 can execute code that may be owned or operated by the IaaS provider. In this embodiment, trusted app subnet 2460 may be communicatively connected to DB subnet 2430 and configured to perform CRUD operations on DB subnet 2430. Untrusted app subnet 2462 may be communicatively connected to DB subnet 2430, but in this embodiment, the untrusted app subnet may be configured to perform read operations within DB subnet 2430. Containers 2471(1)-(N) included in each customer's VMs 2466(1)-(N) and capable of executing code from the customer may not be communicatively connected to DB subnet 2430.

[0224] In other embodiments, the control plane VCN 2416 and the data plane VCN 2418 may not be directly communicatively coupled. In this embodiment, there may be no direct communication between the control plane VCN 2416 and the data plane VCN 2418. However, there may be indirect communication by at least one method. An LPG 2410 may be established by the IaaS provider that can facilitate communication between the control plane VCN 2416 and the data plane VCN 2418. In an example, control plane VCN 2416 or data plane VCN 2418 can make a call to cloud service 2456 through service gateway 2436. For example, a call from control plane VCN 2416 to cloud service 2456 can include a request for a service that can communicate with data plane VCN 2418.

[0225] 25 is a block diagram 2500 illustrating another example parameter of an IaaS architecture, according to at least one embodiment. A service operator 2502 (e.g., service operator 2202 of FIG. 22) may be communicatively connected to a secure host tenancy 2504 (e.g., secure host tenancy 2204 of FIG. 22), which may include a virtual cloud network (VCN) 2506 (e.g., VCN 2206 of FIG. 22) and a secure host subnet 2508 (e.g., secure host subnet 2208 of FIG. 22). VCN 2506 may include an LPG 2510 (e.g., LPG 2210 of FIG. 22), which may be communicatively connected to an SSH VCN 2512 (e.g., SSH VCN 2212 of FIG. 22) via an LPG 2510 included in SSH VCN 2512. VCN 2512 can include SSH subnet 2514 (e.g., SSH subnet 2214 in FIG. 22), which may be communicatively connected to control plane VCN 2516 (e.g., control plane VCN 2216 in FIG. 22) via LPG 2510 included in control plane VCN 2516, and may be communicatively connected to data plane VCN 2518 (e.g., data plane 2218 in FIG. 22) via LPG 2510 included in data plane VCN 2518. Control plane VCN 2516 and data plane VCN 2518 may be included in service tenancy 2519 (e.g., service tenancy 2225 in FIG. 22).

[0226] The control plane VCN 2516 may include a control plane DMZ layer 2520 (e.g., control plane DMZ layer 2220 of FIG. 22 ) that may include a LB subnet 2522 (e.g., LB subnet 2222 of FIG. 22 ), a control plane app layer 2524 (e.g., control plane app layer 2224 of FIG. 22 ) that may include an app subnet 2526 (e.g., app subnet 2226 of FIG. 22 ), and a control plane data layer 2528 (e.g., control plane data layer 2228 of FIG. 22 ) that may include a DB subnet 2530 (e.g., DB subnet 2430 of FIG. 24 ). The LB subnet 2522 included in the control plane DMZ layer 2520 may be communicatively connected to the app subnet 2526 included in the control plane app layer 2524 and to an Internet gateway 2534 (e.g., Internet gateway 2234 of FIG. 22 ), which may be included in the control plane VCN 2516. The app subnet 2526 may be communicatively connected to a DB subnet 2530 included in the control plane data layer 2528, a service gateway 2536 (e.g., the service gateway in FIG. 22) and a network address translation (NAT) gateway 2538 (e.g., the NAT gateway 2238 in FIG. 22). The control plane VCN 2516 may include the service gateway 2536 and the NAT gateway 2538.

[0227] Data plane VCN 2518 can include data plane app layer 2546 (e.g., data plane app layer 2246 in FIG. 22 ), data plane DMZ layer 2548 (e.g., data plane DMZ layer 2248 in FIG. 22 ), and data plane data layer 2550 (e.g., data plane data layer 2250 in FIG. 22 ). Data plane DMZ layer 2548 can include trusted app subnet 2560 (e.g., trusted app subnet 2460 in FIG. 24 ) and untrusted app subnet 2562 (e.g., untrusted app subnet 2462 in FIG. 24 ) of data plane app layer 2546 and LB subnet 2522, which can be communicatively connected to Internet gateway 2534 included in data plane VCN 2518. Net 2560 may be communicatively connected to a service gateway 2536 included in data plane VCN 2518, a NAT gateway 2538 included in data plane VCN 2518, and a DB subnet 2530 included in data plane data layer 2550. Untrusted app subnet 2562 may be communicatively connected to a service gateway 2536 included in data plane VCN 2518 and a DB subnet 2530 included in data plane data layer 2550. Data plane data layer 2550 may include a DB subnet 2530 that may be communicatively connected to a service gateway 2536 included in data plane VCN 2518.

[0228] The untrusted app subnet 2562 may include primary VNICs 2564(1)-(N) that may be communicatively connected to tenant virtual machines (VMs) 2566(1)-(N) that reside in the untrusted app subnet 2562. Each tenant VM 2566(1)-(N) may execute code in a respective container 2567(1)-(N) and may be communicatively connected to an app subnet 2526 that may be included in a data plane app layer 2546 that may be included in a container egress VCN 2568. Each secondary VNIC 2572(1)-(N) may facilitate communication between the untrusted app subnet 2562 included in the data plane VCN 2518 and the app subnet included in the container egress VCN 2568. The container egress VCN may include a NAT gateway 2538 that may be communicatively connected to the public internet 2554 (e.g., public internet 2254 in FIG. 22 ).

[0229] The internet gateway 2534 included in the control plane VCN 2516 and the internet gateway 2534 included in the data plane VCN 2518 may be communicatively connected to a metadata management service 2552 (e.g., metadata management system 2252 of FIG. 22 ), which may be communicatively connected to the public internet 2554. The public internet 2554 may be communicatively connected to the internet gateway 2534 included in the control plane VCN 2516 and the NAT gateway 2538 included in the data plane VCN 2518. The internet gateway 2534 included in the control plane VCN 2516 and the service gateway 2536 included in the data plane VCN 2518 may be communicatively connected to cloud services 2556.

[0230] In some examples, the pattern illustrated by the architecture of block diagram 2500 in FIG. 25 may be considered an exception to the pattern illustrated by the architecture of block diagram 2200 in FIG. 22 and may be desirable for an IaaS provider's customers when the IaaS provider cannot communicate directly with the customers (e.g., in unconnected regions). The customers may have real-time access to each of the containers 2567(1)-(N) included in each customer's VMs 2566(1)-(N). The containers 2567(1)-(N) may be configured to call each of the secondary VNICs 2572(1)-(N) included in the app subnet 2526 of the data plane app tier 2546, which may be included in the container egress VCN 2568. The secondary VNICs 2572(1)-(N) may send the call to a NAT gateway 2538, which may send the call to the public Internet 2554. In this example, containers 2567(1)-(N) that a customer can access in real time may be isolated from control plane VCN 2516 and may be isolated from other entities included in data plane VCN 2518. Containers 2567(1)-(N) may also be isolated from resources of other customers.

[0231] In another example, a customer can use containers 2567(1)-(N) to invoke cloud service 2556. In this example, the customer can execute code in containers 2567(1)-(N) that requests services from cloud service 2556. Container 2567(1)-(N) can send the request to secondary VNIC 2572(1)-(N), which can send the request to a NAT gateway that can send the request to public internet 2554. Public internet 2554 can send the request to LB subnet 2522 included in control plane VCN 2516 via internet gateway 2534. In response to determining that the request is valid, LB subnet 2522 can send the request to app subnet 2526, which can send the request to cloud service 2556 via service gateway 2536.

[0232] It should be noted that the illustrated IaaS architectures 2200, 2300, 2400, and 2500 may include elements other than those shown. Furthermore, the illustrated embodiments are only examples of some of the cloud infrastructure systems that may incorporate embodiments of the present disclosure. In other embodiments, the IaaS system may have more or fewer elements than those shown, may combine two or more elements, or may have a different configuration or arrangement of elements.

[0233] In certain embodiments, the IaaS system described herein may include a suite of application, middleware, and database services that are delivered to customers in a self-service, subscription-based, elastically scalable, reliable, highly available, and secure manner. One example of such an IaaS system is the Oracle® Cloud Infrastructure (OCI) offered by the present applicant.

[0234] 26 illustrates an exemplary computer system 2600 upon which various embodiments may be implemented. System 2600 may be used to implement any of the computer systems described above. As shown, computer system 2600 includes a processing unit 2604 that communicates with a number of peripheral subsystems via a bus subsystem 2602. These peripheral subsystems may include a processing acceleration unit 2606, an I / O subsystem 2608, a storage subsystem 2618, and a communications subsystem 2624. Storage subsystem 2618 includes a tangible computer-readable storage medium 2622 and a system memory 2610.

[0235] Bus subsystem 2602 provides a mechanism for allowing the various components and subsystems of computer system 2600 to communicate with each other as intended. While bus subsystem 2602 is shown schematically as a single bus, alternative embodiments of the bus subsystem may utilize multiple buses. Bus subsystem 2602 may be any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. For example, such architectures may include an Industry Standard Architecture (ISA) bus, a MicroChannel Architecture (MCA), an Bus Extended ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus, which may be implemented as a mezzanine bus manufactured in accordance with the IEEE P1386.1 standard.

[0236] Processing unit 2604, which may be implemented as one or more integrated circuits (e.g., conventional microprocessors or microcontrollers), controls the operation of computer system 2600. Processing unit 2604 may include one or more processors. These processors may include single-core or multi-core processors. In some embodiments, processing unit 2604 may be implemented as one or more independent processing units 2632 and / or 2634, with single-core or multi-core processors included in each processing unit. In other embodiments, processing unit 2604 may be implemented as two or more independent processing units 2632 and / or 2634, with single-core or multi-core processors included in each processing unit. It may also be implemented as a quad-core processing unit formed by integrating two dual-core processors into a single chip.

[0237] In various embodiments, processing unit 2604 may execute various programs in response to program code and may maintain multiple programs or processes executing simultaneously. At any given time, some or all of the program code being executed may reside in processor 2604 and / or storage subsystem 2618. Processor 2604, through appropriate programming, may provide the various functionality discussed above. Computer system 2600 may further include a processing acceleration unit 2606, which may include a digital signal processor (DSP), a special purpose processor, and / or the like.

[0238] The I / O subsystem 2608 can include user interface input devices and user interface output devices. User interface input devices can include a keyboard, a pointing device such as a mouse or trackball, a touchpad or touchscreen integrated into a display, a scroll wheel, a click wheel, dials, buttons, switches, keypads, voice input devices with voice command recognition systems, microphones, and other types of input devices. User interface input devices can also include motion sensing and / or gesture recognition devices, such as a Microsoft Kinect® motion sensor, which can provide input via a natural user interface (NUI) that utilizes gestures and voice commands, such as a Microsoft Xbox® 360 game controller. The user interface input device can control and interact with a force device. The user interface input device can also include an eye gesture recognition device, such as a Google Glass® blink detector. The Google Glass® blink detector detects a user's eye activity (e.g., "blinks" when taking a photo and / or selecting a menu) and converts the eye activity into input for input into the input device (e.g., Google Glass®). Furthermore, the user interface input device may include a voice recognition detection device that enables a user to interact with a voice recognition system (e.g., Siri® Navigator) via voice commands.

[0239] User interface input devices may also include, but are not limited to, three-dimensional (3D) mice, joysticks or pointing sticks, gamepads, graphics tablets, audio / visual devices such as speakers, digital cameras, digital video cameras, portable media players, webcams, image scanners, fingerprint scanners, barcode readers, 3D scanners, 3D printers, laser range finders, and eye-tracking devices. User interface input devices may also include medical imaging input devices such as computed tomography (CT) scanners, magnetic resonance imaging (MRI) scanners, ultrasound emission tomography (EMC) scanners, and medical ultrasound scanners. User interface input devices may also include audio input devices such as MIDI keyboards and electronic musical instruments.

[0240] User interface output devices may also include non-visual displays such as display subsystems, indicator lights, or audio output devices. The display subsystem may be, for example, a flat-panel device using a cathode ray tube (CRT), liquid crystal display (LCD), or plasma display, a projection device, or a touch screen. In general, when the term "output device" is used, it is intended to include all possible types of devices and mechanisms for outputting information from computer system 2600 to a user or to another computer. For example, user interface output devices may include various display devices that visually convey text, images, and audio / video information, such as monitors, printers, speakers, headphones, car navigation systems, plotters, etc. This includes, but is not limited to, computers, audio output devices, and modems. Computer system 2600 may include a storage subsystem 2618. Storage subsystem 2618 comprises software elements, which are illustratively located in system memory 2610. System memory 2610 may store program instructions loadable and executable by processing unit 2604, as well as data generated by the execution of these programs.

[0241] Depending on the configuration and type of computer system 2600, system memory 2610 may be volatile memory (e.g., random access memory (RAM)) and / or non-volatile memory (e.g., read-only memory (ROM), flash memory). Generally, RAM contains data and / or program modules that are immediately accessible to and / or presently being operated on and executed by processing unit 2604. In some implementations, system memory 2610 may be static random access memory (SRAM). In some implementations, the computer system 2600 may include a number of different types of memory, such as a basic input / output system (BIOS), which contains the basic routines that help to transfer information between elements within the computer system 2600, such as during start-up. By way of example and not limitation, the system memory 2610 may be configured to store a client application, a web browser, a mid-tier application, a relational database management system (RDBMS), Also shown are application programs 2612, program data 2614, and operating system 2616, which may include various versions of the Microsoft Windows®, Apple Macintosh®, and / or Linux® operating systems, various commercially available UNIX® or UNIX-like operating systems (including, but not limited to, various GNU / Linux operating systems, Google Chrome® OS, etc.), and / or mobile operating systems such as iOS, Windows® Phone, Android® OS, BlackBerry® 26 OS, and Palm® OS operating systems.

[0242] Additionally, the storage subsystem 2618 may provide a tangible, computer-readable storage medium for storing the basic programming and data constructs that provide the functionality of some embodiments. Software (programs, code modules, instructions) that, when executed by a processor, provide the above-described functionality may be stored in the storage subsystem 2618. These software modules or instructions may be executed by the processing unit 2604. The storage subsystem 2618 may also provide a repository for storing data used in accordance with the present disclosure.

[0243] Storage subsystem 2600 may also include computer-readable storage medium reader 2620 further connectable to computer-readable storage medium 2622. Computer-readable storage medium 2622 may comprehensively represent remote, local, fixed, and / or removable storage devices, as well as storage media for temporarily and / or permanently containing, storing, transmitting, and retrieving computer-readable information together with, or in combination with, system memory 2610 as needed.

[0244] Also, the computer readable storage medium 2622 containing the code or portions of the code is The information may include any suitable medium known or used in the art. Such media include storage and communication media, such as, but not limited to, volatile and nonvolatile, removable and non-removable media, implemented in any method or technology for storing and / or transmitting information. This may include tangible computer-readable storage media, such as RAM, ROM, electronically erasable programmable ROM (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical storage devices, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or other tangible computer-readable media. This may also include intangible computer-readable media, such as a data signal, data transmission, or other medium usable to transmit the desired information and accessible by computer system 2600.

[0245] By way of example, computer-readable storage medium 2622 may include a hard disk drive that reads from or writes to non-removable, non-volatile magnetic media, a magnetic disk drive that reads from or writes to a removable, non-volatile magnetic disk, and a CD The computer-readable storage medium 2622 may include an optical disk drive that reads from or writes to removable, non-volatile optical disks such as ROM, DVD and Blu-ray disks or other optical media. The computer-readable storage medium 2622 may include, but is not limited to, Zip drives, flash memory cards, universal serial bus (USB) flash drives, secure digital (SD) cards, DVD disks, digital video tapes, etc. The computer-readable storage medium 2622 may also include flash memory-based SSDs, enterprise flash drives, solid-state drives (SSDs) based on non-volatile memory such as solid-state ROM, solid-state RAM, dynamic RAM, etc. SSDs may include SSDs based on volatile memory such as dynamic RAM, static RAM, DRAM-based SSDs, magnetoresistive RAM (MRAM) SSDs, and hybrid SSDs that use a combination of DRAM and flash memory-based SSDs. The disk drives and their associated computer-readable media may provide non-volatile storage of computer-readable instructions, data structures, program modules, and other data for the computer system 2600.

[0246] Communications subsystem 2624 provides an interface with other computer systems and networks. Communications subsystem 2624 serves as an interface for receiving data from other systems and transmitting data from computer system 2600 to other systems. For example, communications subsystem 2624 may enable computer system 2600 to connect to one or more devices via the Internet. In some embodiments, communications subsystem 2624 may include radio frequency (RF) transceiver components for accessing wireless voice and / or data networks (e.g., using cellular technologies, advanced data network technologies such as 3G, 4G, or EDGE (enhanced data rates for global evolution)), Wi-Fi, or other wireless technologies. (IEEE 802.11 family of standards or other mobile communication technologies or any combination thereof), global positioning system (GPS) receiver configuration In some embodiments, the communications subsystem 2624 may provide a wired network connection (e.g., Ethernet) in addition to or instead of a wireless interface.

[0247] In some embodiments, the communications subsystem 2624 also communicates structured and / or unstructured data feeds 2626, event streams 2628, event updates 2629, and the like on behalf of one or more users who may be using the computer system 2600. An incoming communication may be received in the form of, for example, 630 .

[0248] As an example, the communications subsystem 2624 may provide Twitter feeds, Facebook updates, Rich Site Summary (RSS) feeds, and other similar services. The data feeds 2626 may be configured to receive data feeds, such as web feeds, in real time from users of social networks and / or other communication services, and / or to receive real-time updates from one or more third-party sources.

[0249] Communications subsystem 2624 may also be configured to receive data in the form of a continuous data stream, which may include an event stream 2628 of real-time events that may be continuous or essentially unbounded with no clear ends and / or event updates 2630. Examples of applications that generate continuous data may include, for example, sensor data applications, financial tickers, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, automobile traffic monitoring, etc.

[0250] The communications subsystem 2624 may also be configured to output structured and / or unstructured data feeds 2626, event streams 2628, event updates 2630, etc. to one or more databases that may communicate with one or more streaming data source computers coupled to the computer system 2600.

[0251] The computer system 2600 may be one of a variety of types, including a handheld mobile device (e.g., an iPhone® mobile phone, an iPad® computing tablet, a PDA), a wearable device (e.g., a Google Glass® head-mounted display), a PC, a workstation, a mainframe, a kiosk, a server rack, or other data processing system.

[0252] In the foregoing description, for purposes of explanation, specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure. However, it will be apparent that various embodiments can be practiced without these specific details. The following description provides examples only and is not intended to limit the scope, applicability, or configuration of the present disclosure. Rather, the following description of the embodiments provides one of ordinary skill in the art with an enabling description for practicing the embodiments. It should be understood that various changes can be made in the function and arrangement of elements without departing from the spirit and scope of the present disclosure, as defined by the appended claims. The drawings and descriptions are not intended to be limiting. Circuits, systems, networks, processes, and other components may be shown as components in block diagram form so as not to obscure the embodiments in unnecessary detail. In other embodiments, well-known circuits, processes, algorithms, structures, and techniques may be shown without unnecessary detail so as not to obscure the embodiments. The teachings of the present disclosure may also be applied to various types of applications, such as mobile applications, non-mobile applications, desktop applications, web applications, enterprise applications, etc. Additionally, the teachings of the present disclosure are not limited to a particular operating environment (eg, operating system, device, platform, etc.) but may be applied to multiple different operating environments.

[0253] Also, note that each embodiment is described as a process that is depicted as a flowchart, flow diagram, data flow diagram, structure diagram, or block diagram. Although a flowchart describes operations as a sequential process, many operations may be performed in parallel or simultaneously. Additionally, the order of operations may be rearranged. A process terminates when that operation is completed, but may include additional steps not shown in the figures. Process A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination may correspond to a return of the calling function or the main function.

[0254] The words "example" and "exemplary" are used herein to mean "serving as an example, instance, or illustration." Any embodiment or design described herein as "exemplary" or "example" is not necessarily to be construed as preferred or advantageous over other embodiments or designs.

[0255] The terms “machine-readable storage medium” or “computer-readable storage medium” include, but are not limited to, portable or non-portable storage devices, optical storage devices, and various other media capable of storing, storing, or transporting instructions and / or data. Machine-readable storage media or computer-readable storage media may also include non-transitory media capable of storing data and that do not involve carrier waves and / or transitory electronic signals propagating via wireless or wired connections. Examples of non-transitory media may include, but are not limited to, magnetic disks or tapes, optical storage media such as compact discs (CDs) or digital versatile discs (DVDs), flash memory, memory, or memory devices. A computer program product may include code and / or machine-executable instructions, which may represent any combination of procedures, functions, subprograms, programs, routines, subroutines, modules, software packages, classes, instructions, data structures, or program statements. A code segment may be coupled to another code segment or a hardware circuit by transferring and / or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc. may be conveyed, forwarded, or transmitted via any suitable means such as memory sharing, message passing, token passing, network transmission, etc.

[0256] Furthermore, embodiments may be implemented by hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware, or microcode, program code or code segments performing the necessary operations may be stored on a machine-readable medium. A processor may perform the necessary operations. The systems illustrated in some of the figures may be provided in various configurations. In some examples, the system may be configured as a distributed system in which one or more elements of the system are distributed across one or more networks in a cloud computing system. When an element is described as being "configured" to perform a particular operation, such configuration may be achieved, for example, by designing electronic circuitry or other hardware to perform the operation, by programming or controlling electronic circuitry (e.g., a microprocessor or other suitable electronic circuitry) to perform the operation, or any combination thereof.

[0257] While specific embodiments of the present disclosure have been described, various modifications, variations, alternative configurations, and equivalents are encompassed within the scope of the present disclosure. The embodiments of the present disclosure are not limited to operating in a particular data processing environment, but can freely operate in multiple data processing environments. Furthermore, while embodiments of the present disclosure have been described using a particular series of actions and steps, it will be apparent to those skilled in the art that the scope of the present disclosure is not limited to the series of actions and steps described. Various features and aspects of the above-described embodiments can be used individually or jointly.

[0258] Additionally, although embodiments of the present disclosure have been described using particular combinations of hardware and software, other combinations of hardware and software are within the scope of the present disclosure. It should be recognized that embodiments of the present disclosure may be implemented using only hardware, only software, or a combination thereof. The various processes described in this disclosure may run on the same processor or any combination of different processors. Thus, when a component or module is described as being configured to perform a particular process, that configuration may be achieved, for example, by designing electronic circuitry to perform the process, by programming a programmable electronic circuit (such as a microprocessor) to perform the process, or a combination thereof. Processes may communicate using a variety of techniques, including, but not limited to, conventional techniques for inter-process communication. Different pairs of processes may use different techniques, or the same pair of processes may use different techniques at different times.

[0259] Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense. It will be apparent, however, that additions, subtractions, deletions, and other modifications and changes may be made without departing from the broad spirit and scope defined by the appended claims. Accordingly, while specific embodiments of the present disclosure have been described, these embodiments are not intended to be limiting. Various modifications and equivalents thereof are intended to be encompassed within the scope of the appended claims.

[0260] The indefinite article "a" / used in the context of describing this disclosure (particularly in the context of the claims) "An," the definite article "the" and similar references are used herein unless otherwise stated or contained in the context. Unless otherwise clearly indicated above, the terms "comprising," "having," "including ... and "containing" should be construed as an open-ended term (i.e., meaning "including, but not limited to") unless otherwise specified. The term "connected" should be construed as partly or wholly contained within, attached to, or joined together, even if there is something intervening. The recitation of ranges of values ​​herein is intended merely as a shorthand method of referring to each individual value falling within the range, and unless otherwise stated herein, each individual value is incorporated herein as if set forth individually herein. Unless otherwise stated herein or unless the context clearly indicates otherwise, all methods described herein can be performed in any suitable order. The use of any and all examples or exemplary language (e.g., "such as") herein is intended to further clarify embodiments of the present disclosure and does not limit the scope of the disclosure, unless otherwise specified. No language in the specification should be construed as indicating any non-claimed element essential to the practice of the disclosure.

[0261] Disjunctive language, such as the phrase "at least one of X, Y, or Z," is intended to be understood in context as generally used to indicate that an item, term, etc. may be either X, Y, or Z, or any combination thereof (e.g., X, Y, and / or Z), unless otherwise specified. Thus, such disjunctive language is not generally intended to or imply that a particular embodiment requires that at least one of X, at least one of Y, or at least one of Z be present.

[0262] Example embodiments of the present disclosure can be described in light of the following sections. Item 1 1. A system comprising: a first virtual network configured for the customer, the first virtual network including a first resource having a first Internet Protocol (IP) address; a second virtual network configured for the service provider, the network includes a second resource having a second IP address and is configured to provide a service to the customer based on access of the first resource by the second resource; one or more network resources; the one or more network resources are configured to determine, based on a NAT mapping, a network address translation (NAT) mapping between the first IP address and a reserved IP address, the reserved IP address being outside a first IP address range of the first virtual network and outside a second IP address range of the second virtual network; the one or more network resources are configured to at least one of send a first packet to the second virtual network or receive a second packet from the second virtual network, wherein a payload of the first packet includes the reserved IP address and a header of the second packet includes the reserved IP address as a destination address; The system, wherein the second resource's access to the first resource is based on a NAT mapping between the first IP address and the reserved IP address.

[0263] Section 2 the one or more network resources include a gateway; The second packet is received by the gateway, The gateway is determining a first IP address based on a NAT mapping between the first IP address and the reserved IP address; updating the second packet by at least replacing the reserved IP address with the first IP address; Item 1. The system of item 1, configured to transmit the updated second packet to the first virtual network.

[0264] Section 3 the one or more network resources include a Domain Name System (DNS) service; The first packet is sent by the DNS service, DNS services are Receives a DNS query containing a fully qualified domain name (FQDN) from the second virtual network, determining a match between the FQDN and a list of FQDNs associated with the customer; Sends the DNS query to the customer's DNS resolver, receiving a DNS response from the DNS resolver, the DNS response including the first IP address; generating a NAT mapping between the first IP address and the reserved IP address; updating the DNS response by at least replacing the first IP address with the reserved IP address based on the NAT mapping; Item 1. The system of claim 1, configured to send the updated DNS response to the second virtual network.

[0265] Section 4 the one or more network resources include a Domain Name System (DNS) service; The first packet is sent by the DNS service, DNS services are Receives a DNS query containing a fully qualified domain name (FQDN) from the second virtual network, Determine that the FQDN is excluded from the list of FQDNs associated with the customer, Sends a DNS query to the service provider's DNS resolver, receiving a DNS response from the DNS resolver that includes the third IP address; configured to send DNS responses to the second virtual network, as described in Section 1. system.

[0266] Section 5 the one or more network resources include a database service; The database service is receiving a database query from the second virtual network; determining a third IP address of the first network interface of the first virtual network and a fourth IP address of the database of the first virtual network; sending the database query to the first virtual network in a third packet that includes the third IP address as a source and the fourth IP address as a destination; receiving a database response from the first virtual network; Determine a fifth IP address of a second network interface of the second virtual network; Item 1. The system of Item 1, configured to transmit the database response to the second virtual network in a fourth packet that includes the fifth IP address as a source and the second IP address as a destination.

[0267] Section 6 Item 1. The system of Item 1, wherein the first IP address range overlaps with the second IP address range.

[0268] Section 7 the first IP address range overlaps with a third IP address range of a third virtual network configured for another customer; Item 1. The system of item 1, wherein the second resource of the service provider is configured to provide services to other customers based on access by the second resource to a third resource of the third virtual network.

[0269] Section 8 further comprising a virtual network interface card associated with the first virtual network; Item 1. The system of Item 1, wherein the virtual network interface card is addressable with a plurality of IP addresses within a first IP address range.

[0270] Section 9 Item 9. The system of item 8, wherein at least one of the plurality of IP addresses is associated with a different service of the service provider.

[0271] Section 10 further comprising a virtual network interface card associated with a second virtual network; Item 1. The system of Item 1, wherein the virtual network interface card is addressable with multiple IP addresses within a second IP address range.

[0272] Section 11 Item 11. The system of item 10, wherein at least two of the plurality of IP addresses are associated with different services of the service provider.

[0273] Section 12 A method implemented by a system, the method comprising: determining a network address translation (NAT) mapping between the first IP address and the reserved IP address based on the NAT mapping, wherein the first IP address is within a first IP address range of a first virtual network configured for the customer and the reserved IP address is outside the first IP address range and outside a second IP address range of a second virtual network configured for the service provider; at least one of transmitting a first packet to a second virtual network or receiving a second packet from the second virtual network, wherein a payload of the first packet includes the reserved IP address and a header of the second packet includes the reserved IP address as a destination address; The method, wherein access to the first resource in the first virtual network by the second resource in the second virtual network is based on a NAT mapping between the first IP address and a reserved IP address.

[0274] Item 13 receiving a customer request for a service from a service provider; 13. The method of clause 12, further comprising invoking at least one of a set of DNS services for the pair of customer identifier and service provider identifier.

[0275] Item 14 determining a tuple including a protocol, a source IP address, a source port, a destination IP address, and a destination port of the second packet; selecting a DNS service from a set of DNS services based on the tuple; Item 14. The method of item 13, further comprising: sending the second packet to a DNS service.

[0276] Item 15 Item 13. The method of Item 12, wherein the first IP address range overlaps with the second IP address range.

[0277] Section 16 the first IP address range overlaps with a third IP address range of a third virtual network configured for another customer; Item 13. The method of item 12, wherein the second resource of the service provider is configured to provide services to other customers based on access by the second resource to a third resource of the third virtual network.

[0278] Item 17 One or more non-transitory computer-readable storage media storing instructions that, when executed on a system, cause the system to perform the following operations: The operation is determining a network address translation (NAT) mapping between the first IP address and the reserved IP address based on the NAT mapping, wherein the first IP address is within a first IP address range of a first virtual network configured for the customer and the reserved IP address is outside the first IP address range and outside a second IP address range of a second virtual network configured for the service provider; at least one of transmitting a first packet to a second virtual network or receiving a second packet from the second virtual network, wherein a payload of the first packet includes the reserved IP address and a header of the second packet includes the reserved IP address as a destination address; A first resource of a first virtual network is connected to a second resource of a second virtual network. and wherein access to the resource is based on a NAT mapping between the first IP address and a reserved IP address.

[0279] Section 18 The operation is determining a first IP address based on a NAT mapping between the first IP address and a reserved IP address; updating the second packet by at least replacing the reserved IP address with the first IP address; and transmitting the updated second packet to the first virtual network.

[0280] Item 19 The operation is receiving a DNS query from a second virtual network, the DNS query including a fully qualified domain name (FQDN); determining a match between the FQDN and a list of FQDNs associated with the customer; sending DNS queries to the customer's DNS resolver; receiving a DNS response from a DNS resolver, the DNS response including the first IP address; generating a NAT mapping between the first IP address and the reserved IP address; updating the DNS response by at least replacing the first IP address with the reserved IP address based on the NAT mapping; and sending the updated DNS response to the second virtual network.

[0281] Section 20 The operation is receiving a database query from a second virtual network; determining a third IP address of the first network interface of the first virtual network and a fourth IP address of the database of the first virtual network; sending the database query in a third packet to the first virtual network, the third packet including the third IP address as a source and the fourth IP address as a destination; receiving a database response from the first virtual network; determining a fifth IP address of a second network interface of the second virtual network; and transmitting the database response to the second virtual network in a fourth packet that includes the fifth IP address as a source and the second IP address of the second resource as a destination.

[0282] Preferred embodiments of the present disclosure are described herein, including the best mode known for carrying out the disclosure. Variations of these preferred embodiments will become apparent to those skilled in the art upon reading the foregoing description. Those skilled in the art can employ such variations as appropriate, and the present disclosure may be practiced otherwise than as specifically described herein. Accordingly, this disclosure includes all modifications and equivalents of the subject matter recited in the claims appended hereto as permitted by applicable law. Moreover, any combination of the above-described elements in all possible variations thereof is encompassed by this disclosure unless otherwise indicated herein.

[0283] All references cited herein, including publications, patent applications, and patents, are incorporated by reference to the same extent as if each individual reference was individually and specifically indicated to be incorporated by reference and was set forth in its entirety herein.

[0284] In the foregoing specification, aspects of the disclosure have been described with reference to specific embodiments thereof, but those skilled in the art will recognize that the disclosure is not limited thereto. Various features and aspects of the above-described disclosure may be used individually or jointly. Moreover, the embodiments can be utilized in any number of environments and applications beyond those described herein without departing from the broader spirit and scope of the specification. Accordingly, the specification and drawings should be regarded in an illustrative rather than a restrictive sense.< / realm>

Claims

1. 1. A system comprising: a first virtual network configured for a customer, the first virtual network including a first resource having a first Internet Protocol (IP) address; a second virtual network configured for a service provider, the second virtual network including a second resource having a second IP address, and configured to provide a service to the customer based on access of the first resource by the second resource; one or more network resources; the one or more network resources are configured to determine a Network Address Translation (NAT) mapping between a first IP address and a reserved IP address based on a NAT mapping, the reserved IP address being outside a first IP address range of the first virtual network and outside a second IP address range of the second virtual network; the one or more network resources are configured to at least one of send a first packet to the second virtual network or receive a second packet from the second virtual network, a payload of the first packet including the reserved IP address and a header of the second packet including the reserved IP address as a destination address; The system, wherein the access of the second resource to the first resource is based on the NAT mapping between the first IP address and the reserved IP address.

2. the one or more network resources include a gateway; the second packet is received by the gateway; The gateway determining the first IP address based on the NAT mapping between the first IP address and the reserved IP address; updating the second packet by at least replacing the reserved IP address with the first IP address; The system of claim 1 , configured to transmit the updated second packet to the first virtual network.

3. the one or more network resources include a domain name system (DNS) service; the first packet is sent by the DNS service; The DNS service receiving a DNS query from the second virtual network, the DNS query including a fully qualified domain name (FQDN); determining a match between the FQDN and a list of FQDNs associated with the customer; sending the DNS query to the customer's DNS resolver; receiving a DNS response from the DNS resolver that includes the first IP address; creating the NAT mapping between the first IP address and the reserved IP address; updating the DNS response by at least replacing the first IP address with the reserved IP address based on the NAT mapping; The system of claim 1 , configured to send the updated DNS response to the second virtual network.

4. The one or more network resources may include a Domain Name System (DNS) service. fruit, the first packet is sent by the DNS service; The DNS service receiving a DNS query from the second virtual network, the DNS query including a fully qualified domain name (FQDN); determining that the FQDN is excluded from a list of FQDNs associated with the customer; sending the DNS query to a DNS resolver of the service provider; receiving a DNS response from the DNS resolver that includes a third IP address; The system of claim 1 , configured to send the DNS response to the second virtual network.

5. the one or more network resources include a database service; The database service is receiving a database query from the second virtual network; determining a third IP address of a first network interface of the first virtual network and a fourth IP address of a database of the first virtual network; sending the database query to the first virtual network in a third packet that includes the third IP address as a source and the fourth IP address as a destination; receiving a database response from the first virtual network; determining a fifth IP address of a second network interface of the second virtual network; 2. The system of claim 1, configured to transmit the database response to the second virtual network in a fourth packet that includes the fifth IP address as a source and the second IP address as a destination.

6. The system of claim 1 , wherein the first IP address range overlaps with the second IP address range.

7. the first IP address range overlaps with a third IP address range of a third virtual network configured for another customer; 2. The system of claim 1, wherein the second resource of the service provider is configured to provide the service to the other customer based on access by the second resource to a third resource of the third virtual network.

8. a virtual network interface card associated with the first virtual network; The system of claim 1 , wherein the virtual network interface card is addressable with a plurality of IP addresses within the first IP address range.

9. The system of claim 8 , wherein at least one of the plurality of IP addresses is associated with a different service of the service provider.

10. a virtual network interface card associated with the second virtual network; The system of claim 1 , wherein the virtual network interface card is addressable with a plurality of IP addresses within the second IP address range.

11. The system of claim 10 , wherein at least two of the plurality of IP addresses are associated with different services of the service provider.

12. 1. A method implemented by a system, the method comprising: determining a Network Address Translation (NAT) mapping between a first IP address and a reserved IP address based on a NAT mapping, wherein the first IP address is within a first IP address range of a first virtual network configured for the customer, and the reserved IP address is outside the first IP address range and outside a second IP address range of a second virtual network configured for the service provider; at least one of transmitting a first packet to the second virtual network or receiving a second packet from the second virtual network, wherein a payload of the first packet includes the reserved IP address and a header of the second packet includes the reserved IP address as a destination address; 11. The method of claim 10, wherein access by a second resource of the second virtual network to a first resource of the first virtual network is based on the NAT mapping between the first IP address and the reserved IP address.

13. receiving the customer's request for services from the service provider; 13. The method of claim 12, further comprising invoking at least one of a set of DNS services for the customer identifier and service provider identifier pair.

14. determining a tuple including a protocol, a source IP address, a source port, a destination IP address, and a destination port of the second packet; selecting a DNS service from the set of DNS services based on the tuple; and transmitting the second packet to the DNS service.

15. The method of claim 12 , wherein the first IP address range overlaps with the second IP address range.

16. the first IP address range overlaps with a third IP address range of a third virtual network configured for another customer; 13. The method of claim 12, wherein the second resource of the service provider is configured to provide services to the other customer based on access by the second resource to a third resource of the third virtual network.

17. One or more non-transitory computer-readable storage media storing instructions that, when executed on a system, cause the system to perform the following operations: The operation is determining a Network Address Translation (NAT) mapping between a first IP address and a reserved IP address based on a NAT mapping, wherein the first IP address is within a first IP address range of a first virtual network configured for the customer, and the reserved IP address is outside the first IP address range and outside a second IP address range of a second virtual network configured for the service provider; at least one of transmitting a first packet to the second virtual network or receiving a second packet from the second virtual network, wherein a payload of the first packet includes the reserved IP address and a header of the second packet includes the reserved IP address as a destination address; a first resource of the first virtual network by a second resource of the second virtual network; one or more non-transitory computer-readable storage media, wherein access to a resource is based on the NAT mapping between the first IP address and the reserved IP address.

18. The operation is determining the first IP address based on the NAT mapping between the first IP address and the reserved IP address; updating the second packet by at least replacing the reserved IP address with the first IP address; and transmitting the updated second packet to the first virtual network.

19. The operation is receiving a DNS query from the second virtual network, the DNS query including a fully qualified domain name (FQDN); determining a match between the FQDN and a list of FQDNs associated with the customer; sending the DNS query to the customer's DNS resolver; receiving a DNS response from the DNS resolver that includes the first IP address; creating the NAT mapping between the first IP address and the reserved IP address; updating the DNS response by at least replacing the first IP address with the reserved IP address based on the NAT mapping; and transmitting the updated DNS response to the second virtual network.

20. The operation is receiving a database query from the second virtual network; determining a third IP address of a first network interface of the first virtual network and a fourth IP address of a database of the first virtual network; sending the database query to the first virtual network in a third packet that includes the third IP address as a source and the fourth IP address as a destination; receiving a database response from the first virtual network; determining a fifth IP address of a second network interface of the second virtual network; 20. The one or more non-transitory computer-readable storage media of claim 17, further comprising: transmitting the database response to the second virtual network in a fourth packet that includes the fifth IP address as a source and the second IP address of the second resource as a destination.

Images