cloud gateway

Abstract

Techniques are described for providing a multi-cloud gateway (MCG) in a first cloud infrastructure, including in a first cloud environment provided by a first cloud service provider. The MCG implemented in the first cloud environment receives a first request requesting performance of a first operation in a second cloud environment. In response to receiving the first request, the MCG generates a first API call directed to the second cloud environment and causes the first API call to be communicated to the second cloud environment. The MCG receives a second request requesting performance of a second operation in a third cloud environment. In response to receiving the second request, the MCG generates a second API call directed to the third cloud environment and causes the second API call to be communicated to the third cloud environment, where each cloud environment is provided by a unique cloud service provider.

Description

[0001] Cross-references to related applications

[0002] This international application claims priority and benefit to U.S. Provisional Application No. 63 / 600,375, filed November 17, 2023, and U.S. Patent Application No. 18 / 645,136, filed April 24, 2024, the contents of which are incorporated herein by reference in their entirety for all purposes. Technical Field

[0003] This disclosure generally relates to cloud architecture, and more specifically to a multi-cloud gateway that provides a single application interface to access different services provided by different third-party cloud service providers. Background Technology

[0004] Over the past few years, the adoption of cloud services has risen dramatically, and this trend is only expected to continue. Different cloud service providers (CSPs) offer a variety of different cloud environments, each providing a set of one or more cloud services. The set of cloud services offered by a cloud environment may include one or more different types of services, including but not limited to Software as a Service (SaaS), Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and other services.

[0005] While various cloud environments exist, each offers a closed ecosystem for its subscribers. Therefore, customers of a cloud environment are typically limited to the services provided by that environment. To enable customers subscribing to a specific cloud environment offered by a particular cloud service provider (CSP) (e.g., a first cloud environment offered by a first CSP) to use / access services offered in different cloud environments offered by different CSPs, considerable effort is required to develop and support each service. Specifically, for each service offered by another CSP (also referred to herein as a third-party cloud provider) and used by customers of the first cloud environment, a unique API needs to be developed to support that service. Therefore, significant effort is required in defining interfaces, permissions, obtaining peer and security audits, and building new control planes. This needs to be done for each service offered by a third-party cloud provider, making it a time-consuming and costly task.

[0006] The embodiments discussed herein address these and other issues. Summary of the Invention

[0007] This disclosure generally relates to improved cloud architectures, and more specifically to a multi-cloud gateway that provides a single application interface to access different services offered by different third-party cloud service providers. Various embodiments are described herein, including methods, systems, and non-transitory computer-readable storage media storing programs, code, or instructions executable by one or more processors. Some embodiments may be implemented using a computer program product comprising computer programs / instructions that, when executed by a processor, cause the processor to perform any of the methods described in this disclosure.

[0008] One embodiment of this disclosure relates to a method comprising: receiving a first request by a multi-cloud gateway (MCG) implemented in a first cloud environment, the first request requesting the execution of a first operation in a second cloud environment; in response to receiving the first request, generating a first API call directed to the second cloud environment by the MCG; transmitting the first API call to the second cloud environment by the MCG; receiving a second request by the MCG, the second request requesting the execution of a second operation in a third cloud environment; generating a second API call directed to the third cloud environment by the MCG in response to receiving the second request; and transmitting the second API call to the third cloud environment by the MCG, wherein each of the first cloud environment, the second cloud environment, and the third cloud environment is provided by a different cloud service provider.

[0009] One aspect of this disclosure provides a computing device including one or more data processors and a non-transitory computer-readable storage medium containing instructions that, when executed on the one or more data processors, cause the computing device to perform some or all of the methods disclosed herein.

[0010] Another aspect of this disclosure provides a computer program product tangibly embodied in a non-transitory machine-readable storage medium, including instructions configured to cause one or more data processors to perform some or all of the methods disclosed herein.

[0011] The foregoing and other features and embodiments will become clearer with reference to the following description, claims and drawings. Attached Figure Description

[0012] The features, embodiments, and advantages of this disclosure can be better understood by reading the following detailed description with reference to the accompanying drawings.

[0013] Figure 1 This is a high-level diagram illustrating, according to certain embodiments, a distributed environment of a virtual or overlay cloud network hosted by a cloud service provider infrastructure.

[0014] Figure 2A simplified architecture diagram of the physical components in the physical network within the CSPI according to certain embodiments is depicted.

[0015] Figure 3 An example arrangement within a CSPI according to certain embodiments is shown, in which a host is connected to multiple network virtualization devices (NVDs).

[0016] Figure 4 The diagram illustrates connectivity between multi-tenant hosts and NVDs, according to certain embodiments, for providing I / O virtualization to support multi-tenant hosts.

[0017] Figure 5 A simplified block diagram of a physical network provided by CSPI according to certain embodiments is depicted.

[0018] Figure 6 A simplified high-level diagram of a distributed environment according to certain embodiments is depicted, the distributed environment comprising multiple cloud environments provided by different cloud service providers (CSPs), wherein the cloud environments include a specific cloud environment that provides specialized infrastructure that enables one or more cloud services provided by the specific cloud environment to be used by customers of other cloud environments.

[0019] Figure 7 An exemplary high-level architecture of a multi-cloud gateway (MCG) according to some embodiments is described.

[0020] Figure 8 An exemplary request sent to a multi-cloud gateway according to some embodiments is described.

[0021] Figure 9 Exemplary flowcharts according to some embodiments are depicted, illustrating the steps performed when a user associated with a first cloud environment executes a request for an operation to be performed on resources / services provided by a second cloud environment.

[0022] Figure 10 An exemplary flowchart depicting the steps performed by an MCG according to some embodiments is shown.

[0023] Figure 11 This is a block diagram illustrating a pattern for implementing a cloud infrastructure-as-a-service system according to at least one embodiment.

[0024] Figure 12 This is a block diagram illustrating another pattern for implementing a cloud infrastructure-as-a-service system according to at least one embodiment.

[0025] Figure 13 This is a block diagram illustrating another pattern for implementing a cloud infrastructure-as-a-service system according to at least one embodiment.

[0026] Figure 14 This is a block diagram illustrating another pattern for implementing a cloud infrastructure-as-a-service system according to at least one embodiment.

[0027] Figure 15 This is a block diagram illustrating an example computer system according to at least one embodiment. Detailed Implementation

[0028] In the following description, specific details are set forth for illustrative purposes in order to provide a thorough understanding of certain embodiments. However, it will be clear that various embodiments may be practiced without these specific details. The accompanying drawings and descriptions are not intended to be limiting. The word “exemplary” is used herein to mean “serves as an example, instance, or illustration.” Any embodiment or design described herein as “exemplary” is not necessarily to be construed as being more preferred or advantageous than other embodiments or designs.

[0029] This disclosure generally relates to improved cloud architectures, and more specifically to a multi-cloud gateway that provides a single application interface to access different services offered by different third-party cloud service providers. Various embodiments are described herein, including methods, systems, and non-transitory computer-readable storage media storing programs, code, or instructions executable by one or more processors. Some embodiments may be implemented using a computer program product comprising computer programs / instructions that, when executed by a processor, cause the processor to perform any of the methods described in this disclosure.

[0030] Cloud Network Example

[0031] The term cloud service generally refers to services provided by a cloud service provider (CSP) to users or customers on demand (e.g., via a subscription model) using systems and infrastructure (cloud infrastructure) provided by the CSP. Typically, the servers and systems that make up the CSP's infrastructure are separate from the customer's own on-premises servers and systems. Therefore, customers can utilize cloud services provided by the CSP without having to purchase separate hardware and software resources for the service. Cloud services are designed to provide subscribers with simple, scalable access to applications and computing resources without requiring customers to invest in the infrastructure used to provide the service.

[0032] Several cloud service providers offer various types of cloud services. There are various different types or models of cloud services, including Software as a Service (SaaS), Platform as a Service (PaaS), Infrastructure as a Service (IaaS), etc.

[0033] Customers can subscribe to one or more cloud services provided by a CSP. Customers can be any entity, such as individuals, organizations, or businesses. When a customer subscribes to or registers for a service provided by a CSP, a lease or account is created for that customer. The customer can then access one or more cloud resources associated with that account through their subscription.

[0034] As mentioned above, Infrastructure as a Service (IaaS) is a specific type of cloud computing service. In the IaaS model, a CSP provides customers with the infrastructure (called Cloud Service Provider Infrastructure or CSPI) that they can use to build their own customizable networks and deploy customer resources. Therefore, the customer's resources and network are hosted in a distributed environment by the infrastructure provided by the CSP. This differs from traditional computing, where the customer's resources and network are hosted by the infrastructure provided by the customer.

[0035] CSPI can include interconnected high-performance computing resources, including various host machines, memory resources, and network resources forming a physical network, also known as a substrate network or underlying network. Resources in CSPI can be distributed across one or more data centers, which may be geographically distributed across one or more geographic regions. Virtualization software can be executed by these physical resources to provide a virtualized distributed environment. Virtualization creates overlay networks (also known as software-based networks, software-defined networks, or virtual networks) on the physical network. The CSPI physical network provides the underlying foundation for creating one or more overlay or virtual networks on top of the physical network. The physical network (or substrate network or underlying network) includes physical network devices such as physical switches, routers, computers, and host machines. An overlay network is a logical (or virtual) network that runs on top of the physical substrate network. A given physical network can support one or more overlay networks. Overlay networks typically use encapsulation techniques to distinguish traffic belonging to different overlay networks. Virtual or overlay networks are also known as Virtual Cloud Networks (VCNs). Virtual networks are created using software virtualization technologies (e.g., hypervisors, virtualization functions implemented by network virtualization devices (NVDs) (e.g., smartNICs), top-of-rack (TOR) switches, intelligent TORs that implement one or more functions performed by NVDs, and other mechanisms) to create a network abstraction layer that can run on top of a physical network. Virtual networks can take many forms, including peer-to-peer networks, IP networks, etc. Virtual networks are typically Layer 3 IP networks or Layer 2 VLANs. This method of virtual or overlay networking is often referred to as virtual or overlay Layer 3 networking. Examples of protocols developed for virtual networks include IP-in-IP (or Generic Routing Encapsulation (GRE)), Virtual Extensible LAN (VXLAN—IETF RFC 7348), Virtual Private Networks (VPNs) (e.g., MPLS Layer 3 Virtual Private Network (RFC 4364)), VMware's NSX, GENEVE (Generic Network Virtualization Encapsulation), etc.

[0036] For IaaS, the infrastructure provided by a CSP (Center for Service Providers) can be configured to offer virtualized computing resources over a public network (e.g., the Internet). In the IaaS model, cloud service providers can host infrastructure components (e.g., servers, storage devices, network nodes (e.g., hardware), deployment software, platform virtualization (e.g., hypervisor layer), etc.). In some cases, IaaS providers can also provision various services accompanying those infrastructure components (e.g., billing, monitoring, logging, security, load balancing, and clustering, etc.). Therefore, since these services can be policy-driven, IaaS users can implement policies to drive load balancing to maintain application availability and performance. CSPI provides infrastructure and a set of complementary cloud services, enabling customers to build and run a wide range of applications and services in a highly available, hosted, distributed environment. CSPI provides high-performance computing resources and capabilities, as well as storage capacity, in a flexible virtual network securely accessible from various networked locations, such as the customer's local network. When a customer subscribes to or enrolls in an IaaS service provided by a CSP, the lease created for that customer is a secure and isolated partition within the CSP, where the customer can create, organize, and manage their cloud resources.

[0037] Customers can build their own virtual networks using the compute, storage, and networking resources provided by CSPI. One or more customer resources or workloads, such as compute instances, can be deployed on these virtual networks. For example, a customer can use resources provided by CSPI to build one or more customizable and private virtual networks, called Virtual Cloud Networks (VCNs). A customer can deploy one or more customer resources, such as compute instances, on a customer VCN. Compute instances can take the form of virtual machines, bare metal instances, etc. Therefore, CSPI provides the infrastructure and a complementary set of cloud services that enable customers to build and run a wide range of applications and services in a highly available, virtually managed environment. Customers do not manage or control the underlying physical resources provided by CSPI, but they have control over the operating system, storage, and deployed applications; and may also have limited control over chosen networking components (e.g., firewalls).

[0038] The CSP can provide a console that enables customers and network administrators to configure, access, and manage resources deployed in the cloud using CSPI resources. In some embodiments, the console provides a web-based user interface that can be used to access and manage the CSPI. In other embodiments, the console is a web-based application provided by the CSP.

[0039] CSPI can support single-tenant or multi-tenant architectures. In a single-tenant architecture, software (e.g., applications, databases) or hardware components (e.g., host machines or servers) serve a single customer or tenant. In a multi-tenant architecture, the software or hardware components serve multiple customers or tenants. Therefore, in a multi-tenant architecture, CSPI resources are shared among multiple customers or tenants. In a multi-tenant scenario, precautions and safeguards are implemented in CSPI to ensure that each tenant's data is isolated and invisible to other tenants.

[0040] In a physical network, a network endpoint is a computing device or system that connects to and communicates with the physical network. Network endpoints in a physical network can connect to a Local Area Network (LAN), a Wide Area Network (WAN), or other types of physical networks. Examples of traditional endpoints in a physical network include modems, hubs, bridges, switches, routers and other network devices, physical computers (or host machines), etc. Each physical device in a physical network has a fixed network address that can be used to communicate with the device. This fixed network address can be a Layer 2 address (e.g., a MAC address), a fixed Layer 3 address (e.g., an IP address), etc. In a virtualized environment or virtual network, endpoints can include various virtual endpoints, such as virtual machines hosted by components of the physical network (e.g., hosted by physical host machines). These endpoints in a virtual network are addressed using overlay addresses, such as overlay Layer 2 addresses (e.g., overlay MAC addresses) and overlay Layer 3 addresses (e.g., overlay IP addresses). Network overlays provide flexibility by allowing network administrators to move around the overlay addresses associated with network endpoints using software management (e.g., via software implementing a control plane for virtual networks). Therefore, unlike physical networks, in virtual networks, overlay addresses (e.g., overlay IP addresses) can be moved from one endpoint to another using network management software. Since virtual networks are built on top of physical networks, communication between components within a virtual network involves both the virtual network and the underlying physical network. To facilitate this communication, CSPI components are configured to learn and store mappings that map overlay addresses in the virtual network to actual physical addresses in the baseboard network, and vice versa. These mappings are then used to facilitate communication. Client traffic is encapsulated to facilitate routing within the virtual network.

[0041] Therefore, physical addresses (e.g., physical IP addresses) are associated with components in a physical network, and overlay addresses (e.g., overlay IP addresses) are associated with entities in a virtual or overlay network. A physical IP address is an IP address associated with a physical device (e.g., a network device) in a substrate or physical network. For example, each NVD has an associated physical IP address. An overlay IP address is an overlay address associated with an entity in an overlay network, such as an overlay address associated with a compute instance in a customer's Virtual Cloud Network (VCN). Two different customers or tenants, each with their own private VCN, can potentially use the same overlay IP address in their VCN without knowing about each other. Both physical IP addresses and overlay IP addresses are types of real IP addresses. They are separate from virtual IP addresses. A virtual IP address is typically a single IP address that represents or maps to multiple real IP addresses. A virtual IP address provides a one-to-many mapping between virtual IP addresses and multiple real IP addresses. For example, a load balancer can use a VIP to map or represent multiple servers, each with its own real IP address.

[0042] Cloud infrastructure, or CSPI, is physically hosted in one or more data centers in one or more regions of the world. CSPI may include components in a physical or substrate network and virtualized components (e.g., virtual networks, compute instances, virtual machines, etc.) in a virtual network built on top of the physical network components. In some embodiments, CSPI is organized and hosted in domains, regions, and availability domains. A region is typically a localized geographical area containing one or more data centers. Regions are generally independent of each other and can be geographically distant, for example, spanning countries or even continents. For example, one region might be in Australia, another in Japan, another in India, and so on. CSPI resources are partitioned between regions, such that each region has its own independent subset of CSPI resources. Each region can provide a set of core infrastructure services and resources, such as compute resources (e.g., bare metal servers, virtual machines, containers, and related infrastructure); storage resources (e.g., block volume storage, file storage, object storage, archive storage); networking resources (e.g., virtual cloud networks (VCNs), load balancing resources, connectivity to local networks); database resources; edge networking resources (e.g., DNS); and access management and monitoring resources, etc. Each region typically has multiple paths connecting it to other regions within the domain.

[0043] Generally, applications are deployed in the areas where they are used most frequently (i.e., on the infrastructure associated with that area) because using nearby resources is faster than using resources far away. Applications may also be deployed in different areas for various reasons, such as redundancy to mitigate the risk of events within a region (such as large weather systems or earthquakes), or to meet different requirements such as legal jurisdiction, tax domain, and other business or social standards.

[0044] Data centers within a region can be further organized and subdivided into Availability Domains (ADs). An Availability Domain can correspond to one or more data centers located within the region. A region can consist of one or more Availability Domains. In this distributed environment, CSPI resources are either region-specific, such as Virtual Cloud Networks (VCNs), or Availability Domain-specific, such as compute instances.

[0045] Availability Zones (ADs) within a region are isolated from each other, fault-tolerant, and configured to make simultaneous failures highly unlikely. This is achieved by ensuring that ADs do not share critical infrastructure resources (such as networking, physical cabling, cable paths, cable entry points, etc.), making a failure at one AD within a region unlikely to affect the availability of other ADs in the same region. ADs within the same region can be interconnected via low-latency, high-bandwidth networks, enabling highly available connectivity to other networks (e.g., the internet, customer local networks, etc.) and allowing for replication systems across multiple ADs to achieve both high availability and disaster recovery. Cloud services use multiple ADs to ensure high availability and prevent resource failures. As the infrastructure provided by IaaS providers grows, more regions and ADs, along with additional capacity, can be added. Traffic between availability domains is typically encrypted.

[0046] In some embodiments, regions are grouped into domains. A domain is a logical collection of regions. Domains are isolated from each other and do not share any data. Regions within the same domain can communicate with each other, but regions in different domains cannot. A customer's lease or account with the CSP resides in a single domain and can be distributed across one or more regions belonging to that domain. Typically, when a customer subscribes to an IaaS service, a lease or account is created for that customer in a region within the domain that the customer designates (called the "primary" region). A customer can extend their lease to one or more other regions within the domain. A customer cannot access regions that are not within the domain where their lease resides.

[0047] IaaS providers can offer multiple domains, each catering to a specific set of customers or users. For example, a business domain can be offered for business customers. As another example, a domain can be offered for customers within a specific country. As yet another example, a government domain can be offered for governments, and so on. For instance, a government domain can cater to a specific government and may have a higher level of security than the business domain. For example, Oracle Cloud Infrastructure (OCI) currently offers domains for its business region and two domains for its government cloud region (e.g., FedRAMP licensed and IL5 licensed).

[0048] In some embodiments, an Active Directory (AD) can be subdivided into one or more fault domains. A fault domain is a grouping of infrastructure resources within an AD to provide anti-affinity. Fault domains allow for the distribution of compute instances such that these instances do not reside on the same physical hardware within a single AD. This is known as anti-affinity. A fault domain refers to a group of hardware components (computers, switches, etc.) that share a single point of failure. Compute pools are logically divided into fault domains. Therefore, a hardware failure or compute hardware maintenance event affecting one fault domain does not affect instances in other fault domains. Depending on the embodiment, the number of fault domains per AD can vary. For example, in some embodiments, each AD contains three fault domains. Fault domains act as logical data centers within the AD.

[0049] When a customer subscribes to IaaS services, resources from CSPI are provisioned to the customer and associated with the customer's lease. Customers can use these provisioned resources to build private networks and deploy resources on those networks. Customer networks hosted in the cloud by CSPI are called Virtual Cloud Networks (VCNs). Customers can use the CSPI resources allocated to them to set up one or more VCNs. A VCN is a virtual or software-defined private network. Customer resources deployed in a customer's VCN can include compute instances (e.g., virtual machines, bare metal instances) and other resources. These compute instances can represent various customer workloads, such as applications, load balancers, databases, etc. Compute instances deployed on a VCN can communicate with publicly accessible endpoints (“public endpoints”) via public networks (such as the Internet), with other instances in the same VCN or other VCNs (e.g., other VCNs belonging to the customer or VCNs not belonging to the customer), with the customer's on-premises data center or network, and with service endpoints and other types of endpoints.

[0050] CSPs can use CSPIs to provide various services. In some cases, CSPI clients can act as service providers themselves and use CSPI resources to provide services. Service providers can expose service endpoints characterized by identification information such as IP addresses, DNS names, and ports. Client resources (e.g., compute instances) can access a specific service by accessing the service endpoints exposed by the service for that specific service. These service endpoints are generally publicly accessible to users via public communication networks (such as the Internet) using the public IP addresses associated with the endpoints. Publicly accessible network endpoints are sometimes also called public endpoints.

[0051] In some embodiments, a service provider may expose the service via an endpoint used for the service (sometimes referred to as a service endpoint). Customers of the service can then use this service endpoint to access the service. In some implementations, the service endpoint provided for the service can be accessed by multiple customers intending to consume the service. In other implementations, a dedicated service endpoint can be provided to a customer, so that only that customer can use that dedicated service endpoint to access the service.

[0052] In some embodiments, when a VCN is created, it is associated with a Private Overlay Classless Inter-Domain Routing (CIDR) address space, which is a set of private overlay IP addresses (e.g., 10.0 / 16) assigned to the VCN. A VCN includes associated subnets, routing tables, and gateways. A VCN resides within a single area but can span one or more of the availability domains within that area. A gateway is a virtual interface configured for the VCN and enables traffic to and from the VCN to one or more endpoints outside the VCN. One or more different types of gateways can be configured for a VCN to enable communication to and from different types of endpoints.

[0053] A VCN can be subdivided into one or more subnets, such as one or more subnets. Therefore, a subnet is a configurable unit or subdivision that can be created within a VCN. A VCN can have one or more subnets. Each subnet within a VCN is associated with a contiguous range of overlay IP addresses (e.g., 10.0.0.0 / 24 and 10.0.1.0 / 24), which do not overlap with other subnets within that VCN and represent a subset of the VCN's address space.

[0054] Each compute instance is associated with a Virtual Network Interface Card (VNIC), which enables the compute instance to participate in subnets within a VCN. A VNIC is the logical representation of a physical network interface card (NIC). Generally, a VNIC is the interface between an entity (e.g., a compute instance, a service) and a virtual network. A VNIC exists within a subnet and has one or more associated IP addresses, along with associated security rules or policies. A VNIC is equivalent to a Layer 2 port on a switch. A VNIC is attached to both the compute instance and the subnet within the VCN. The VNIC associated with a compute instance enables the compute instance to become part of a VCN's subnet and allows the compute instance to communicate (e.g., send and receive packets) with endpoints on the same subnet as the compute instance, with endpoints in different subnets within the VCN, or with endpoints outside the VCN. Therefore, the VNIC associated with a compute instance determines how the compute instance connects to endpoints inside and outside the VCN. When a compute instance is created and added to a subnet within a VCN, a VNIC is created for the compute instance and associated with that compute instance. For a subnet that includes a set of compute instances, the subnet contains a VNIC corresponding to that set of compute instances, and each VNIC is attached to a compute instance within that set of compute instances.

[0055] A private overlay IP address is assigned to each compute instance via the VNIC associated with it. This private overlay network IP address is assigned to the VNIC associated with the compute instance when the compute instance is created and is used to route traffic to and from the compute instance. All VNICs within a given subnet use the same routing table, security list, and DHCP options. As described above, each subnet within a VCN is associated with a contiguous range of overlay IP addresses (e.g., 10.0.0.0 / 24 and 10.0.1.0 / 24) that do not overlap with other subnets within that VCN and represent a subset of the address space within the VCN's address space. For a VNIC on a specific subnet of a VCN, the private overlay IP address assigned to that VNIC is an address derived from the contiguous range of overlay IP addresses allocated to the subnet.

[0056] In some embodiments, in addition to a private overlay IP address, a compute instance may optionally be assigned additional overlay IP addresses, such as one or more public IP addresses, for example, if in a public subnet. These multiple addresses are assigned either on the same VNIC or on multiple VNICs associated with the compute instance. However, each instance has a primary VNIC, which is created during instance startup and associated with the overlay private IP address assigned to that instance—this primary VNIC cannot be deleted. Additional VNICs, called secondary VNICs, can be added to an existing instance in the same availability domain as the primary VNIC. All VNICs are in the same availability domain as the instance. The secondary VNIC can be located in a subnet within the same VCN as the primary VNIC, or in a different subnet within the same VCN or different VCNs.

[0057] If a compute instance is in a public subnet, it can optionally be assigned a public IP address. When creating a subnet, it can be specified as either a public or private subnet. A private subnet means that resources within the subnet (such as compute instances) and associated VNICs cannot have public overriding IP addresses. A public subnet means that resources within the subnet and associated VNICs can have public IP addresses. Customers can specify that a subnet exists within a single availability domain or across multiple availability domains in a region or domain.

[0058] As described above, a VCN can be subdivided into one or more subnets. In some embodiments, a virtual router (VR) configured for a VCN (referred to as a VCN VR or simply VR) enables communication between the subnets of the VCN. For a subnet within a VCN, the VR represents a logical gateway for that subnet, enabling that subnet (i.e., compute instances on that subnet) to communicate with endpoints on other subnets within the VCN as well as with other endpoints outside the VCN. The VCN VR is a logical entity configured to route traffic between the VNIC within the VCN and the virtual gateway (“gateway”) associated with the VCN. The following section discusses… Figure 1Further description of the gateway. A VCNVR is a Layer 3 / IP layer concept. In one embodiment, there exists a VCN VR for a VCN, where the VCN VR potentially has an unlimited number of ports addressable via IP addresses, one port for each subnet of the VCN. In this way, the VCN VR has a different IP address for each subnet within the VCN to which the VCN VR is attached. The VR also connects to various gateways configured for the VCN. In some embodiments, a specific overlay IP address within a range of overlay IP addresses for a subnet is reserved for the port of the VCN VR for that subnet. For example, consider a VCN with two subnets, associated with address ranges 10.0 / 16 and 10.1 / 16. For the first subnet in the VCN with an address range of 10.0 / 16, addresses within this range are reserved for the port of the VCN VR for that subnet. In some cases, the first IP address within the range can be reserved for the VCN VR. For example, for a subnet with an overlay IP address range of 10.0 / 16, the IP address 10.0.0.1 could be reserved for the port of the VCN VR for that subnet. For a second subnet within the same VCN with an address range of 10.1 / 16, the VCN VR can have a port with IP address 10.1.0.1 for the second subnet. The VCN VR has a different IP address for each subnet within the VCN.

[0059] In some other embodiments, each subnet within a VCN may have its own associated VR, which can be addressed by the subnet using a reserved or default IP address associated with the VR. For example, the reserved or default IP address may be the first IP address in the range of IP addresses associated with the subnet. The VNIC in the subnet can use this default or reserved IP address to communicate with the VR associated with the subnet (e.g., send and receive packets). In this embodiment, the VR is the ingress / egress point of the subnet. The VR associated with a subnet within the VCN can communicate with other VRs associated with other subnets within the VCN. The VR can also communicate with the gateway associated with the VCN. The VR functionality of the subnet runs on or is performed by one or more NVDs that perform the VNIC functionality of the VNICs within the subnet.

[0060] You can configure routing tables, security rules, and DHCP options for a VCN. A routing table is a virtual routing table used by the VCN and contains rules that route traffic from subnets within the VCN to destinations outside the VCN via gateways or specially configured instances. The VCN's routing table can be customized to control how packets are forwarded / routed to and from the VCN. DHCP options refer to configuration information automatically provided to the instance when it starts up.

[0061] Security rules configured for a VCN represent overlay firewall rules used by the VCN. Security rules can include ingress and egress rules, specifying the types of traffic allowed to enter and exit instances within the VCN (e.g., based on protocol and port). Clients can choose whether a given rule is stateful or stateless. For example, a client can allow incoming SSH traffic from anywhere to a set of instances by setting a stateful ingress rule with source CIDR 0.0.0.0 / 0 and destination TCP port 22. Security rules can be implemented using network security groups or security lists. A network security group consists of a set of security rules that apply only to resources within that group. A security list, on the other hand, includes rules applicable to all resources in any subnet using that security list. A default security list with default security rules can be provided to the VCN. DHCP options configured for the VCN provide configuration information that is automatically provided to instances within the VCN when the instance starts.

[0062] In some embodiments, configuration information for a VCN is determined and stored by the VCN control plane. For example, the configuration information for a VCN may include information about: the address range associated with the VCN, subnets and associated information within the VCN, one or more VRs associated with the VCN, compute instances and associated VNICs within the VCN, NVDs (e.g., VNICs, VRs, gateways) that perform various virtualization network functions associated with the VCN, status information for the VCN, and other VCN-related information. In some embodiments, the VCN distribution service publishes the configuration information or portions thereof stored by the VCN control plane to the NVD. The distributed information can be used to update information stored by the NVD and used to forward packets to and from compute instances within the VCN (e.g., forwarding tables, routing tables, etc.).

[0063] In some embodiments, the creation of VCNs and subnets is handled by the VCN control plane (CP), and the initiation of compute instances is handled by the compute control plane. The compute control plane is responsible for allocating physical resources to the compute instances and then invoking the VCN control plane to create VNICs and attach them to the compute instances. The VCN CP also maps VCN data to the VCN data plane, which is configured to perform packet forwarding and routing functions. In some embodiments, the VCN CP provides a distribution service responsible for providing updates to the VCN data plane. Examples of VCN control planes are also available in... Figure 6 , Figure 7 , Figure 8 and Figure 9 The figures are depicted in (see reference numerals 616, 716, 816 and 916) and described below.

[0064] Customers can create one or more VCNs using resources hosted by CSPI. Compute instances deployed on a customer's VCN can communicate with different endpoints. These endpoints can include endpoints hosted by CSPI and endpoints outside of CSPI.

[0065] Various architectures are used to implement cloud-based services using CSPI. Figure 1 , Figure 2 , Figure 3 , Figure 4 , Figure 5 As depicted in Figures 18-22 and described below. Figure 1 This is a high-level diagram of a distributed environment 100, illustrating an overlay or client VCN hosted by CSPI according to certain embodiments. Figure 1 The distributed environment described includes multiple components in the overlay network. Figure 1 The distributed environment 100 depicted herein is merely an example and is not intended to unduly limit the scope of the claimed embodiments. Many variations, substitutions, and modifications are possible. For example, in some implementations, Figure 1 The distributed environment described in the text can have more than Figure 1 The more or fewer systems or components shown can be combined into two or more systems, or can have different system configurations or arrangements.

[0066] like Figure 1 As illustrated in the example, distributed environment 100 includes a CSPI 101 that provides services and resources that customers can subscribe to and use to build their Virtual Cloud Network (VCN). In some embodiments, CSPI 101 provides IaaS services to subscribing customers. Data centers within CSPI 101 can be organized into one or more regions. Figure 1 The example shown is Region US 102. The customer has already configured a customer VCN c / o Oracle International for Region 102. The customer can deploy various compute instances on VCN 104, which can include virtual machines or bare metal instances. Examples of instances include applications, databases, load balancers, etc.

[0067] exist Figure 1 In the embodiment depicted, customer VCN 104 includes two subnets, namely "Subnet-1" and "Subnet-2", each with its own CIDR IP address range. Figure 1In this configuration, subnet-1 covers the IP address range of 10.0 / 16, and subnet-2 covers the address range of 10.1 / 16. VCN Virtual Router 105 represents a logical gateway for the VCN, enabling communication between subnets within VCN 104 and with other endpoints outside the VCN. VCN VR 105 is configured to route traffic between the VNICs within VCN 104 and the gateway associated with VCN 104. VCN VR 105 provides a port for each subnet of VCN 104. For example, VR 105 could provide a port with IP address 10.0.0.1 for subnet-1 and a port with IP address 10.1.0.1 for subnet-2.

[0068] Multiple compute instances can be deployed on each subnet, where compute instances can be virtual machine instances and / or bare metal instances. Compute instances within a subnet can be hosted by one or more host machines within CSPI 101. Compute instances participate in the subnet via a VNIC associated with the compute instance. For example, as... Figure 1 As shown, compute instance C1 becomes part of subnet-1 via the VNIC associated with it. Similarly, compute instance C2 becomes part of subnet-1 via the VNIC associated with it. In a similar manner, multiple compute instances (which can be virtual machine instances or bare metal instances) can be part of subnet-1. Each compute instance is assigned a private overlay IP address and MAC address via its associated VNIC. For example, in... Figure 1 In this context, compute instance C1 has an overlay IP address of 10.0.0.2 and a MAC address of M1, while compute instance C2 has a private overlay IP address of 10.0.0.3 and a MAC address of M2. Each compute instance in subnet-1 (including compute instances C1 and C2) has a default route to VCN VR 105 using IP address 10.0.0.1, which is the IP address of the port used by VCN VR 105 in subnet-1.

[0069] Multiple compute instances, including virtual machine instances and / or bare metal instances, can be deployed on subnet-2. For example, such as Figure 1 As shown, compute instances D1 and D2 become part of subnet-2 via the VNIC associated with the respective compute instance. Figure 1 In the illustrated embodiment, compute instance D1 has an overlay IP address of 10.1.0.2 and a MAC address of MM1, while compute instance D2 has a private overlay IP address of 10.1.0.3 and a MAC address of MM2. Each compute instance in subnet-2 (including compute instances D1 and D2) has a default route to VCN VR 105 using IP address 10.1.0.1, which is the IP address of the port of VCN VR105 in subnet-2.

[0070] VCN A 104 may also include one or more load balancers. For example, a load balancer can be provided for a subnet, and the load balancer can be configured to load balance traffic across multiple compute instances on the subnet. Load balancers can also be provided to load balance traffic across subnets within a VCN.

[0071] A specific compute instance deployed on VCN 104 can communicate with a variety of different endpoints. These endpoints can include endpoints hosted by CSPI 200 and endpoints outside of CSPI 200. Endpoints hosted by CSPI 101 can include: endpoints on the same subnet as the specific compute instance (e.g., communication between two compute instances in subnet-1); endpoints on different subnets but within the same VCN (e.g., communication between a compute instance in subnet-1 and a compute instance in subnet-2); endpoints in different VCNs within the same region (e.g., communication between a compute instance in subnet-1 and an endpoint in a VCN within the same region 106 or 110, or communication between a compute instance in subnet-1 and an endpoint in service point 110 within the same region); or endpoints in VCNs in different regions (e.g., communication between a compute instance in subnet-1 and an endpoint in a VCN within a different region 108). Compute instances in subnets hosted by CSPI 101 can also communicate with endpoints not hosted by CSPI 101 (i.e., outside of CSPI 101). These external endpoints include endpoints in the customer’s local network 116, endpoints in other remote cloud-hosted networks 118, public endpoints 114 that are accessible via public networks such as the Internet, and other endpoints.

[0072] VNICs associated with source and destination compute instances facilitate communication between compute instances on the same subnet. For example, compute instance C1 in subnet-1 might want to send packets to compute instance C2 in subnet-1. For packets originating from the source compute instance and destined for another compute instance in the same subnet, the packet is first processed by the VNIC associated with the source compute instance. Processing performed by the VNIC associated with the source compute instance may include determining the packet's destination information from the packet header, identifying any policies (e.g., security lists) configured for the VNIC associated with the source compute instance, determining the packet's next hop, performing any packet encapsulation / decapsulation functions as needed, and then forwarding / routing the packet to the next hop to facilitate communication to its intended destination. When the destination compute instance is in the same subnet as the source compute instance, the VNIC associated with the source compute instance is configured to identify the VNIC associated with the destination compute instance and forward packets to that VNIC for processing. The VNIC associated with the destination compute instance then performs the forwarding and forwarding of packets to the destination compute instance.

[0073] For packets to be transferred from compute instances in a subnet to endpoints in different subnets within the same VCN, communication is facilitated by the VNIC associated with the source and destination compute instances, as well as the VCN VR. For example, if Figure 1 If compute instance C1 in subnet-1 wants to send a packet to compute instance D1 in subnet-2, the packet is first processed by the VNIC associated with compute instance C1. The VNIC associated with compute instance C1 is configured to route the packet to VCN VR 105 using the default route or port 10.0.0.1 of the VCN VR. VCN VR 105 is configured to route the packet to subnet-2 using port 10.1.0.1. Then, the VNIC associated with D1 receives and processes the packet and forwards it to compute instance D1.

[0074] For packets to be transferred from a compute instance in VCN 104 to an endpoint outside VCN 104, communication is facilitated by the VNIC associated with the source compute instance, VCN VR 105, and the gateway associated with VCN 104. One or more types of gateways can be associated with VCN 104. A gateway is an interface between a VCN and another endpoint located outside the VCN. A gateway is a Layer 3 / IP concept and enables a VCN to communicate with endpoints outside the VCN. Therefore, a gateway facilitates traffic flow between a VCN and other VCNs or networks. Various different types of gateways can be configured for a VCN to facilitate different types of communication with different types of endpoints. Depending on the gateway, communication can be conducted over a public network (e.g., the Internet) or over a private network. Various communication protocols can be used for these communications.

[0075] For example, compute instance C1 might want to communicate with an endpoint outside of VCN 104. The packet can first be processed by the VNIC associated with the source compute instance C1. The VNIC processing determines that the packet's destination is outside C1's subnet-1. The VNIC associated with C1 can then forward the packet to VCN VR 105 for VCN 104. VCN VR 105 then processes the packet and, as part of the processing, determines a specific gateway associated with VCN 104 as the next hop for the packet based on its destination. VCN VR 105 can then forward the packet to the identified specific gateway. For example, if the destination is an endpoint within a customer's local network, the packet can be forwarded by VCN VR 105 to the Dynamic Routing Gateway (DRG) gateway 122 configured for VCN 104. The packet can then be forwarded from the gateway to the next hop to facilitate delivery to its final intended destination.

[0076] Various types of gateways can be configured for a VCN. Examples of gateways that can be configured for a VCN are available in [link to example]. Figure 1 Examples of gateways associated with the VCN are also depicted in Figures 18, 19, 20, and 21 (e.g., gateways referenced by reference numerals 1834, 1836, 1838, 1934, 1936, 1938, 2034, 2036, 2038, 2134, 2136, and 2138) and described below. Figure 1As depicted in the embodiments, a Dynamic Routing Gateway (DRG) 122 can be added to or associated with a customer VCN 104 and provides a path for private network traffic communication between the customer VCN 104 and another endpoint, which can be the customer's local network 116, a VCN 108 in a different region of CSPI 101, or another remote cloud network 118 not hosted by CSPI 101. The customer's local network 116 can be a customer network or customer data center built using the customer's resources. Access to the customer's local network 116 is generally very restricted. For customers who have both a customer's local network 116 and one or more VCNs 104 deployed or hosted in the cloud by CSPI 101, the customer may want their local network 116 and their cloud-based VCN 104 to be able to communicate with each other. This allows customers to build extended hybrid environments encompassing the customer's VCN 104 hosted by CSPI 101 and their local network 116. The DRG 122 enables this communication. To enable this type of communication, a communication channel 124 is established, with one endpoint located in the customer's local network 116 and the other endpoint located in CSPI 101 and connected to the customer's VCN 104. Communication channel 124 can be via a public communication network (such as the Internet) or a private communication network. Various different communication protocols can be used, such as IPsec VPN technology over a public communication network (such as the Internet), Oracle's FastConnect technology using a private network instead of a public network, etc. The device or equipment forming one endpoint of communication channel 124 in the customer's local network 116 is called a Customer Premises Equipment (CPE), such as... Figure 1 The CPE 126 is depicted in the diagram. On the CSPI 101 side, the endpoint can be the host machine executing DRG 122.

[0077] In some embodiments, a remote peering connection (RPC) can be added to the DRG, which allows a customer to peer one VCN with another VCN in a different region. Using such an RPC, customer VCN 104 can use DRG 122 to connect to VCN 108 in another region. DRG 122 can also be used to communicate with other remote cloud networks 118 not hosted by CSPI 101, such as Microsoft Azure Cloud, Amazon AWS Cloud, etc.

[0078] like Figure 1As shown, an Internet Gateway (IGW) 120 can be configured for customer VCN 104, enabling compute instances on VCN 104 to communicate with a public endpoint 114 accessible via a public network such as the Internet. IGW 120 is a gateway connecting the VCN to a public network such as the Internet. IGW 120 allows public subnets within the VCN (such as VCN 104), where resources have publicly overriding IP addresses, to directly access a public endpoint 112 on the public network 114 (such as the Internet). Using IGW 120, connections can be initiated from subnets within VCN 104 or from the Internet.

[0079] Network Address Translation (NAT) gateway 128 can be configured for a customer's VCN 104, enabling cloud resources within the customer's VCN that do not have dedicated public overlay IP addresses to access the Internet, and it does so without exposing those resources to direct inbound Internet connections (e.g., L4-L7 connections). This allows private subnets within the VCN (such as private subnet-1 in VCN 104) to privately access public endpoints on the Internet. In a NAT gateway, connections can only be initiated from private subnets to the public Internet, and not from the Internet to private subnets.

[0080] In some embodiments, a Service Gateway (SGW) 126 may be configured for a client VCN 104 and provide a path for private network traffic between VCN 104 and service endpoints supported in service network 110. In some embodiments, service network 110 may be provided by a CSP and may offer a variety of services. An example of such a service network is Oracle's service network, which provides a variety of services available to clients. For example, compute instances (e.g., database systems) in a private subnet of client VCN 104 may back up data to service endpoints (e.g., object storage) without requiring a public IP address or access to the Internet. In some embodiments, a VCN may have only one SGW, and connections may only originate from subnets within the VCN, not from service network 110. If a VCN is peered to another, resources in the other VCN typically cannot access the SGW. Resources in the local network of a VCN connected using FastConnect or VPN Connect may also use the service gateway configured for that VCN.

[0081] In some implementations, the SGW 126 uses the concept of a Service Classless Inter-Domain Routing (CIDR) label, which is a string representing the range of all regional public IP addresses used for the service or group of services of interest. Customers use the Service CIDR label when configuring the SGW and associated routing rules to control traffic to the service. Customers can optionally use it when configuring security rules without needing to adjust those rules if the public IP addresses of the service change in the future.

[0082] Local peering gateway (LPG) 132 is a gateway that can be added to customer VCN 104 and enable VCN 104 to peer with another VCN in the same area. Peering refers to VCNs communicating using private IP addresses without traffic traversing public networks (such as the Internet) or routing traffic through the customer's local network 116. In a preferred embodiment, a VCN has a separate LPG for each peering it establishes. Local peering, or VCN peering, is a common practice for establishing network connectivity between different applications or infrastructure management functions.

[0083] Service providers (such as service providers in service network 110) can offer access to services using different access models. Under the public access model, a service can be exposed as a public endpoint accessible to compute instances within a customer's VCN via a public network (such as the Internet), and / or privately accessible via SGW 126. Under a specific private access model, a service can be accessed as a private IP endpoint within a private subnet of the customer's VCN. This is called Private Endpoint (PE) access and enables service providers to expose their services as instances within the customer's private network. A private endpoint resource represents a service within the customer's VCN. Each PE is represented as a VNIC (called a PE-VNIC, with one or more private IPs) within the customer's VCN in a subnet chosen by the customer. Thus, the PE provides a way to present services within a private customer VCN subnet using a VNIC. Because the endpoint is exposed as a VNIC, all characteristics associated with the VNIC (such as routing rules, security lists, etc.) are now available for the PE VNIC.

[0084] Service providers can register their services to enable access via PE. Providers can associate policies with services, which restricts the visibility of services to customer leases. Providers can register multiple services under a single Virtual IP address (VIP), especially for multi-tenant services. Multiple such private endpoints (across multiple VCNs) can represent the same service.

[0085] Compute instances in a private subnet can then access the service using the private IP address or service DNS name of the PE VNIC. Compute instances in a customer VCN can access the service by sending traffic to the private IP address of the PE in the customer VCN. A Private Access Gateway (PAGW) 130 is a gateway resource that can be attached to a service provider VCN (e.g., a VCN in service network 110), which acts as the ingress / egress point for all traffic originating from / to the private endpoint of the customer subnet. PAGW 130 allows providers to scale the number of PE connections without utilizing their internal IP address resources. A provider only needs to configure one PAGW for any number of services registered in a single VCN. A provider can represent a service as a private endpoint in multiple VCNs of one or more customers. From the customer's perspective, the PE VNIC is not an instance attached to the customer, but rather appears to be attached to the service the customer wishes to interact with. Traffic to the private endpoint is routed to the service via PAGW 130. These are referred to as customer-to-service private connections (C2S connections).

[0086] The PE concept can also be used to extend private access to services to the customer's on-premises network and data center by allowing traffic to flow through FastConnect / IPsec links and private endpoints within the customer's VCN. Furthermore, private access to services can be extended to the customer's peering VCN by allowing traffic to flow between the LPG132 and the PE within the customer's VCN.

[0087] Customers can control routing within a VCN at the subnet level, allowing them to specify which subnets within a customer's VCN (such as VCN104) use each gateway. The VCN's routing table is used to determine whether traffic is allowed to leave the VCN via a specific gateway. For example, in a given instance, the routing table for public subnets within customer VCN 104 might send non-local traffic via IGW 120. The routing table for private subnets within the same customer VCN 104 might send traffic destined for CSP services via SGW 126. All remaining traffic might be sent via NAT gateway 128. The routing table only controls traffic leaving the VCN.

[0088] Security lists associated with a VCN are used to control traffic entering the VCN via a gateway through inbound connections. All resources within a subnet use the same routing tables and security lists. Security lists can be used to control specific types of traffic allowed to enter or leave instances within a subnet of the VCN. Security list rules can include inbound and outbound rules. For example, inbound rules can specify allowed source address ranges, while outbound rules can specify allowed destination address ranges. Security rules can specify specific protocols (e.g., TCP, ICMP), specific ports (e.g., port 22 for SSH, port 3389 for Windows RDP), etc. In some implementations, the instance's operating system can enforce its own firewall rules that conform to the security list rules. Rules can be stateful (e.g., tracking connections and automatically allowing responses without explicit security list rules for response traffic) or stateless.

[0089] Access from a customer's VCN (i.e., through resources or compute instances deployed on VCN 104) can be categorized as public access, private access, or dedicated access. Public access refers to an access model that uses a public IP address or NAT to access a public endpoint. Private access enables customer workloads (e.g., resources in a private subnet) with private IP addresses within VCN 104 to access services without traversing a public network such as the Internet. In some embodiments, CSPI 101 enables customer VCN workloads with private IP addresses to access the service's (public service endpoint) using a service gateway. Thus, the service gateway provides a private access model by establishing a virtual link between the customer's VCN and the public endpoint of the service residing outside the customer's private network.

[0090] Furthermore, CSPI can provide private public access using technologies such as FastConnect public peering, where a customer's local instance can access one or more services within a customer's VCN using a FastConnect connection without traversing a public network such as the internet. CSPI can also provide private private access using FastConnect private peering, where a customer's local instance with a private IP address can access a customer's VCN workloads using a FastConnect connection. FastConnect is a network connectivity alternative to using the public internet to connect a customer's local network to CSPI and its services. Compared to internet-based connections, FastConnect offers a simple, flexible, and cost-effective way to create private and private connections with higher bandwidth options and a more reliable and consistent network experience.

[0091] Figure 1The accompanying description above describes the various virtualized components in the example virtual network. As mentioned above, the virtual network is built on the underlying physical or substrate network. Figure 2 A simplified architecture diagram of the physical components within the physical network of the CSPI 200, which provides the underlying layer for virtual networks according to certain embodiments, is depicted. As shown, the CSPI 200 provides a distributed environment including components and resources (e.g., compute, storage, and network resources) provided by a cloud service provider (CSP). These components and resources are used to provide cloud services (e.g., IaaS services) to subscribed customers (i.e., customers who have subscribed to one or more services provided by the CSP). Based on the services subscribed to by the customer, a subset of the resources of the CSPI 200 (e.g., compute, storage, and network resources) is provisioned to the customer. The customer can then use the physical compute, storage, and networking resources provided by the CSPI 200 to build their own cloud-based (i.e., CSPI-hosted) customizable and private virtual networks. As indicated above, these customer networks are referred to as Virtual Cloud Networks (VCNs). Customers can deploy one or more customer resources, such as compute instances, on these customer VCNs. Compute instances can take the form of virtual machines, bare metal instances, etc. The CSPI 200 provides infrastructure and a complementary set of cloud services that enable customers to build and run a wide range of applications and services in a highly available managed environment.

[0092] exist Figure 2 In the example embodiment depicted, the physical components of CSPI 200 include one or more physical host machines or physical servers (e.g., 202, 206, 208), network virtualization devices (NVDs) (e.g., 210, 212), top-of-rack (TOR) switches (e.g., 214, 216), and a physical network (e.g., 218), as well as switches within physical network 218. The physical host machines or servers can host and execute various compute instances participating in one or more subnets of the VCN. Compute instances can include virtual machine instances and bare metal instances. For example, Figure 1 The various computational examples described in the text can be derived from... Figure 2 The physical host machine described in the diagram is used for hosting virtual machine compute instances in a VCN. Virtual machine compute instances in a VCN can be executed by one host machine or multiple different host machines. A physical host machine can also host virtual host machines, container-based hosts, or functions, etc. Figure 1 The VNIC and VCN VR described in the text can be generated by Figure 2 The NVD execution described in the text. Figure 1 The gateway described herein can be a host machine and / or a... Figure 2 The NVD execution described in [the document / document].

[0093] A host machine or server can execute a hypervisor (also known as a virtual machine monitor or VMM) that creates and enables virtualized environments on the host machine. Virtualized or virtualized environments facilitate cloud-based computing. One or more compute instances can be created, executed, and managed on the host machine by a hypervisor on that host machine. The hypervisor on the host machine enables the host machine's physical computing resources (e.g., compute, memory, and network resources) to be shared among various compute instances executed by the host machine.

[0094] For example, such as Figure 2 As depicted, host machines 202 and 208 execute hypervisors 260 and 266, respectively. These hypervisors can be implemented using software, firmware, or hardware, or a combination thereof. Typically, a hypervisor is a process or software layer located above the host machine's operating system (OS), which in turn executes on the host machine's hardware processor. Hypervisors provide a virtualized environment by enabling the host machine's physical computing resources (e.g., processing resources such as processors / cores, memory resources, network resources) to be shared among various virtual machine computing instances executed by the host machine. For example, in... Figure 2 In this configuration, the hypervisor 260 can reside on top of the operating system of the host machine 202, enabling the computing resources (e.g., processing, memory, and network resources) of the host machine 202 to be shared among computing instances (e.g., virtual machines) executed by the host machine 202. Virtual machines can have their own operating systems (called guest operating systems), which may be the same as or different from the host machine's operating system. The operating system of a virtual machine executed by the host machine can be the same as or different from the operating system of another virtual machine executed by the same host machine. Therefore, the hypervisor enables multiple operating systems to be executed simultaneously, sharing the same computing resources of the host machine. Figure 2 The host machines described in the text may have the same or different types of management programs.

[0095] A compute instance can be a virtual machine instance or a bare metal instance. Figure 2 In the diagram, compute instance 268 on host machine 202 and compute instance 274 on host machine 208 are examples of virtual machine instances. Host machine 206 is an example of a bare metal instance provided to a customer.

[0096] In some cases, an entire host machine can be provisioned to a single customer, and one or more compute instances (or virtual machines or bare metal instances) hosted on that host machine all belong to the same customer. In other cases, the host machine can be shared among multiple customers (i.e., multiple tenants). In this multi-tenancy scenario, the host machine can host virtual machine compute instances belonging to different customers. These compute instances can be members of different VCNs for different customers. In some embodiments, bare metal compute instances are hosted by bare metal servers without a hypervisor. When provisioning bare metal compute instances, a single customer or tenant maintains control over the physical CPU, memory, and network interfaces of the host machine hosting the bare metal instance, and the host machine is not shared with other customers or tenants.

[0097] As previously described, each compute instance, as part of a VCN, is associated with a VNIC that enables that compute instance to become a member of a subnet of the VCN. The VNIC associated with a compute instance facilitates packet or frame communication to and from the compute instance. The VNIC is associated with the compute instance when the compute instance is created. In some embodiments, for compute instances executed by a host machine, the VNIC associated with that compute instance is executed by an NVD connected to the host machine. For example, in Figure 2 In this example, host machine 202 executes a virtual machine compute instance 268 associated with VNIC 276, and VNIC 276 is executed by NVD 210 connected to host machine 202. As another example, a bare metal instance 272 hosted by host machine 206 is associated with VNIC 280 executed by NVD 212 connected to host machine 206. As yet another example, VNIC 284 is associated with compute instance 274 executed by host machine 208, and VNIC 284 is executed by NVD 212 connected to host machine 208.

[0098] For compute instances hosted by a host machine, NVDs connected to that host machine also execute VCN VR corresponding to the VCN where the compute instance is a member. For example, in Figure 2 In the embodiment depicted, NVD 210 executes VCN VR 277 corresponding to the VCN of compute instance 268, which is a member of NVD 212. NVD 212 may also execute one or more VCN VR 283 corresponding to the VCNs of compute instances hosted by host machines 206 and 208.

[0099] The host machine may include one or more network interface cards (NICs) that enable the host machine to connect to other devices. The NIC on the host machine may provide one or more ports (or interfaces) that allow the host machine to communicatively connect to another device. For example, the host machine may use one or more ports (or interfaces) provided on the host machine and the NVD to connect to the NVD. The host machine may also connect to other devices, such as another host machine.

[0100] For example, in Figure 2 In this configuration, host machine 202 is connected to NVD 210 via link 220, which extends between port 234 provided by NIC 232 of host machine 202 and port 236 of NVD 210. Host machine 206 is connected to NVD 212 via link 224, which extends between port 246 provided by NIC 244 of host machine 206 and port 248 of NVD 212. Host machine 208 is connected to NVD 212 via link 226, which extends between port 252 provided by NIC 250 of host machine 208 and port 254 of NVD 212.

[0101] The NVD is then connected to top-of-rack (TOR) switches via communication links, which are connected to physical network 218 (also known as the switching architecture). In some embodiments, the links between the host machine and the NVD, and between the NVD and the TOR switches, are Ethernet links. For example, in Figure 2 In this configuration, NVDs 210 and 212 are connected to TOR switches 214 and 216 via links 228 and 230, respectively. In some embodiments, links 220, 224, 226, 228, and 230 are Ethernet links. The collection of host machines and NVDs connected to the TOR is sometimes referred to as a rack.

[0102] Physical network 218 provides a communication architecture that enables TOR switches to communicate with each other. Physical network 218 can be a multi-layer network. In some implementations, physical network 218 is a multi-layer Clos network of switches, where TOR switches 214 and 216 represent leaf-level nodes of the multi-layer and multi-node physical switching network 218. Different Clos network configurations are possible, including but not limited to Layer 2 networks, Layer 3 networks, Layer 4 networks, Layer 5 networks, and general "n"-layer networks. Examples of Clos networks are provided in... Figure 5 It is depicted in the middle and described below.

[0103] Various connection configurations can exist between the host machine and the NVD, such as one-to-one, many-to-one, and one-to-many configurations. In a one-to-one configuration, each host machine connects to its own individual NVD. For example, in... Figure 2 In this configuration, host machine 202 connects to NVD 210 via its NIC 232. In a many-to-one configuration, multiple host machines connect to a single NVD. For example, in... Figure 2 In this configuration, host machines 206 and 208 are connected to the same NVD 212 via NICs 244 and 250, respectively.

[0104] In a one-to-many configuration, a host machine connects to multiple NVDs. Figure 3 An example within the CSPI 300 is shown, where a host machine is connected to multiple NVDs. (Example follows) Figure 3 As shown, host machine 302 includes a network interface card (NIC) 304, which includes multiple ports 306 and 308. Host machine 300 is connected to a first NVD 310 via port 306 and link 320, and to a second NVD 312 via port 308 and link 322. Ports 306 and 308 can be Ethernet ports, and links 320 and 322 between host machine 302 and NVDs 310 and 312 can be Ethernet links. NVD 310 is further connected to a first TOR switch 314, and NVD 312 is connected to a second TOR switch 316. Links between NVDs 310 and 312 and TOR switches 314 and 316 can be Ethernet links. TOR switches 314 and 316 represent Layer 0 switching devices in a multi-layer physical network 318.

[0105] Figure 3 The layout depicted provides two separate physical network paths from physical switch network 318 to host machine 302: the first path goes through TOR switch 314 to NVD 310 and then to host machine 302, and the second path goes through TOR switch 316 to NVD 312 and then to host machine 302. These separate paths provide enhanced availability (referred to as high availability) for host machine 302. If one of the paths (e.g., a link in one of the paths breaks) or a device (e.g., a particular NVD is not running) experiences a problem, the other path can be used for communication to / from host machine 302.

[0106] exist Figure 3 In the configuration depicted, the host machine connects to two different NVDs using two different ports provided by the host machine's NIC. In other embodiments, the host machine may include multiple NICs that enable the host machine to connect to multiple NVDs.

[0107] Go back to reference Figure 2An NVD is a physical device or component that performs one or more network and / or storage virtualization functions. An NVD can be any device with one or more processing units (e.g., CPU, Network Processing Unit (NPU), FPGA, packet processing pipeline, etc.), cached memory, and ports. Various virtualization functions can be executed by software / firmware running on one or more processing units of the NVD.

[0108] NVDs can be implemented in various different forms. For example, in some embodiments, an NVD is implemented as an interface card called a smartNIC or a smart NIC with an onboard embedded processor. A smartNIC is a device separate from the NIC on the host machine. Figure 2 In this context, NVD 210 and 212 can be implemented as smartNICs connected to host machine 202 and host machines 206 and 208, respectively.

[0109] However, smartNIC is just one example of an NVD implementation. Various other implementations are possible. For example, in some other implementations, the NVD, or one or more functions performed by the NVD, may be integrated into or performed by one or more host machines, one or more TOR switches, and other components of the CSPI 200. For example, the NVD may be implemented within a host machine, where the functions performed by the NVD are performed by the host machine. As another example, the NVD may be part of a TOR switch, or the TOR switch may be configured to perform functions performed by the NVD, enabling the TOR switch to perform various complex packet transformations for public clouds. TORs performing the functions of the NVD are sometimes referred to as smart TORs. In other implementations that serve virtual machine (VM) instances rather than bare metal (BM) instances to customers, the functions performed by the NVD may be implemented within the hypervisor of the host machine. In some other implementations, some of the functions of the NVD may be offloaded to a centralized service running on a set of host machines.

[0110] In some embodiments, such as when implemented as Figure 2 As shown in the smartNIC diagram, the NVD can include multiple physical ports that enable it to connect to one or more host machines and one or more TOR switches. Ports on the NVD can be classified as host-facing ports (also known as "south ports") or network-facing or TOR-facing ports (also known as "north ports"). The host-facing ports of the NVD are those used to connect the NVD to the host machine. Figure 2 Examples of host-facing ports include port 236 on the NVD 210 and ports 248 and 254 on the NVD 212. Network-facing ports on the NVD are used to connect the NVD to a TOR switch. Figure 2 Examples of network-facing ports include port 256 on the NVD 210 and port 258 on the NVD 212. Figure 2 As shown, NVD 210 is connected to TOR switch 214 via link 228, which extends from port 256 of NVD 210 to TOR switch 214. Similarly, NVD 212 is connected to TOR switch 216 via link 230, which extends from port 258 of NVD 212 to TOR switch 216.

[0111] The NVD receives packets and frames from the host machine (e.g., packets and frames generated by compute instances hosted by the host machine) via its host-facing port, and after performing the necessary packet processing, can forward the packets and frames to the TOR switch via its network-facing port. The NVD can also receive packets and frames from the TOR switch via its network-facing port, and after performing the necessary packet processing, can forward the packets and frames to the host machine via its host-facing port.

[0112] In some embodiments, there can be multiple ports and associated links between the NVD and TOR switches. These ports and links can be aggregated to form a link aggregation group (called a LAG) of multiple ports or links. Link aggregation allows multiple physical links between two endpoints (e.g., between the NVD and TOR switches) to be treated as a single logical link. All physical links in a given LAG can operate at the same speed in full-duplex mode. LAGs help increase the bandwidth and reliability of the connection between two endpoints. If one of the physical links in the LAG fails, traffic is dynamically and transparently reassigned to one of the other physical links in the LAG. Aggregated physical links deliver higher bandwidth than each individual link. Multiple ports associated with an LAG are treated as a single logical port. Traffic can be load balanced across multiple physical links in the LAG. One or more LAGs can be configured between two endpoints. These endpoints can be located between the NVD and TOR switches, between a host machine and the NVD, etc.

[0113] The NVD implements or performs network virtualization functions. These functions are performed by software / firmware executed by the NVD. Examples of network virtualization functions include, but are not limited to: packet encapsulation and decapsulation functions; functions for creating VCN networks; functions for implementing network policies, such as VCN security list (firewall) functionality; functions for facilitating the routing and forwarding of packets to and from compute instances in the VCN; and so on. In some embodiments, upon receiving a packet, the NVD is configured to perform a packet processing pipeline for processing the packet and determining how to forward or route the packet. As part of this packet processing pipeline, the NVD may perform one or more virtual functions associated with the overlay network, such as performing VNICs associated with compute instances in the VCN, performing virtual routers (VRs) associated with the VCN, encapsulating and decapsulating packets to facilitate forwarding or routing in the virtual network, performing certain gateways (e.g., local peer gateways), implementing security lists, network security groups, Network Address Translation (NAT) functionality (e.g., translating public IPs to private IPs host-by-host), throttling functions, and other functions.

[0114] In some embodiments, the packet processing data path in the NVD may include multiple packet pipelines, each consisting of a series of packet transformation stages. In some implementations, upon receiving a packet, the packet is parsed and classified into a single pipeline. The packet is then processed linearly, stage by stage, until the packet is dropped or sent out through the NVD's interface. These stages provide basic functional packet processing building blocks (e.g., header verification, throttling, insertion of new Layer 2 headers, L4 firewall enforcement, VCN encapsulation / decapsulation, etc.) so that new pipelines can be built by combining existing stages, and new functionality can be added by creating new stages and inserting them into existing pipelines.

[0115] The NVD can perform both control plane and data plane functions corresponding to the VCN's control plane and data plane. Examples of the VCN control plane are also depicted in Figures 18, 19, 20, and 21 (see reference numerals 1816, 1916, 2016, and 2116) and described below. Examples of the VCN data plane are depicted in Figures 18, 19, 20, and 21 (see reference numerals 1818, 1918, 2018, and 2118) and described below. Control plane functions include functions for configuring how control data is forwarded on the network (e.g., setting routes and routing tables, configuring VNICs, etc.). In some embodiments, a VCN control plane is provided that centrally computes all overlay-to-baseboard mappings and publishes them to the NVD and virtual network edge devices (such as various gateways, such as DRGs, SGWs, IGWs, etc.). Firewall rules can also be published using the same mechanism. In some embodiments, the NVD only obtains mappings associated with that NVD. Data plane functions include functions for actually routing / forwarding packets based on the configuration established using the control plane. The VCN data plane is implemented by encapsulating customer network packets before they traverse the substrate network. Encapsulation / decapsulation functionality is implemented on the NVD. In some embodiments, the NVD is configured to intercept all network packets entering and leaving the host machine and perform network virtualization functions.

[0116] As indicated above, NVD performs various virtualization functions, including VNIC and VCN VR. NVD can execute VNICs associated with compute instances hosted on one or more host machines connected to the VNIC. For example, as... Figure 2 As depicted, NVD 210 performs the functionality of VNIC 276 associated with compute instance 268 hosted by host machine 202 connected to NVD 210. As another example, NVD 212 performs VNIC 280 associated with bare-metal compute instance 272 hosted by host machine 206, and VNIC 284 associated with compute instance 274 hosted by host machine 208. Host machines can host compute instances belonging to different VCNs (which belong to different customers), and NVDs connected to host machines can perform VNICs corresponding to compute instances (i.e., perform VNIC-related functionality).

[0117] NVD also executes a VCN virtual router corresponding to the VCN of the compute instance. For example, in Figure 2In the embodiments depicted, NVD 210 executes VCN VR 277 corresponding to the VCN to which compute instance 268 belongs. NVD 212 executes one or more VCN VR 283 corresponding to one or more VCNs to which compute instances hosted by host machines 206 and 208 belong. In some embodiments, the VCN VR corresponding to a VCN is executed by all NVDs connected to host machines hosting at least one compute instance belonging to that VCN. If a host machine hosts compute instances belonging to different VCNs, then NVDs connected to that host machine can execute VCN VR corresponding to those different VCNs.

[0118] In addition to VNIC and VCN VR, NVD can execute various software (e.g., daemons) and includes one or more hardware components that facilitate various network virtualization functions performed by NVD. For simplicity, these various components are grouped together as... Figure 2 The term "packet processing component" is shown in the diagram. For example, NVD 210 includes packet processing component 286 and NVD 212 includes packet processing component 288. For instance, a packet processing component for an NVD may include a packet processor configured to interact with the NVD's ports and hardware interfaces to monitor all packets received by and transmitted using the NVD and to store network information. This network information may include, for example, network flow information identifying different network flows handled by the NVD and per-flow information (e.g., per-flow statistics). In some embodiments, network flow information may be stored on a per-VNIC basis. The packet processor may perform per-packet manipulation and implement stateful NAT and L4 firewall (FW). As another example, a packet processing component may include a replication agent configured to replicate information stored by the NVD to one or more different replication target repositories. As yet another example, a packet processing component may include a logging agent configured to perform logging functions of the NVD. The packet processing component may also include software for monitoring the performance and health of the NVD and may also monitor the status and health of other components connected to the NVD.

[0119] Figure 1 The components of an example virtual or overlay network are shown, including a VCN, subnets within the VCN, compute instances deployed on the subnets, VNICs associated with the compute instances, a VR for the VCN, and a set of gateways configured for the VCN. Figure 1 The overlay component described in the text can be made by Figure 2 One or more executions or hosts are described in the physical components. For example, a compute instance in a VCN can be executed or managed by... Figure 2The VNIC described herein is executed or hosted by one or more host machines. For a compute instance hosted by a host machine, the VNIC associated with that compute instance is typically executed by an NVD connected to that host machine (i.e., VNIC functionality is provided by an NVD connected to that host machine). The VCN VR functionality for a VCN is executed by all NVDs connected to the host machine hosting or executing a compute instance as part of that VCN. The gateway associated with a VCN can be executed by one or more different types of NVDs. For example, some gateways can be executed by smartNICs, while others can be executed by one or more host machines or other implementations of NVDs.

[0120] As described above, compute instances in a client VCN can communicate with various endpoints, which may be in the same subnet as the source compute instance, in a different subnet but within the same VCN as the source compute instance, or outside the source compute instance's VCN. These communications are facilitated using VNICs, VCN VRs, and gateways associated with the VCNs.

[0121] For communication between two compute instances on the same subnet within a VCN, a VNIC associated with both the source and destination compute instances facilitates the communication. The source and destination compute instances can be hosted by the same host machine or different host machines. Packets originating from the source compute instance can be forwarded from the host machine hosting the source compute instance to an NVD connected to that host machine. On the NVD, packets are processed using a packet processing pipeline, which may include the execution of the VNIC associated with the source compute instance. Because the destination endpoint of the packet is within the same subnet, the execution of the VNIC associated with the source compute instance results in the packet being forwarded to the NVD executing the VNIC associated with the destination compute instance, which then processes the packet and forwards it to the destination compute instance. The VNIC associated with the source and destination compute instances can execute on the same NVD (e.g., when the source and destination compute instances are hosted by the same host machine) or on different NVDs (e.g., when the source and destination compute instances are hosted by different host machines connected to different NVDs). The VNIC can use a routing / forwarding table stored by the NVD to determine the next hop for the packet.

[0122] For packets destined for endpoints in different subnets within the same VCN, the packet originating from the source compute instance is forwarded from the host machine hosting the source compute instance to the NVD connected to that host machine. On the NVD, the packet is processed using a packet processing pipeline, which may include the execution of one or more VNICs and the VR associated with the VCN. For example, as part of the packet processing pipeline, the NVD executes or invokes a function corresponding to the VNIC associated with the source compute instance (also known as executing the VNIC). Functions executed by the VNIC may include viewing the VLAN tag on the packet. Since the packet's destination is outside the subnet, the VCN VR function is then invoked and executed by the NVD. The VCN VR then routes the packet to the NVD executing the VNIC associated with the destination compute instance. The VNIC associated with the destination compute instance then processes the packet and forwards it to the destination compute instance. The VNICs associated with the source and destination compute instances may execute on the same NVD (e.g., when the source and destination compute instances are hosted by the same host machine) or on different NVDs (e.g., when the source and destination compute instances are hosted by different host machines connected to different NVDs).

[0123] If the packet's destination is outside the source compute instance's VCN, the packet originating from the source compute instance is forwarded from the host machine hosting the source compute instance to the NVD connected to that host machine. The NVD executes the VNIC associated with the source compute instance. Since the packet's destination endpoint is outside the VCN, the packet is subsequently processed by the VCN VR used by that VCN. The NVD invokes VCN VR functionality, which may result in the packet being forwarded to the NVD executing the appropriate gateway associated with the VCN. For example, if the destination is an endpoint within a customer's local network, the packet may be forwarded by the VCN VR to the NVD executing the DRG gateway configured for the VCN. The VCN VR may execute on the same NVD as the NVD executing the VNIC associated with the source compute instance, or it may be executed by a different NVD. The gateway may be executed by the NVD, which may be a smartNIC, a host machine, or another NVD implementation. The packet is then processed by the gateway and forwarded to the next hop, which facilitates the packet's delivery to its intended destination endpoint. For example, in Figure 2 In the embodiment depicted, packets originating from compute instance 268 can be transmitted from host machine 202 to NVD 210 via link 220 (using NIC 232). On NVD 210, VNIC 276 is invoked because it is the VNIC associated with the source compute instance 268. VNIC 276 is configured to examine the information encapsulated in the packet and determine the next hop for forwarding the packet, with the aim of facilitating the transmission of the packet to its intended destination endpoint, and then forwarding the packet to the determined next hop.

[0124] Compute instances deployed on a VCN can communicate with a variety of endpoints. These endpoints can include endpoints hosted by CSPI 200 and endpoints outside of CSPI 200. Endpoints hosted by CSPI 200 can include instances within the same VCN or other VCNs, which can be the customer's VCN or a VCN not belonging to the customer. Communication between endpoints hosted by CSPI 200 can be performed via physical network 218. Compute instances can also communicate with endpoints not hosted by CSPI 200 or outside of CSPI 200. Examples of these endpoints include endpoints within the customer's local network or data center, or public endpoints accessible via public networks such as the Internet. Communication with endpoints outside of CSPI 200 can use various communication protocols over public networks (e.g., the Internet). Figure 2 (not shown in the image) or a dedicated network ( Figure 2 (Not shown in the image) to execute.

[0125] Figure 2 The architecture of the CSPI 200 depicted herein is merely an example and is not intended to be limiting. Variations, alternatives, and modifications are possible in alternative embodiments. For example, in some implementations, the CSPI 200 may have a more advanced architecture than... Figure 2 The systems or components shown may include more or fewer systems or components, and may combine two or more systems, or may have different system configurations or arrangements. Figure 2 The systems, subsystems, and other components described herein may be implemented in software (e.g., code, instructions, programs) executed by one or more processing units (e.g., processors, cores) of the respective system, using hardware, or a combination thereof. The software may be stored on a non-transitory storage medium (e.g., a memory device).

[0126] Figure 4 The connectivity between the host machine and the NVD, according to certain embodiments, is described for providing I / O virtualization to support multi-tenancy. For example... Figure 4 As depicted, host machine 402 executes a hypervisor 404 that provides a virtualized environment. Host machine 402 executes two virtual machine instances, VM1 406 belonging to customer / tenant #1 and VM2 408 belonging to customer / tenant #2. Host machine 402 includes a physical NIC 410 connected to NVD 412 via link 414. Each compute instance is attached to a VNIC executed by NVD 412. Figure 4 In the embodiment, VM1 406 is attached to VNIC-VM1 420 and VM2 408 is attached to VNIC-VM2 422.

[0127] like Figure 4As shown, NIC 410 includes two logical NICs, logical NIC A 416 and logical NIC B 418. Each virtual machine is attached to its own logical NIC and configured to work with its own logical NIC. For example, VM1 406 is attached to logical NIC A 416 and VM2 408 is attached to logical NIC B 418. Although host machine 402 includes only one physical NIC 410 shared by multiple tenants, each tenant's virtual machines believe they have their own host machine and network interface card due to the logical NICs.

[0128] In some embodiments, each logical NIC is assigned its own VLAN ID. Thus, a specific VLAN ID is assigned to logical NIC A 416 for tenant #1, and a different VLAN ID is assigned to logical NIC B 418 for tenant #2. When a packet is transmitted from VM1 406, the hypervisor appends a tag assigned to tenant #1 to the packet, and the packet is then transmitted from host machine 402 to NVD 412 via link 414. Similarly, when a packet is transmitted from VM2 408, the hypervisor appends a tag assigned to tenant #2 to the packet, and the packet is then transmitted from host machine 402 to NVD 412 via link 414. Therefore, a packet 424 transmitted from host machine 402 to NVD 412 has an associated tag 426 identifying the specific tenant and the associated VM. On the NVD, for a packet 424 received from the host machine 402, the tag 426 associated with the packet is used to determine whether the packet is processed by VNIC-VM1 420 or VNIC-VM2 422. The packet is then processed by the corresponding VNIC. Figure 4 The configuration described in the document allows each tenant's compute instance to believe that it owns its own host machine and NIC. Figure 4 The setup described in [the document] provides I / O virtualization to support multi-tenancy.

[0129] Figure 5 A simplified block diagram of a physical network 500 according to certain embodiments is depicted. Figure 5 The embodiments depicted are structured as Clos networks. Clos networks are a specific type of network topology designed to provide connectivity redundancy while maintaining high bandwidth and maximum resource utilization. Clos networks are non-blocking, multi-stage or multi-layer switching networks, where the number of stages or layers can be two, three, four, five, etc. Figure 5 The embodiment depicted is a Layer 3 network, including Layers 1, 2, and 3. TOR switch 504 represents a Layer 0 switch in a Clos network. One or more NVDs are connected to the TOR switch. Layer 0 switches are also referred to as edge devices of the physical network. Layer 0 switches are connected to Layer 1 switches, also known as leaf switches. Figure 5 In the embodiments depicted, a group of "n" Layer 0 TOR switches are connected to a group of "n" Layer 1 switches, forming a pod. Each Layer 0 switch in the pod is interconnected to all Layer 1 switches in that pod, but there is no switch connectivity between pods. In some implementations, two pods are called blocks. Each block is served by or connected to a group of "n" Layer 2 switches (sometimes called backbone switches). There can be several blocks in the physical network topology. The Layer 2 switches are then connected to "n" Layer 3 switches (sometimes called super backbone switches). Packet communication on the physical network 500 is typically performed using one or more Layer 3 communication protocols. Typically, all layers of the physical network (except the TOR layer) are n-way redundant, thus allowing high availability. Policies can be specified for pods and blocks to control the visibility of switches to each other in the physical network, thereby enabling scaling of the physical network.

[0130] A key characteristic of Clos networks is that the maximum number of hops from one Layer 0 switch to another (or from an NVD connected to a Layer 0 switch to another NVD connected to a Layer 0 switch) is fixed. For example, in a Layer 3 Clos network, a packet takes a maximum of seven hops to reach another NVD, where the source and destination NVDs are connected to the leaf layers of the Clos network. Similarly, in a Layer 4 Clos network, a packet takes a maximum of nine hops to reach another NVD, where the source and destination NVDs are connected to the leaf layers of the Clos network. Therefore, the Clos network architecture maintains consistent latency throughout the network, which is important for communication within and between data centers. Clos topologies are horizontally scalable and cost-effective. Network bandwidth / throughput capacity can be easily increased by adding more switches at each layer (e.g., more leaf switches and backbone switches) and by increasing the number of links between switches in adjacent layers.

[0131] In some implementations, each resource within CSPI is assigned a unique identifier called a Cloud Identifier (CID). This identifier is included as part of the resource's information and can be used to manage the resource, for example, via a console or API. An example syntax for a CID is: ocid1.<Resource Type>.<Domain>.[Region][.Future Use].<Unique ID> in, ocid1: A text string indicating the version of the CID; Resource type: The type of resource (e.g., instance, volume, VCN, subnet, user, group, etc.); Domain: The domain in which the resource resides. Example values ​​are "c1" for the commercial domain, "c2" for the government cloud domain, or "c3" for the federal government cloud domain, etc. Each domain can have its own domain name; Region: The region where the resource is located. This section may be empty if the region is not applicable to the resource. Future use: Reserved for future use.

[0132] Unique ID: The unique part of the ID. The format may vary depending on the type of resource or service.

[0133] Cloudy

[0134] Figure 6 A simplified high-level diagram of a distributed environment 600 comprising multiple cloud environments provided by different cloud service providers (CSPs) is depicted according to certain embodiments, wherein the cloud environments include a specific cloud environment providing specialized infrastructure that enables one or more cloud services provided by that specific cloud environment to be used by customers of other cloud environments. Figure 6 As described herein, various cloud environments (also referred to as “clouds”) can be provided by different cloud service providers (CSPs), each cloud environment or cloud providing one or more cloud services that can be subscribed to by one or more customers of that cloud environment. A set of cloud services provided by a cloud environment offered by a CSP can include one or more different types of cloud services, including but not limited to Software as a Service (SaaS), Infrastructure as a Service (IaaS), Platform as a Service (PaaS), Database as a Service (DBaaS), and so on. Examples of cloud environments provided by various CSPs include Oracle® Cloud Infrastructure (OCI) provided by Oracle Corporation, Microsoft® Azure provided by Microsoft Corporation, Google Cloud™ provided by Google LLC, and Amazon Web Services (AWS®) provided by Amazon Corporation. The cloud services provided by a particular cloud environment may differ from the set of cloud services provided by another cloud environment.

[0135] In a typical cloud environment, a CSP provides a Cloud Service Provider Infrastructure (CSPI), which is used to provide its customers with one or more cloud services offered by that cloud environment. The CSPI provided by the CSP can include various types of hardware and software resources, including computing resources, storage resources, networking resources, consoles for accessing cloud services, etc. Customers of a cloud environment provided by a CSP can subscribe to one or more cloud services offered by that cloud environment. The CSP can offer various subscription models to its customers. After a customer subscribes to a cloud service offered by the cloud environment, one or more users can be associated with the subscribed customer, and these users can use the cloud services subscribed to. In some implementations, when a customer subscribes to a cloud service offered by a particular cloud environment, a customer account or customer lease is created for that customer. One or more users can then be associated with the customer lease, and these users can then use the services subscribed to under the customer lease. Information about the services subscribed to by the customer, the users associated with the customer lease, etc., is typically stored within the cloud environment and associated with the customer lease.

[0136] For example, Figure 6 The document describes three different cloud environments provided by three different CSPs. These include cloud environment A (Cloud A) 610 provided by CSP A, cloud environment B (Cloud B) 640 provided by CSP B, and cloud environment C (Cloud C) 660 provided by CSP C. Cloud A 610 includes infrastructure CSPI_A 612 provided by CSP A, and this infrastructure can be used to provide a set of services "Service A" 614 provided by Cloud A 610. One or more customers (e.g., customer A1 616-1, customer A2 616-2) can subscribe to one or more services in Service A 614 provided by Cloud A 610. One or more users 618-1 can be associated with customer A1 616-1 and can use the services subscribed by customer A1 616-1 in Cloud A 610. Similarly, one or more users 618-2 can be associated with customer A2 616-2 and can use the services subscribed by customer A2 616-2 in Cloud A 610. In various use cases, the services subscribed to by customer A1 616-1 may differ from the services subscribed to by customer A2 616-2.

[0137] like Figure 6As described, Cloud B 640 includes infrastructure CSPI_B 642 provided by CSP B, and this infrastructure can be used to provide a set of services "Service B" 644 provided by Cloud B 640. One or more customers (e.g., customer B1 646-1) can subscribe to one or more services in Service B 644. One or more users 648-1 can be associated with customer B1 646-1 and can use the services subscribed to by customer B1 646-1 in Cloud B 640.

[0138] like Figure 6 As described, cloud C 660 includes infrastructure CSPI_C 662 provided by CSP C, and this infrastructure can be used to provide a set of services "Service C" 664 provided by cloud C 660. One or more customers (e.g., customer C1 666-1) can subscribe to one or more services in service C 664. One or more users 668-1 can be associated with customer C1 666-1 and can use the services subscribed to by customer C1 666-1 in cloud C 660. It should be noted that service A 614, service B 644 and service C 664 can be different from each other.

[0139] In existing cloud implementations, each cloud provides a closed ecosystem for its subscribers and associated users. Therefore, customers and their associated users in a cloud environment are limited to the services provided by the cloud that the customer subscribes to. For example, customer B1646-1 and user 648-1 are limited to using service B644 provided by cloud B640 and cannot use their accounts in cloud B640 to access services from different cloud environments, such as services in service A614 provided by cloud A610 or services in service C664 provided by cloud C660. The teachings described herein overcome this limitation. As described in this disclosure, various techniques are described that enable the creation of links between two cloud environments, allowing services provided by a first cloud environment by a first CSP to be used by customers (and associated users) in a different second cloud environment by a second, different CSP, using the customer's accounts in the second cloud environment.

[0140] For example, in Figure 6In the embodiments depicted, in addition to other infrastructure 620, the infrastructure CSPI_A 612 provided by CSP A includes special infrastructure 622 (referred to as Multi-Cloud Enabled Infrastructure 622, MEI 622, or Multi-Cloud Infrastructure 622), which enables one or more services 614 provided by Cloud A to be used by customers and associated users of other clouds (such as Cloud B 640 and C 660) using customer accounts in those other clouds. In some implementations, customers of Cloud B and C do not need to open separate accounts in Cloud A to use one or more of the services 614 provided by Cloud A 610. Customer B1 646-1 and associated user 648-1 of Cloud B 640 can use their customer accounts or leases in Cloud B 640 to use one or more services 614 provided by Cloud A 610. As another example, customer C1666-1 and related user 668-1 of cloud C 660 can use their customer accounts or leases in cloud C 660 to use one or more services 614 provided by cloud A 610.

[0141] In some implementations, MEI 622 enables the creation of links between Cloud A 610 and other clouds, whereby these links can be used by customers of the other clouds and their associated users to access and use services provided by Cloud A 610. This in Figure 6 The links 670 and 672 are symbolically represented as those between Cloud A 610 and Cloud B 640, and between Cloud A 610 and Cloud C 660. Through link 670, customers of Cloud B 640 can access or use one or more services 614 provided by Cloud A 610. Similarly, through link 672, customers of Cloud C 660 can access or use one or more services 614 provided by Cloud A 610.

[0142] There are different ways to implement MEI 612. In some embodiments, MEI 612 may include components that enable links to different clouds. For example, in Figure 6 In this embodiment, MEI 622 includes infrastructure component 624 responsible for enabling link 670 with cloud B 640, and infrastructure component 626 for enabling link 672 with cloud C 660. Similarly, MEI 622 may include other components that enable and facilitate links with other clouds. In some embodiments, components of MEI 622 may also facilitate links with multiple different clouds.

[0143] There are several reasons why a cloud customer might want or expect to use cloud services provided by different clouds. Figure 6For example, customer B1 646-1 of Cloud B 640 might want to use cloud service 614 provided by Cloud A 610 for several reasons. In one use case scenario, this might happen because Cloud A 610 provides cloud services with functionality not offered by Cloud B 640. In another use case scenario, Cloud A and B might offer similar services, but the service provided by Cloud A 610 might be better than the corresponding service provided by Cloud B 640 (e.g., more features / functionality, faster speed, etc.). In yet another use case scenario, customer B1 646-1 of Cloud B 640 might want to use cloud services provided by Cloud A 610 because the service is cheaper than that provided by Cloud B 640. In some cases, there might be geographical limitations or other reasons why customer B1 646-1 of Cloud B 640 might want to use cloud services provided by Cloud A 610. For example, Cloud A 610 might provide the desired service in a geographic area where Cloud B 640 does not provide services, or Cloud B 640 might not provide a specific service in the geographic area where the customer requested the service. There may also be several other use case scenarios where a cloud customer might want to use services provided by different clouds.

[0144] In some embodiments, MEI 622 provides the capability and performs functions to create a link between cloud A 610 and another cloud, enabling users associated with a customer of the other cloud to seamlessly access and use services provided by cloud A 610 from the other cloud itself. For example, MEI 622 enables user 648-1 associated with customer B1 646-1 of cloud 640 to seamlessly access services in service A 614 provided by cloud A 610. In some embodiments, a user interface (e.g., a console) accessible to user 648-1 from within cloud B 640 can be provided, allowing the user to see a list of services 614 provided by cloud A 610 and select a specific service that user 648-1 wishes to access. In response to the user selection, MEI 622 is responsible for performing the process of establishing link 670 between clouds A and B to enable access to the requested service. The process for establishing link 670 is substantially performed automatically by MEI 622. Customer B1 646-1 or associated user 648-1 need not worry about any system, networking, or other configuration changes required to facilitate the creation, maintenance, and use of link 670 between clouds A 610 and B 640. Users or customers bear no burden when creating links between clouds. Links are created quickly and efficiently using the techniques described in this disclosure.

[0145] MEI 622 can use various technologies to make the creation and use of links seamless for users and clients, thereby providing an enhanced user experience. In some implementations, MEI 622 makes the user interface (e.g., a graphical user interface GUI) and processing flow (such as for requesting services from cloud A 610 and for accessing the requested services from cloud A 610) that the client / user interacts with in cloud B 640 substantially similar to the interface and processing flow that the client / user will experience in cloud B 640. In this way, clients or users who may be accustomed to the interface and processing flow of cloud B 640 do not need to learn a new interface and processing flow to access service 614 from cloud A 610. MEI 622 can present different interfaces and processing flows for users in different cloud environments. For example, a first set of user interfaces and processes substantially similar to those of cloud B can be presented to users from cloud B 640, while another set of user interfaces and processes substantially similar to those of cloud C can be presented to users accessing cloud A 610 from cloud C 660. This is done to simplify accessing Cloud A 610 services from other clouds and thus enhance the user experience.

[0146] As another example, each cloud environment typically includes an identity management system configured to provide security for the cloud environment. The identity management system is configured to protect resources within the cloud environment, including resources provided by the CSP and resources deployed by subscribed cloud customers within the cloud environment. Functions performed by the identity management system include, for example, managing identity credentials (e.g., usernames, passwords, etc.) associated with cloud subscribers and related users, controlling user access to cloud resources and services using identity credentials based on permission / access policies configured for the cloud environment, and other functions. Different clouds may use different identity management systems and associated technologies. For example, the identity management system and associated processes in Cloud A 610 may be completely different from those in Cloud B 640, and the identity management system and associated processes in Cloud B 640 may be completely different from those in Cloud C 660. In some implementations, although the identity management systems and associated processes differ between different cloud environments, the techniques described herein enable users associated with customers in the first cloud to access cloud services provided by different clouds using the same identity credentials associated with customers and users in the first cloud.

[0147] For example, in Figure 6In the embodiments depicted, Cloud B 640, provided by CSP B, may include an identity management system that assigns or distributes identity credentials to its subscription customers and associated users, such as customer B1 646-1 and associated user 648-1. These identity credentials are associated with a lease created for customer B1 646-1 in Cloud B 640. In some implementations, MEI 622, provided by Cloud A 610, enables user 648-1, associated with Cloud B customer B1 646-1, to access services from service A 614 in Cloud A 610 using the identity credentials associated with user 648-1 and customer B1 646-1 in Cloud B 640. This significantly enhances the user experience for user 648-1, as they do not have to create new identity credentials specific to Cloud A 610 just to access service 614 in Cloud A 610. MEI 622 facilitates this access.

[0148] As an example, customer B1 of cloud B 640 can choose to use a service, such as Database as a Service (DBaaS), from a set of services 614 provided by cloud A 610. In response to this choice, MEI 622 enables the automatic creation of a link 670 between cloud A 610 and cloud B 640, allowing user 648-1 associated with customer B1 646-1 to use the DBaaS service provided by cloud A 610. The automatic establishment of link 670 is facilitated by MEI 622. After link 670 is established, user 648-1 can use the DBaaS service in cloud A 610 via cloud B 640. As part of using this service, user 648-1 can send a request to cloud A 610 to create a database resource via cloud B 640. In response, CSPI_A 612 can create the requested database in cloud A 610. In some implementations, the created database can be provisioned in a virtual network (e.g., a Virtual Cloud Network or VCN) created for customer B1 in cloud A 610, and user 648-1 can access it via cloud B 640. User 648-1 can then send requests from cloud B 640 to cloud A 610 to use the provisioned database. These requests may include, for example, requests to write data to the database, update data stored in the database, delete data in the database, delete the database, create additional databases, etc. In some use cases, these requests may originate from user 648-1 via cloud B 640 or from service 644 provided by cloud B 640. In this way, MEI 622 provided by cloud A 610 enables users associated with customers of different clouds provided by different CSPs to seamlessly access services provided by cloud A 610.

[0149] Figure 6The distributed environment 600 depicted herein is merely an example and is not intended to unduly limit the scope of the claimed embodiments. Many variations, alternatives, and modifications are possible. For example, in alternative embodiments, the distributed environment 600 may have more or fewer cloud environments. The cloud environment may also have more or fewer systems and components, or may have different configurations or arrangements of systems and components. Figure 6 The systems and components described herein may be implemented in software (e.g., code, instructions, programs) executed by one or more processing units (e.g., processors, cores) of the respective system, using hardware or a combination thereof. The software may be stored on a non-transitory storage medium (e.g., on a memory device).

[0150] Multi-cloud gateway

[0151] Cloud service providers (CSPs) offer cloud infrastructure (i.e., cloud environments) to provide their customers with a range of services, such as SaaS (Software as a Service), IaaS (Infrastructure as a Service), and PaaS (Platform as a Service). This cloud environment provides a closed ecosystem for its subscribers. Therefore, customers of a particular cloud environment are typically limited to using the services offered within that environment.

[0152] With the rapid increase in cloud service adoption, customers in specific cloud environments need to use / access different services offered by other cloud providers (referred to as third-party cloud providers in this paper). Typically, a cloud service provider (CSP) for a specific cloud environment can achieve this by designing application programming interfaces (APIs) to support those specific services offered by other cloud environments. One drawback of this approach is scalability. Specifically, a CSP for a specific cloud environment must design a unique (i.e., separate) API for each service offered in other cloud environments. Therefore, developing and supporting each service requires considerable effort. For example, deploying APIs involves significant effort in defining interfaces, permissions, acquiring peers and conducting security audits, and building new control planes. Furthermore, the operational costs of deploying multiple APIs to support different services may be economically infeasible. Overall, this approach is time-consuming and not scalable.

[0153] Embodiments of this disclosure describe a multi-cloud gateway (MCG) designed to reduce the effort required to develop and support new services. Essentially, the MCG can receive and forward requests to each third-party cloud (3PC) service provider, eliminating the need for modifications when exposing new services. It should be understood that the effort involved in defining interfaces (e.g., the MCG API), permissions, obtaining peer and security audits, and building the control plane is a one-time overhead. As described below, in the MCG architecture, authorization is configured to be handled almost exclusively by the 3PC service provider to avoid duplication of effort in specific cloud environments. For simplicity, in the following text, the cloud environment in which the MCG is deployed is referred to as the first cloud infrastructure provided by the first cloud service provider, while other / external cloud environments (provided by other CSPs) are referred to as 3PC environments.

[0154] Figure 7 An exemplary high-level architecture of a multi-cloud gateway (MCG) deployed in a first cloud infrastructure, according to some embodiments, is shown. Figure 7 As shown, the first cloud infrastructure provided by the first cloud service provider CSP 701 includes a console 703, service leases 705, and customer leases 720. The console 703 includes a per-service open API software development kit (SDK) unit 703A and a marshaling unit 703B. Service lease 705 includes a multi-cloud gateway (MCG) infrastructure 710, an authorization platform 707, policies 706, and cloud link objects 709. The MCG infrastructure 710 includes a cloud link lookup unit 710A, an identity mapping unit 710B, a request creation unit 710C, and a request transport unit 710D. Customer lease 720 includes a token exchange module 720A and an identity domain module 720B, such as the identity management system unit of the first cloud infrastructure 701. Furthermore, as... Figure 7 As shown, two external cloud service provider environments are illustrated: a first 3PC provided by a second CSP 730 and a second 3PC provided by a third CSP 740. Each external cloud service provider environment includes a customer account, which is associated with the corresponding cloud environment's identity and access management system (e.g., IAM systems 730A and 740A) and corresponding authentication policies (e.g., authentication policies 730B and 740B).

[0155] The Per-Service Open API Software Development Kit (SDK) unit 703A included in console 703 corresponds to a unit configured to acquire the API specification of an external cloud environment and generate an SDK (based on the acquired specification) using a code generator. This SDK is configured to route all requests (related to accessing services from external cloud providers) to MCG infrastructure 710. Note that the API specification of the external cloud is typically provided by the external cloud service provider (i.e., publicly available). This specification corresponds to, for example, API configuration information, such as API elements and the functionality of the different elements contained in the API. The code generator is configured to generate a "generic" request and forward it to MCG infrastructure 710 for further processing. In other words, the code generator generates an SDK that encodes requests to the external cloud into a generic format and transmits this generic request to MCG infrastructure 710. It should be understood that the marshalling unit 703B included in console 703 is configured to direct this request to MCG infrastructure 710 for authorization (and for further processing purposes). References will follow below. Figure 8 Describes an exemplary request transmitted to MCG infrastructure 710.

[0156] The request transmitted by console 703 is directed to the authorization platform 707 included in the service lease 705 of the first cloud infrastructure 701. Authorization platform 707 is configured to determine whether a user (who issued the request transmitted from console 703 to authorization platform 707) is permitted to access MCG infrastructure 710. Specifically, authorization platform 707 verifies whether the user has sufficient permissions / access rights to use MCG infrastructure 710 according to policy 706. It should be understood that the policy may be pre-configured for each user during the registration / subscription process. If authorization platform 707 successfully determines that the user is permitted to use MCG infrastructure 710, the request is forwarded to MCG infrastructure 710 for further processing. However, if authorization platform 707 determines that the user is not permitted to use MCG infrastructure 710, the request is discarded, and a message indicating this situation is sent back to the user, for example, via console 703.

[0157] The MCG infrastructure 710 processes requests based on the concept of a cloud link object 709. In some embodiments, a cloud link object (also referred to as a cloud link resource object) is a data object that includes a mapping from a first identifier (e.g., a lease name in the first cloud environment) associated with a user's account in the first cloud infrastructure to a second identifier (e.g., a tenant ID and associated subscription in the 3PC environment) associated with that user's account in an external cloud infrastructure. Furthermore, the cloud link object 709 stores location information related to the region / location of the first cloud environment and the 3PC environment, where linking between user accounts in the two cloud environments is enabled. In some implementations, linking a user's lease in the first cloud environment to the user's account in the 3PC environment includes storing in the link resource object a mapping from the first identifier associated with the user's lease in the first cloud environment to the second identifier associated with the user's account in the 3PC environment. In some implementations, the cloud link resource object 709 may be stored in a compartment (e.g., a root compartment) associated with the leases included in the first cloud environment. Further details regarding the creation and use of cloud links can be found in U.S. Patent Application No. 18 / 162,924, the entire contents of which are incorporated herein by reference.

[0158] According to some embodiments, after receiving an authorized request from the authorization platform 707, the MCG infrastructure 710 triggers the cloud link lookup unit 710A included in the MCG infrastructure 710. The cloud link lookup unit 710A is configured to determine whether mapping information exists, i.e., mapping information between the first cloud infrastructure and the 3PC infrastructure (e.g., as referred to below). Figure 8 The request includes the identifier of the 3PC. In one implementation, the cloud link lookup unit 710A can query the cloud link object 709 to obtain such mapping information. The identity mapping unit 710B of the MCG infrastructure 710 is configured to map the user's identity information in the first cloud infrastructure 701 to the user's identity in the corresponding 3PC environment. The request creation unit 710C of the MCG infrastructure 710 is configured to generate a request that will ultimately be transmitted to the endpoint of the 3PC. According to some embodiments, the request creation unit 710C receives a generic request formulated by the SDK and sent to the MCG infrastructure 710 (e.g., refer to...). Figure 8 The request is described and converted into a regular REST API request that can be transmitted (e.g., by request transmission unit 710D) to an endpoint in the 3PC environment.

[0159] The request transmitted by the request transmission unit 710D of the MCG infrastructure 710 is sent to the customer lease 720. This is done to perform a token exchange process executed by the token exchange module 720A. According to some embodiments, the first cloud environment is integrated with one or more external cloud provider environments (e.g., the first 3PC environment 730 or the second 3PC environment 740) for authentication and authorization purposes. In other words, the burden of authenticating users accessing third-party services is configured to be performed by the corresponding 3PC infrastructure. The token exchange module 720A is configured to exchange a token from the source cloud (e.g., a token issued in the first cloud environment) for another token valid in the cloud environment providing the service (e.g., the first 3PC environment 730).

[0160] According to some embodiments, the framework for exchanging a token generated by the service caller's cloud environment (e.g., a token issued by a first cloud environment) for another token (i.e., a token usable by the cloud environment providing the service (e.g., cloud environment 730 or 740)) relies on the identity-as-a-service (IDaaS) support of the cloud environment providing the service to enable user identity propagation within its cloud environment. Identity propagation is achieved through token exchange, where a standards-based token (e.g., JWT or SAML assertion) generated by the service caller's cloud environment can be exchanged for an internal principal (of the cloud environment providing the service), such as a User Principal Session Token (UPST), which can then be used to access the services provided in cloud environment 730 or 740. It should be understood that for such token exchange to be allowed, trust needs to be established between the identity provider of the first cloud environment (e.g., the service caller's cloud environment) and the cloud environment providing the service. In some implementations, trust is established through a configuration (referred to herein as cloud identity propagation configuration) introduced (e.g., introduced by an administrator) in an identity and access management system. It should be noted that the authentication policy 730B or 740B of the 3PC environment is configured to authenticate the user issuing the request, and upon successful authentication, 3PC executes the request and provides a response to the first cloud environment (e.g., MCG infrastructure 710 of the first cloud environment 701). The type of operation performed in 3PC may correspond to CRUD operations performed on one or more resources (e.g., database resources) provided by 3PC. Furthermore, details regarding the trust establishment and token exchange mechanism can be found in U.S. Patent Application No. 18 / 162,947, the entire contents of which are incorporated herein by reference.

[0161] Figure 8 An exemplary request 800, sent from console 703 to MCG infrastructure 710 according to some embodiments, is shown. Note that request 800 corresponds to a generic service request that can be further processed by MCG infrastructure 710 and sent to any 3PC environment. Figure 8As shown, request 800 includes several components: (a) a first component 801, which corresponds to the type of external CSP to which the request will be directed (e.g., Google Cloud); (b) a second component 802, which corresponds to the header portion of the request (e.g., an HTTP header); (c) a third component 803, which corresponds to the type of operation requested by the request, such as a GET operation; (d) a fourth component 804, which corresponds to the endpoint address in the 3PC to which the request will be sent; (e) a fifth component 805, which includes a path, such as a path in the 3PC that provides the location (e.g., a project) where operation 803 will be performed; (f) a sixth component 806, which corresponds to query parameters associated with the request (e.g., page size); (g) a seventh component 807, which corresponds to the body of the request; and (h) an eighth component 808, which includes a multi-cloud link, i.e., linking information that links the user's account in the first cloud infrastructure to the user's account in the 3PC.

[0162] Transfer to Figure 9 According to some embodiments, an exemplary flowchart is shown that illustrates when interacting with a first cloud environment (e.g., Figure 7 The first cloud environment (701) is associated with a user who performs actions against a user in a second cloud environment (e.g., Figure 7 The steps performed when requesting an operation to be performed using resources / services provided by a third-party cloud environment (730). Figure 9 The processing illustrated can be implemented in software (e.g., code, instructions, programs), hardware, or a combination thereof, executed by one or more processing units (e.g., processors, cores) of a corresponding system. The software can be stored on a non-transitory storage medium (e.g., a storage device). Figure 9 The methods presented and described below are intended to be illustrative rather than restrictive. Although Figure 9 Various processing steps that occur in a specific sequence or order are described, but this is not intended to be limiting. In some alternative embodiments, these steps may be performed in a different order, or some steps may be performed in parallel.

[0163] The process begins at step 905, where the SDK interface (e.g., an SDK interface generated by a code generator) imports an instance of the resource associated with the service provided by 3PC. For example, this instance could be represented as "Instance = Type OfService.GetInstance(CloudLink ID, "myProject", "instance1")". Note that the parameters of this instance include a cloud link identifier (e.g., containing mapping information of the user's account in the first cloud environment and the 3PC environment), the project name containing the instance (e.g., myProject), and the name of the instance (e.g., instance 1).

[0164] In step 910, the SDK specifies a general request. For example, the SDK generates a request like... Figure 8 The general request shown is transmitted to the MCG infrastructure, for example... Figure 7 The MCG 710. Furthermore, in step 915, the MCG (deployed in the first cloud environment) translates the request generated in step 910 into a regular REST API call / request. The translated request is then transmitted from the first cloud environment to the endpoint of the specific service in the 3PC environment. Additionally, in step 920, the response to this request obtained from the 3PC (by the multi-cloud gateway infrastructure) is populated into the instance object and ultimately served to the user.

[0165] Figure 10 An exemplary flowchart depicting the steps performed by an MCG according to some embodiments is shown. Figure 10 The processing described herein can be implemented in software (e.g., code, instructions, programs) executed by one or more processing units (e.g., processors, kernels) of the corresponding system, or in hardware, or in a combination thereof. The software can be stored in a non-transitory storage medium (e.g., a storage device). Figure 10 The methods presented and described below are intended to be illustrative rather than restrictive. Although Figure 10 Various processing steps that occur in a specific sequence or order are described, but this is not intended to be limiting. In some alternative embodiments, these steps may be performed in a different order, or some steps may be performed in parallel.

[0166] The process begins at step 1005, where in the first cloud environment (e.g., Figure 7 The first cloud environment (701) implements a multi-cloud gateway (MCG) that receives a first request, which requests a second cloud environment (e.g., Figure 7 The first operation is performed using resources provided by the 3PC environment (730). Note that MCG can perform the operation from the console (e.g., ...). Figure 7The console (703) receives the first request. Furthermore, the first request may have the characteristics described in the previous reference. Figure 8 The general format explained. Furthermore, the first request corresponds to a request issued by a user associated with the first cloud infrastructure, who expects to utilize services provided by the second cloud environment.

[0167] As previously referenced Figure 9 As described in step 915, in step 1010, MCG generates a first API call (e.g., a REST API call), which will be directed to a first endpoint associated with a service provided by the second cloud environment. In step 1015, the request generated in step 1010 is transmitted to the first endpoint in the second cloud environment.

[0168] Then, the process proceeds to step 1020, where the MCG receives the request in a third cloud environment (e.g., Figure 7 The second request performs the second operation in the 3PC environment 740. Note that all requests originating from the first cloud environment that need to access services provided by different 3PC environments are directed to the same API interface of MCG. That is, MCG handles all requests directed to different cloud environments, thereby avoiding the need to design and generate separate API interfaces for each service provided by different 3PCs.

[0169] In step 1025, MCG generates a second API call (e.g., another REST API call) that will be directed to a second endpoint (different from the first endpoint) associated with a service provided by the third cloud environment. In step 1030, the request generated in step 1025 is transmitted to the second endpoint in the third cloud environment. Furthermore, it should be noted that MCG is configured to receive responses from each of the 3PC environments regarding the first and second requests transmitted in steps 1015 and 1030, respectively. As previously described, MCG can be configured to include the payload of each response within an instance object created for the corresponding 3PC and provide the response to the user.

[0170] Examples of cloud infrastructure

[0171] As noted above, Infrastructure as a Service (IaaS) is a specific type of cloud computing. IaaS can be configured to provide virtualized computing resources over a public network (e.g., the Internet). In the IaaS model, cloud providers can host infrastructure components (e.g., servers, storage devices, network nodes (e.g., hardware), deployment software, platform virtualization (e.g., hypervisor layer), etc.). In some cases, IaaS providers can also offer various services accompanying these infrastructure components (e.g., billing, monitoring, logging, security, load balancing, and clustering, etc.). Therefore, because these services may be policy-driven, IaaS users can implement policies to drive load balancing to maintain application availability and performance.

[0172] In some cases, IaaS customers can access resources and services over a wide area network (WAN) such as the Internet and can use the cloud provider's services to install the remaining elements of the application stack. For example, a user can log in to the IaaS platform to create virtual machines (VMs), install an operating system (OS) on each VM, deploy middleware such as databases, create buckets for workloads and backups, and even install enterprise software into that VM. The customer can then use the provider's services to perform various functions, including balancing network traffic, troubleshooting application issues, monitoring performance, and managing disaster recovery.

[0173] In most cases, cloud computing models will require the involvement of cloud providers. Cloud providers can, but are not necessarily, third-party providers specializing in (e.g., provisioning, renting, selling) IaaS services. Entities may also choose to deploy private clouds, thus becoming their own infrastructure service providers.

[0174] In some examples, IaaS deployment is the process of placing a new application or a new version of an application onto a prepared application server, etc. It may also include the processing of server preparation (e.g., installation libraries, daemons, etc.). This is typically managed by the cloud provider, below the hypervisor layer (e.g., servers, storage devices, network hardware, and virtualization). Therefore, the customer can be responsible for processing (OS), middleware, and / or application deployment (e.g., on self-service virtual machines, etc., which can be started on demand).

[0175] In some examples, IaaS provisioning can refer to acquiring computers or virtual hosts for use, or even installing necessary libraries or services on them. In most cases, deployment does not include provisioning, and provisioning may need to be performed first.

[0176] In some cases, IaaS provisioning presents two distinct challenges. First, there are initial challenges in provisioning the initial infrastructure set before anything is operational. Second, once everything is provisioned, there are challenges in evolving the existing infrastructure (e.g., adding new services, changing services, removing services, etc.). In some cases, both challenges can be addressed by enabling configuration that declaratively defines the infrastructure. In other words, the infrastructure (e.g., which components are needed and how they interact) can be defined by one or more configuration files. Therefore, the overall topology of the infrastructure (e.g., which resources depend on which resources and how they work together) can be described declaratively. In some cases, once the topology is defined, workflows for creating and / or managing the different components described in the configuration files can be generated.

[0177] In some examples, the infrastructure can have many interconnected elements. For example, there may be one or more Virtual Private Clouds (VPCs) (e.g., potential on-demand pools of configurable and / or shared computing resources), also known as the core network. In some examples, one or more security group rules may also be provisioned to define how the network's security is configured, as well as one or more virtual machines (VMs). Other infrastructure elements, such as load balancers, databases, etc., may also be provisioned. The infrastructure can evolve incrementally as more and / or more infrastructure elements are expected and added.

[0178] In some cases, continuous deployment techniques can be used to enable the deployment of infrastructure code across various virtual computing environments. Furthermore, the described techniques enable infrastructure management within these environments. In some examples, service teams may write code that they expect to deploy to one or more, but often many, different production environments (e.g., across various geographical locations, sometimes spanning the entire world). However, in some examples, the infrastructure on which the code will be deployed must first be set up. In some cases, provisioning can be done manually, resources can be provisioned using provisioning tools, and / or once the infrastructure is provisioned, the code can be deployed using deployment tools.

[0179] Figure 11This is a block diagram 1100 illustrating an example pattern of an IaaS architecture according to at least one embodiment. Service provider 1102 may be communicatively coupled to a secure host lease 1104, which may include a virtual cloud network (VCN) 1106 and a secure host subnet 1108. In some examples, service provider 1102 may use one or more client computing devices, which may be portable handheld devices (e.g., iPhones, cellular phones, iPads, computing tablets, personal digital assistants (PDAs)) or wearable devices (e.g., Google Glass head-mounted displays), running software (such as Microsoft Windows Mobile) and / or various mobile operating systems (such as iOS, Windows Phone, Android, BlackBerry 8, Palm OS, etc.), and supporting the Internet, email, short message service (SMS), Blackberry, or other communication protocols. Alternatively, client computing devices may be general-purpose personal computers, including, for example, personal computers and / or laptops running various versions of Microsoft Windows, Apple Macintosh, and / or Linux operating systems. Client computing devices may be workstation computers running various commercially available UNIX or UNIX-like operating systems, including but not limited to any of various GNU / Linux operating systems (such as, for example, Google Chrome OS). Alternatively or additionally, the client computing device may be any other electronic device, such as a thin client computer, an internet-enabled gaming system (e.g., a Microsoft Xbox game console with or without Kinect gesture input), and / or a personal messaging device capable of communicating over a network that can access VCN 1106 and / or the internet.

[0180] VCN 1106 may include a local peering gateway (LPG) 1110, which may be communicatively coupled to a secure shell (SSH) VCN 1112 via LPG 1110 contained in SSH VCN 1112. SSH VCN 1112 may include an SSH subnet 1114, and SSH VCN 1112 may be communicatively coupled to a control plane VCN 1116 via LPG 1110 contained in control plane VCN 1116. Furthermore, SSH VCN 1112 may be communicatively coupled to a data plane VCN 1118 via LPG 1110. Control plane VCN 1116 and data plane VCN 1118 may be contained in a service lease 1119 that may be owned and / or operated by an IaaS provider.

[0181] The control plane VCN 1116 may include a control plane demilitarized zone (DMZ) layer 1120 that acts as a peripheral network (e.g., a portion of a corporate network between a corporate intranet and an external network). DMZ-based servers can assume limited liability and help control security vulnerabilities. Furthermore, the DMZ layer 1120 may include one or more load balancer (LB) subnets 1122, a control plane application layer 1124 that may include one or more application subnets 1126, and a control plane data layer 1128 that may include one or more database (DB) subnets 1130 (e.g., one or more front-end DB subnets and / or one or more back-end DB subnets). One or more LB subnets 1122 contained in the control plane DMZ layer 1120 may be communicatively coupled to one or more application subnets 1126 contained in the control plane application layer 1124 and an Internet gateway 1134 that may be contained in the control plane VCN 1116. The application subnets 1126 may be communicatively coupled to one or more DB subnets 1130 contained in the control plane data layer 1128, as well as a service gateway 1136 and a Network Address Translation (NAT) gateway 1138. The control plane VCN 1116 may include the service gateway 1136 and the NAT gateway 1138.

[0182] The control plane VCN 1116 may include a data plane mirror application layer 1140, which may include one or more application subnets 1126. The one or more application subnets 1126 included in the data plane mirror application layer 1140 may include a virtual network interface controller (VNIC) 1142 capable of executing a compute instance 1144. The compute instance 1144 may communicatively couple the one or more application subnets 1126 of the data plane mirror application layer 1140 to the one or more application subnets 1126 that may be included in the data plane application layer 1146.

[0183] Data plane VCN 1118 may include data plane application layer 1146, data plane DMZ layer 1148, and data plane data layer 1150. Data plane DMZ layer 1148 may include one or more LB subnets 1122 communicatively coupled to one or more application subnets 1126 of data plane application layer 1146 and Internet gateway 1134 of data plane VCN 1118. One or more application subnets 1126 communicatively coupled to service gateway 1136 and NAT gateway 1138 of data plane VCN 1118. Data plane data layer 1150 may also include one or more DB subnets 1130 communicatively coupled to one or more application subnets 1126 of data plane application layer 1146.

[0184] The Internet gateway 1134 of control plane VCN 1116 and data plane VCN 1118 can be communicatively coupled to metadata management service 1152, which can be communicatively coupled to public Internet 1154. Public Internet 1154 can be communicatively coupled to NAT gateway 1138 of control plane VCN 1116 and data plane VCN 1118. Service gateway 1136 of control plane VCN 1116 and data plane VCN 1118 can be communicatively coupled to cloud service 1156.

[0185] In some examples, service gateway 1136, either control plane VCN 1116 or data plane VCN 1118, can make application programming interface (API) calls to cloud service 1156 without traversing the public internet 1154. API calls from service gateway 1136 to cloud service 1156 can be unidirectional: service gateway 1136 can make API calls to cloud service 1156, and cloud service 1156 can send requested data to service gateway 1136. However, cloud service 1156 may not initiate API calls to service gateway 1136.

[0186] In some examples, secure host lease 1104 can be directly connected to service lease 1119, which would otherwise be isolated. Secure host subnet 1108 can communicate with SSH subnet 1114 via LPG 1110, which enables bidirectional communication between otherwise isolated systems. Connecting secure host subnet 1108 to SSH subnet 1114 allows secure host subnet 1108 to access other entities within service lease 1119.

[0187] Control plane VCN 1116 may allow users of service lease 1119 to configure or otherwise provision desired resources. Desired resources provisioned in control plane VCN 1116 may be deployed or otherwise used in data plane VCN 1118. In some examples, control plane VCN 1116 may be isolated from data plane VCN 1118, and the data plane mirror application layer 1140 of control plane VCN 1116 may communicate with the data plane application layer 1146 of data plane VCN 1118 via VNIC 1142, which may be included in both the data plane mirror application layer 1140 and the data plane application layer 1146.

[0188] In some examples, users or clients of the system can make requests, such as create, read, update, or delete (CRUD) operations, via the public internet 1154, which can transmit requests to the metadata management service 1152. The metadata management service 1152 can transmit requests to the control plane VCN 1116 via internet gateway 1134. Requests can be received by one or more LB subnets 1122 contained in the control plane DMZ layer 1120. The LB subnets 1122 can determine that the request is valid, and in response to this determination, they can transmit the request to one or more application subnets 1126 contained in the control plane application layer 1124. If the request is validated and requires a call to the public internet 1154, the call to the public internet 1154 can be transmitted to a NAT gateway 1138 that can make calls to the public internet 1154. The request may expect the storage to be located in one or more DB subnets 1130.

[0189] In some examples, the data plane mirroring application layer 1140 can facilitate direct communication between the control plane VCN 1116 and the data plane VCN 1118. For example, it may be desirable to apply configuration changes, updates, or other appropriate modifications to resources contained in the data plane VCN 1118. Through VNIC 1142, the control plane VCN 1116 can communicate directly with the resources contained in the data plane VCN 1118, and thus can perform configuration changes, updates, or other appropriate modifications.

[0190] In some embodiments, the control plane VCN 1116 and data plane VCN 1118 may be included in service lease 1119. In this case, the system's users or customers may not own or operate the control plane VCN 1116 or data plane VCN 1118. Alternatively, the IaaS provider may own or operate both the control plane VCN 1116 and data plane VCN 1118, and both planes may be included in service lease 1119. This embodiment can enable the isolation of networks that might prevent users or customers from interacting with resources of other users or customers. Furthermore, this embodiment can allow users or customers of the system to privately store databases without relying on the public Internet 1154, which may not have the desired level of security for storage.

[0191] In other embodiments, one or more LB subnets 1122 included in the control plane VCN 1116 may be configured to receive signals from the service gateway 1136. In this embodiment, the control plane VCN 1116 and the data plane VCN 1118 may be configured to be invoked by the IaaS provider's customers without invoking the public internet 1154. The IaaS provider's customers may expect this embodiment because the database(s) used by the customer can be controlled by the IaaS provider and can be stored on service lease 1119, which may be isolated from the public internet 1154.

[0192] Figure 12 This is a block diagram 1200 illustrating another example pattern of an IaaS architecture according to at least one embodiment. Service operator 1202 (e.g., Figure 11 Service provider 1102) can communicatively couple to secure host lease 1204 (e.g., Figure 11 Secure hosting lease 1104), the secure hosting lease 1204 may include a virtual cloud network (VCN) 1206 (e.g., Figure 11 VCN 1106) and Secure Host Subnet 1208 (e.g., Figure 11 The secure host subnet 1108). VCN 1206 may include a local peering gateway (LPG) 1210 (e.g., Figure 11 The LPG 1110), which can be communicatively coupled to the Secure Shell (SSH) VCN 1912 (e.g., via the LPG 1110 contained in the SSH VCN 1912) Figure 11 SSH VCN 1112). SSH VCN 1912 can include SSH subnet 1914 (e.g., Figure 11 SSH subnet 1114), and SSH VCN 1912 can be communicatively coupled to control plane VCN 1216 via LPG 1210 included in control plane VCN 1216 (e.g., Figure 11 Control plane VCN 1216). Control plane VCN 1216 may be included in service lease 1219 (e.g., Figure 11 In the service lease 1119), and the data plane VCN 1218 (e.g., Figure 11 The data plane VCN 1118 may be included in a customer lease 1221 that may be owned or operated by the system’s users or customers.

[0193] Control plane VCN 1216 may include control plane DMZ layer 1220 (e.g., Figure 11 The control plane DMZ layer 1120), which may include one or more LB subnets 1222 (e.g., Figure 11 (One or more) LB subnets 1122), may include (one or more) application subnets 1226 (e.g., Figure 11 The control plane application layer 1224 of (one or more) application subnets 1126 (e.g., Figure 11 The control plane application layer 1124) may include one or more database (DB) subnets 1230 (e.g., similar to...). Figure 11 The control plane data layer 1228 of (one or more) DB subnets 1130 (e.g., Figure 11 The control plane data layer 1128). One or more LB subnets 1222 contained in the control plane DMZ layer 1220 can be communicatively coupled to one or more application subnets 1226 contained in the control plane application layer 1224 and an Internet gateway 1234 that can be contained in the control plane VCN 1216 (e.g., Figure 11 Internet gateway 1134), and application subnet(s) 1226 can communicatively couple to DB subnet(s) 1230 contained in control plane data layer 1228 and service gateway 1236 (e.g., Figure 11 The service gateway) and Network Address Translation (NAT) gateway 1238 (e.g., Figure 11 (NAT gateway 1138). The control plane VCN 1216 may include the service gateway 1236 and the NAT gateway 1238.

[0194] The control plane VCN 1216 may include a data plane mirror application layer 1240 that may contain one or more application subnets 1226 (e.g., Figure 11 The data plane mirror application layer 1140). One or more application subnets 1226 contained in the data plane mirror application layer 1240 may include computational instances 1244 (e.g., similar to...). Figure 11 The virtual network interface controller (VNIC) 1242 (e.g., the VNIC of 1142) of the computing instance 1144. The computing instance 1244 may facilitate the mirroring of the application subnet(s) 1226 of the application layer 1240 in the data plane and may be included in the application layer 1246 in the data plane (e.g., Figure 11 Communication between one or more application subnets 1226 in the data plane application layer 1146 via VNIC 1242 contained in the data plane mirror application layer 1240 and VNIC 1242 contained in the data plane application layer 1246.

[0195] The Internet gateway 1234 included in the control plane VCN 1216 can be communicatively coupled to the metadata management service 1252 (e.g., Figure 11 Metadata management service 1152), which can communicatively couple to the public Internet 1254 (e.g., Figure 11 The public internet 1254 can communicatively couple to a NAT gateway 1238 contained in a control plane VCN 1216. The service gateway 1236 contained in the control plane VCN 1216 can communicatively couple to a cloud service 1256 (e.g., ...). Figure 11 Cloud services (1156).

[0196] In some examples, data plane VCN 1218 may be included in customer lease 1221. In this case, the IaaS provider may provide control plane VCN 1216 for each customer, and the IaaS provider may set up a unique compute instance 1244 for each customer, included in service lease 1219. Each compute instance 1244 may allow communication between control plane VCN 1216 included in service lease 1219 and data plane VCN 1218 included in customer lease 1221. Compute instance 1244 may allow resources provisioned in control plane VCN 1216 included in service lease 1219 to be deployed or otherwise used in data plane VCN 1218 included in customer lease 1221.

[0197] In other examples, an IaaS provider's customer may have a database residing in customer lease 1221. In this example, control plane VCN 1216 may include data plane mirror application layer 1240, which may include one or more application subnets 1226. Data plane mirror application layer 1240 may reside in data plane VCN 1218, but may not reside in data plane VCN 1218. That is, data plane mirror application layer 1240 may have access to customer lease 1221, but may not reside in data plane VCN 1218 or be owned or operated by an IaaS provider's customer. Data plane mirror application layer 1240 may be configured to invoke data plane VCN 1218, but may not be configured to invoke any entity contained in control plane VCN 1216. Customers may expect to deploy or otherwise use resources provided in the control plane VCN 1216 in the data plane VCN 1218, and the data plane mirroring application layer 1240 can facilitate the customer's expected deployment or other use of resources.

[0198] In some embodiments, an IaaS provider's customer may apply filters to data plane VCN 1218. In this embodiment, the customer may determine what data plane VCN 1218 can access, and the customer may restrict access from data plane VCN 1218 to the public Internet 1254. The IaaS provider may not be able to apply filters or otherwise control data plane VCN 1218's access to any external networks or databases. Applying filters and controls to data plane VCN 1218 contained in customer lease 1221 can help isolate data plane VCN 1218 from other customers and the public Internet 1254.

[0199] In some embodiments, cloud service 1256 may be invoked by service gateway 1236 to access services that may not exist on public internet 1254, control plane VCN 1216, or data plane VCN 1218. The connection between cloud service 1256 and control plane VCN 1216 or data plane VCN 1218 may not be real-time or continuous. Cloud service 1256 may reside on different networks owned or operated by an IaaS provider. Cloud service 1256 may be configured to receive calls from service gateway 1236 and may be configured not to receive calls from public internet 1254. Some cloud services 1256 may be isolated from other cloud services 1256, and control plane VCN 1216 may be isolated from cloud services 1256 that may not be in the same region as control plane VCN 1216. For example, control plane VCN 1216 may be located in "Region 1," and cloud service "Deployment 13" may be located in both "Region 1" and "Region 2." If the service gateway 1236, contained in the control plane VCN 1216 located in region 1, makes a call to deployment 13, then that call can be transmitted to deployment 13 in region 1. In this example, the control plane VCN 1216 or deployment 13 in region 1 may not be communicatively coupled to or otherwise communicate with deployment 13 in region 2.

[0200] Figure 13 This is a block diagram 1300 illustrating another example pattern of an IaaS architecture according to at least one embodiment. Service operator 1302 (e.g., Figure 11 Service provider 1102) can communicatively couple to secure host lease 1304 (e.g., Figure 11 Secure hosting lease 1104), the secure hosting lease 1304 may include a virtual cloud network (VCN) 1306 (e.g., Figure 11 VCN 1106) and Secure Host Subnet 1308 (e.g., Figure 11 The secure host subnet 1108). VCN 1306 can include LPG 1310 (e.g., Figure 11 The LPG 1110), which can be communicatively coupled to the SSH VCN 1312 via the LPG 1310 included in the SSH VCN 1312 (e.g., Figure 11 SSH VCN 1112). SSH VCN 1312 can include SSH subnet 1314 (e.g., Figure 11 SSH subnet 1114), and SSH VCN 1112 can be communicatively coupled to control plane VCN 1316 via LPG 1310 contained in control plane VCN 1316 (e.g., Figure 11 The control plane VCN 1116) and coupled to the data plane VCN 1318 via the LPG 1310 contained in the data plane VCN 1318 (e.g., Figure 11 Data plane 1118). Control plane VCN 1316 and data plane VCN 1318 may be included in service lease 1319 (e.g., Figure 11 In the service rental (1119).

[0201] The control plane VCN 1116 may include a subnet 1122 that may contain one or more load balancer (LB) subnets (e.g., Figure 11 The control plane DMZ layer 1120 of (one or more) LB subnets 1122) (e.g., Figure 11 The control plane DMZ layer 1120 may include one or more application subnets 1326 (e.g., similar to...). Figure 11 The control plane application layer 1324 of (one or more) application subnets 1126 (e.g., Figure 11 The control plane application layer 1124), and the control plane data layer 1328, which may include (one or more) DB subnets 1330, for example, Figure 11 The control plane data layer 1128). One or more LB subnets 1322 contained in the control plane DMZ layer 1320 can be communicatively coupled to one or more application subnets 1326 contained in the control plane application layer 1324 and an Internet gateway 1134 that can be contained in the control plane VCN 1316 (e.g., Figure 11 Internet gateway 1134), and application subnet(s) 1326 can communicatively couple to DB subnet(s) 1130 contained in control plane data layer 1128 and service gateway 1136 (e.g., Figure 11 The service gateway) and Network Address Translation (NAT) gateway 1138 (e.g., Figure 11 (NAT gateway 1138). The control plane VCN 1316 may include the service gateway 1336 and the NAT gateway 1338.

[0202] Data plane VCN 1318 may include data plane application layer 1346 (e.g., Figure 11 Data plane application layer 1146), data plane DMZ layer 1348 (e.g., Figure 11 Data plane DMZ layer 1148), and data plane data layer 1350 (e.g., Figure 11 The data plane data layer 1150). The data plane DMZ layer 1348 may include one or more trusted application subnets 1360 and one or more untrusted application subnets 1362 that can be communicatively coupled to the data plane application layer 1346, and one or more LB subnets 1322 of the Internet gateway 1334 contained in the data plane VCN 1318. The one or more trusted application subnets 1360 may be communicatively coupled to the service gateway 1336 contained in the data plane VCN 1318, the NAT gateway 1338 contained in the data plane VCN 1318, and one or more DB subnets 1330 contained in the data plane data layer 1350. The one or more untrusted application subnets 1362 may be communicatively coupled to the service gateway 1336 contained in the data plane VCN 1318 and the one or more DB subnets 1330 contained in the data plane data layer 1350. The data plane data layer 1350 may include one or more DB subnets 1330 that can be communicatively coupled to the service gateway 1336 contained in the data plane VCN 1318.

[0203] One or more untrusted application subnets 1362 may include one or more primary VNICs 1364(1)-(N) that can be communicatively coupled to tenant virtual machines (VMs) 1366(1)-(N). Each tenant VM 1366(1)-(N) may be communicatively coupled to a corresponding application subnet 1367(1)-(N) that may be contained in a corresponding container egress VCN 1368(1)-(N), which may be contained in a corresponding customer lease 1370(1)-(N). A corresponding secondary VNIC 1372(1)-(N) may facilitate communication between one or more untrusted application subnets 1362 contained in data plane VCN 1318 and application subnets contained in container egress VCN 1368(1)-(N). Each container exit VCN 1368(1)-(N) may include a NAT gateway 1338, which may communicatively couple to the public Internet 1354 (e.g., Figure 11 The public internet (1154).

[0204] Internet gateway 1334, contained in control plane VCN 1316 and data plane VCN 1318, can be communicatively coupled to metadata management service 1352 (e.g., Figure 11 A metadata management system 1152 is provided, which can communicatively couple to the public internet 1354. The public internet 1354 can communicatively couple to a NAT gateway 1338 contained in a control plane VCN 1316 and a data plane VCN 1318. A service gateway 1336 contained in a control plane VCN 1316 and a data plane VCN 1318 can communicatively couple to a cloud service 1356.

[0205] In some embodiments, the data plane VCN 1318 may be integrated with the customer lease 1370. Such integration may be useful or desirable for the IaaS provider's customers in certain situations, such as when support may be expected during code execution. Customers may provide code that could be destructive, might communicate with other customer resources, or might otherwise cause undesirable effects. In response, the IaaS provider may determine whether to run the code provided by the customer to the IaaS provider.

[0206] In some examples, an IaaS provider's customer may grant the IaaS provider temporary network access and request functionality to be attached to data plane layer application 1346. The code running this functionality may execute in VMs 1366(1)-(N) and may not be configured to run anywhere else on data plane VCN 1318. Each VM 1366(1)-(N) may be connected to a customer lease 1370. The corresponding container 1371(1)-(N) contained in VMs 1366(1)-(N) may be configured to run the code. In this case, there may be dual isolation (e.g., container 1371(1)-(N) running the code, where container 1371(1)-(N) may be contained in at least one or more untrusted application subnets 1362 containing VMs 1366(1)-(N)), which can help prevent incorrect or otherwise unintended code from corrupting the IaaS provider's network or the networks of different customers. Containers 1371(1)-(N) may be communicatively coupled to customer lease 1370 and may be configured to transmit or receive data from customer lease 1370. Containers 1371(1)-(N) may not be configured to transmit or receive data from any other entity in the data plane VCN 1318. After the code execution is complete, the IaaS provider may terminate or otherwise dispose of containers 1371(1)-(N).

[0207] In some embodiments, one or more trusted application subnets 1360 may run code that can be owned or operated by an IaaS provider. In this embodiment, one or more trusted application subnets 1360 may be communicatively coupled to one or more database subnets 1330 and configured to perform CRUD operations in one or more database subnets 1330. One or more untrusted application subnets 1362 may be communicatively coupled to one or more database subnets 1330, but in this embodiment, one or more untrusted application subnets may be configured to perform read operations in one or more database subnets 1330. Containers 1371(1)-(N) that may be contained in each customer's VM 1366(1)-(N) and may run code from the customer may not be communicatively coupled to one or more database subnets 1330.

[0208] In other embodiments, the control plane VCN 1316 and the data plane VCN 1318 may be coupled without direct communication. In this embodiment, there may be no direct communication between the control plane VCN 1316 and the data plane VCN 1318. However, communication can occur indirectly through at least one method. The LPG 1310 may be established by an IaaS provider, which can facilitate communication between the control plane VCN 1316 and the data plane VCN 1318. In another example, either the control plane VCN 1316 or the data plane VCN 1318 may invoke the cloud service 1356 via the service gateway 1336. For example, an invocation of the cloud service 1356 from the control plane VCN 1316 may include a request for a service that can communicate with the data plane VCN 1318.

[0209] Figure 14 This is a block diagram 1400 illustrating another example pattern of an IaaS architecture according to at least one embodiment. Service operator 1402 (e.g., Figure 11 Service provider 1102) can communicatively couple to secure host lease 1404 (e.g., Figure 11 Secure hosting lease 1104), the secure hosting lease 1404 may include a virtual cloud network (VCN) 1406 (e.g., Figure 11 VCN 1106) and Secure Host Subnet 1408 (e.g., Figure 11 The secure host subnet 1108). VCN 1406 may include LPG 1410 (e.g., Figure 11 The LPG 1110), the LPG 1410 can be accessed via SSH VCN 1412 (e.g., LPG 1110), Figure 11The LPG 1410 in SSH VCN 1112 is communicatively coupled to SSH VCN 1412. SSH VCN 1412 may include SSH subnet 1414 (e.g., Figure 11 SSH subnet 1114), and SSH VCN 1412 can be communicatively coupled to control plane VCN 1416 via LPG 1410 contained in control plane VCN 1416 (e.g., Figure 11 The control plane VCN 1116) and coupled to the data plane VCN 1418 via the LPG 1410 contained in the data plane VCN 1418 (e.g., Figure 11 Data plane 1118). Control plane VCN 1416 and data plane VCN 1418 may be contained in service lease 1419 (e.g., Figure 11 In the service rental (1119).

[0210] The control plane VCN 1416 may include one or more LB subnets 1422 (e.g., Figure 11 The control plane DMZ layer 1420 of (one or more) LB subnets 1122) (e.g., Figure 11 The control plane DMZ layer 1120 may include (one or more) application subnets 1426 (e.g., Figure 11 The control plane application layer 1424 of (one or more) application subnets 1126 (e.g., Figure 11 The control plane application layer 1124) may include (one or more) DB subnets 1430 (e.g., Figure 13 The control plane data layer 1428 of (one or more) DB subnets 1330 (e.g., Figure 11 The control plane data layer 1128). One or more LB subnets 1422 contained in the control plane DMZ layer 1420 can be communicatively coupled to one or more application subnets 1426 contained in the control plane application layer 1424 and an Internet gateway 1434 that can be contained in the control plane VCN 1416 (e.g., Figure 11 Internet gateway 1134), and application subnet(s) 1426 can communicatively couple to DB subnet(s) 1430 contained in control plane data layer 1428 and service gateway 1436 (e.g., Figure 11 The service gateway) and Network Address Translation (NAT) gateway 1438 (e.g., Figure 11 (NAT gateway 1138). The control plane VCN 1416 may include the service gateway 1436 and the NAT gateway 1438.

[0211] Data plane VCN 1418 may include data plane application layer 1446 (e.g., Figure 11 Data plane application layer 1146), data plane DMZ layer 1448 (e.g., Figure 11 Data plane DMZ layer 1448), and data plane data layer 1450 (e.g., Figure 11 The data plane data layer 1150). The data plane DMZ layer 1448 may include one or more trusted application subnets 1460 that can be communicatively coupled to the data plane application layer 1446 (e.g., Figure 13 (one or more) trusted application subnets 1360 and (one or more) untrusted application subnets 1462 (e.g., Figure 13 The data plane includes one or more untrusted application subnets 1362 and one or more LB subnets 1422 of an Internet gateway 1434 contained in data plane VCN 1418. One or more trusted application subnets 1460 may communicatively couple to a service gateway 1436 contained in data plane VCN 1418, a NAT gateway 1438 contained in data plane VCN 1418, and one or more DB subnets 1430 contained in data plane data layer 1450. One or more untrusted application subnets 1462 may communicatively couple to a service gateway 1436 contained in data plane VCN 1418 and one or more DB subnets 1430 contained in data plane data layer 1450. Data plane data layer 1450 may include one or more DB subnets 1430 that may communicatively couple to a service gateway 1436 contained in data plane VCN 1418.

[0212] One or more untrusted application subnets 1462 may include a primary VNIC 1464(1)-(N) communicatively coupled to tenant virtual machines (VMs) 1466(1)-(N) residing within one or more untrusted application subnets 1462. Each tenant VM 1466(1)-(N) may run code in a corresponding container 1467(1)-(N) and is communicatively coupled to an application subnet 1426 that may be contained in a data plane application layer 1446 contained in a container egress VCN 1468. A corresponding secondary VNIC 1472(1)-(N) may facilitate communication between one or more untrusted application subnets 1462 contained in a data plane VCN 1418 and the application subnet contained in a container egress VCN 1468. The container egress VCN may include a public internet 1454 (e.g., Figure 11 The public internet (1154) uses NAT gateway 1438.

[0213] Internet gateway 1434, contained in control plane VCN 1416 and data plane VCN 1418, can be communicatively coupled to metadata management service 1452 (e.g., Figure 11 A metadata management system 1152 is provided, which can communicatively couple to the public internet 1454. The public internet 1454 can communicatively couple to a NAT gateway 1438 contained in a control plane VCN 1416 and a data plane VCN 1418. A service gateway 1436 contained in a control plane VCN 1416 and a data plane VCN 1418 can communicatively couple to a cloud service 1456.

[0214] In some examples, Figure 14 The architecture shown in block diagram 1400 can be considered as Figure 13 This is an exception to the pattern shown in the architecture of block diagram 1300, and this pattern may be what the IaaS provider's customers would expect if the IaaS provider cannot communicate directly with the customer (e.g., in a disconnected region). The customer can access in real time the corresponding container 1467(1)-(N) contained in each customer's VM 1466(1)-(N). Container 1467(1)-(N) can be configured to invoke the corresponding auxiliary VNIC 1472(1)-(N) contained in one or more application subnets 1426 of the data plane application layer 1446, which may be contained in the container egress VCN 1468. The auxiliary VNIC 1472(1)-(N) can transmit the invocation to NAT gateway 1438, which can then transmit the invocation to the public internet 1454. In this example, containers 1467(1)-(N), which can be accessed by clients in real time, can be isolated from the control plane VCN 1416 and from other entities contained in the data plane VCN 1418. Containers 1467(1)-(N) can also be isolated from resources from other clients.

[0215] In other examples, a client can use containers 1467(1)-(N) to invoke cloud service 1456. In this example, the client can run code within containers 1467(1)-(N) requesting a service from cloud service 1456. Container 1467(1)-(N) can then forward the request to a secondary VNIC 1472(1)-(N), which can then forward the request to a NAT gateway, which can forward the request to the public internet 1454. The public internet 1454 can then forward the request via internet gateway 1434 to one or more LB subnets 1422 contained in control plane VCN 1416. In response to determining that the request is valid, the one or more LB subnets can forward the request to one or more application subnets 1426, which can then forward the request to cloud service 1456 via service gateway 1436.

[0216] It should be recognized that the IaaS architectures 1100, 1200, 1300, and 1400 depicted in the figures may have other components besides those depicted. Furthermore, the embodiments shown in the figures are merely some examples of cloud infrastructure systems that can be incorporated into embodiments of this disclosure. In some other embodiments, the IaaS system may have more or fewer components than shown in the figures, may combine two or more components, or may have different configurations or component arrangements.

[0217] In some embodiments, the IaaS system described herein may include application suites, middleware, and database service offerings delivered to customers in a self-service, subscription-based, elastically scalable, reliable, highly available, and secure manner. An example of such an IaaS system is the Oracle Cloud Infrastructure (OCI) provided by this assignee.

[0218] Figure 15 An example computer system 1500, in which various embodiments can be implemented, is illustrated. System 1500 can be used to implement any of the computer systems described above. As shown, computer system 1500 includes a processing unit 1504 that communicates with a plurality of peripheral subsystems via a bus subsystem 1502. These peripheral subsystems may include a processing acceleration unit 1506, an I / O subsystem 1508, a storage subsystem 1518, and a communication subsystem 1524. Storage subsystem 1518 includes a tangible computer-readable storage medium 1522 and system memory 1510.

[0219] Bus subsystem 1502 provides a mechanism for allowing various components and subsystems of computer system 1500 to communicate with each other as intended. While bus subsystem 1502 is schematically shown as a single bus, alternative embodiments of the bus subsystem may utilize multiple buses. Bus subsystem 1502 can be any of several types of bus architectures, including memory buses or memory controllers, peripheral buses, and local buses using any of the various bus architectures available. For example, such architectures may include Industry Standard Architecture (ISA) buses, Micro Channel Architecture (MCA) buses, Enhanced ISA (EISA) buses, Video Electronics Standards Association (VESA) local buses, and Peripheral Component Interconnect (PCI) buses, which may be implemented as Mezzanine buses manufactured according to the IEEE P1386.1 standard.

[0220] A processing unit 1504, which may be implemented as one or more integrated circuits (e.g., a conventional microprocessor or microcontroller), controls the operation of the computer system 1500. One or more processors may be included in the processing unit 1504. These processors may include single-core or multi-core processors. In some embodiments, the processing unit 1504 may be implemented as one or more independent processing units 1532 and / or 1534, each including a single-core or multi-core processor. In other embodiments, the processing unit 1504 may also be implemented as a quad-core processing unit formed by integrating two dual-core processors into a single chip.

[0221] In various embodiments, processing unit 1504 can execute various programs in response to program code and can maintain multiple concurrently executing programs or processes. At any given time, some or all of the program code to be executed can reside in processor(s) 1504 and / or storage subsystem 1518. With appropriate programming, processor(s) 1504 can provide the various functions described above. Computer system 1500 may additionally include processing acceleration unit 1506, which may include digital signal processor (DSP), dedicated processor, etc.

[0222] I / O subsystem 1508 may include user interface input devices and user interface output devices. User interface input devices may include keyboards, pointing devices such as mice or trackballs, touchpads or touchscreens integrated into a display, scroll wheels, click wheels, dials, buttons, switches, audio input devices with voice command recognition systems, microphones, and other types of input devices. User interface input devices may include, for example, motion sensing and / or gesture recognition devices, such as the Microsoft Kinect motion sensor, which enables users to control and interact with input devices such as the Microsoft Xbox 360 game controller via a natural user interface using gestures and voice commands. User interface input devices may also include eye gesture recognition devices, such as the Google Glass blink detector, which detects eye activity from the user (e.g., "blinking" when taking a photo and / or making menu selections) and translates the eye gesture into input in an input device (e.g., Google Glass). Furthermore, user interface input devices may include voice recognition sensing devices that enable users to interact with a voice recognition system (e.g., the Siri navigator) via voice commands.

[0223] User interface input devices may also include, but are not limited to, 3D mice, joysticks or pointing sticks, game panels and drawing tablets, as well as audio / video devices such as speakers, digital cameras, digital camcorders, portable media players, webcams, image scanners, fingerprint scanners, barcode readers, 3D scanners, 3D printers, laser rangefinders, and eye-tracking devices. Furthermore, user interface input devices may include, for example, medical imaging input devices such as computed tomography (CT), magnetic resonance imaging (MRI), positron emission tomography (PET), and medical ultrasound equipment. User interface input devices may also include, for example, audio input devices such as MIDI keyboards, digital musical instruments, etc.

[0224] User interface output devices may include display subsystems, indicator lights, or non-visual displays such as audio output devices, etc. Display subsystems may be cathode ray tubes (CRTs), flat panel devices such as those using liquid crystal displays (LCDs) or plasma displays, projection devices, touchscreens, etc. Generally, the term "output device" is intended to include all possible types of devices and mechanisms for outputting information from computer system 1500 to a user or other computer. For example, user interface output devices may include, but are not limited to, various display devices that visually convey text, graphics, and audio / video information, such as monitors, printers, speakers, headphones, car navigation systems, plotters, voice output devices, and modems.

[0225] Computer system 1500 may include a storage subsystem 1518 containing software elements, shown as currently located in system memory 1510. System memory 1510 may store program instructions that can be loaded and executed on processing unit 1504, as well as data generated during the execution of these programs.

[0226] Depending on the configuration and type of computer system 1500, system memory 1510 may be volatile (such as random access memory (RAM)) and / or non-volatile (such as read-only memory (ROM), flash memory, etc.). RAM typically contains data and / or program modules that can be immediately accessed by processing unit 1504 and / or are currently being operated and executed by processing unit 1504. In some implementations, system memory 1510 may include various different types of memory, such as static random access memory (SRAM) or dynamic random access memory (DRAM). In some implementations, a basic input / output system (BIOS), which contains basic routines that facilitate the transfer of information between elements of computer system 1500 during startup, may typically be stored in ROM. As an example, but not a limitation, system memory 1510 also includes application programs 1512, program data 1514, and an operating system 1516, which may include client applications, web browsers, middleware applications, relational database management systems (RDBMS), etc. As an example, operating system 1516 may include various versions of Microsoft Windows, Apple Macintosh and / or Linux operating systems, various commercially available UNIX or UNIX-like operating systems (including but not limited to various GNU / Linux operating systems, Google Chrome operating system, etc.) and / or mobile operating systems such as iOS, Windows Phone, Android OS, BlackBerry 17 OS and Palm OS.

[0227] Storage subsystem 1518 may also provide a tangible computer-readable storage medium for storing basic programming and data structures that provide the functionality of some embodiments. Software (programs, code modules, instructions) that provides the above-described functionality when executed by a processor may be stored in storage subsystem 1518. These software modules or instructions may be executed by processing unit 1504. Storage subsystem 1518 may also provide a repository for storing data used according to this disclosure.

[0228] The storage subsystem 1500 may also include a computer-readable storage medium reader 1520 that can be further connected to the computer-readable storage medium 1522. Together with and optionally in conjunction with the system memory 1510, the computer-readable storage medium 1522 can comprehensively represent a remote, local, fixed, and / or removable storage device plus storage medium for temporarily and / or more persistently containing, storing, transmitting, and retrieving computer-readable information.

[0229] The computer-readable storage medium 1522 containing code or portions thereof may also include any suitable medium known or used in the art, including storage and communication media, such as, but not limited to, volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing and / or transmitting information. This may include tangible computer-readable storage media such as RAM, ROM, electrically erasable programmable ROM (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic tape cassettes, magnetic tape, disk storage or other magnetic storage devices, or other tangible computer-readable media. This may also include non-tangible computer-readable media such as data signals, data transmissions, or any other medium that can be used to transmit desired information and can be accessed by the computing system 1500.

[0230] As an example, computer-readable storage medium 1522 may include a hard disk drive that reads or writes to a non-removable non-volatile magnetic medium, a disk drive that reads or writes to a removable non-volatile magnetic disk, and an optical disc drive that reads or writes to a removable non-volatile optical disc (such as a CD-ROM, DVD, and Blu-ray disc or other optical media). Computer-readable storage medium 1522 may include, but is not limited to, Zip drives, flash memory cards, Universal Serial Bus (USB) flash memory drives, Secure Digital (SD) cards, DVD discs, digital audio tapes, and so on. Computer-readable storage medium 1522 may also include solid-state drives (SSDs) based on non-volatile memory (such as flash memory-based SSDs, enterprise flash drives, solid-state ROMs, etc.), volatile memory-based SSDs (such as solid-state RAM, dynamic RAM, static RAM), DRAM-based SSDs, magnetoresistive RAM (MRAM) SSDs, and hybrid SSDs using a combination of DRAM-based and flash memory-based SSDs. Disk drives and their associated computer-readable media can provide non-volatile storage for computer-readable instructions, data structures, program modules and other data for computer system 1500.

[0231] The communication subsystem 1524 provides an interface to other computer systems and networks. The communication subsystem 1524 serves as an interface for receiving data from other systems and sending data from computer system 1500 to other systems. For example, the communication subsystem 1524 enables computer system 1500 to connect to one or more devices via the Internet. In some embodiments, the communication subsystem 1524 may include radio frequency (RF) transceiver components (e.g., advanced data network technologies using cellular telephone technologies, such as 3G, 4G, or EDGE (Enhanced Data Rates for Global Evolution), Wi-Fi (IEEE 802.11 series standards), or other mobile communication technologies, or any combination thereof), GPS receiver components, and / or other components for accessing wireless voice and / or data networks. In some embodiments, as an addition to or alternative to the wireless interface, the communication subsystem 1524 may provide a wired network connection (e.g., Ethernet).

[0232] In some embodiments, the communication subsystem 1524 may also represent one or more users who can use the computer system 1500 to receive input communications in the form of structured and / or unstructured data feeds 1526, event streams 1528, event updates 1530, etc.

[0233] As an example, the communication subsystem 1524 can be configured to receive data feeds 1526 in real time from users of social networks and / or other communication services, such as Twitter feeds, Facebook updates, web feeds such as Rich Site Summary (RSS) feeds, and / or real-time updates from one or more third-party information sources.

[0234] Furthermore, the communication subsystem 1524 can also be configured to receive data in the form of a continuous data stream, which may include event streams 1528 and / or event updates 1530 that are essentially continuous or unbounded real-time events without a clearly defined termination. Examples of applications that generate continuous data may include, for example, sensor data applications, financial quotation machines, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, vehicle traffic monitoring, and so on.

[0235] The communication subsystem 1524 can also be configured to output structured and / or unstructured data feeds 1526, event streams 1528, event updates 1530, etc. to one or more databases, which can communicate with one or more streaming data source computers coupled to the computer system 1500.

[0236] The computer system 1500 can be one of a variety of types, including handheld portable devices (e.g., iPhone cellular phones, iPad computing tablets, PDAs), wearable devices (e.g., Google Glass head-mounted displays), PCs, workstations, mainframes, information stations, server racks, or any other data processing systems.

[0237] Due to the ever-evolving nature of computers and networks, the description of the computer system 1500 depicted in the figures is merely a concrete example. Many other configurations with more or fewer components than the system depicted in the figures are possible. For example, custom hardware may be used and / or specific elements may be implemented using hardware, firmware, software (including applets), or combinations thereof. Additionally, connections to other computing devices, such as network input / output devices, may also be employed. Based on the disclosure and teachings provided herein, those skilled in the art will recognize other ways and / or methods for implementing the various embodiments.

[0238] While specific embodiments of this disclosure have been described, various modifications, alterations, alternative constructions, and equivalents are also included within the scope of this disclosure. The embodiments of this disclosure are not limited to operation within certain specific data processing environments, but can be freely operated within multiple data processing environments. Furthermore, although embodiments of this disclosure have been described using a specific series of transactions and steps, those skilled in the art will understand that the scope of this disclosure is not limited to the described series of transactions and steps. Various features and aspects of the above embodiments can be used individually or in combination.

[0239] Furthermore, while embodiments of this disclosure have been described using specific combinations of hardware and software, it should be recognized that other combinations of hardware and software are also within the scope of this disclosure. Embodiments of this disclosure may be implemented using only hardware, or only software, or a combination thereof. The various processes described herein may be implemented in any combination on the same processor or on different processors. Accordingly, where a component or module is described as being configured to perform certain operations, such configuration may be accomplished, for example, by designing electronic circuitry to perform the operations, by programming programmable electronic circuitry (such as a microprocessor), or any combination thereof. Processes may communicate using a variety of technologies, including but not limited to conventional technologies for inter-process communication, and different pairs of processes may use different technologies, or the same pair of processes may use different technologies at different times.

[0240] Accordingly, the specification and drawings are to be considered illustrative rather than restrictive. However, it will be apparent that additions, omissions, deletions, and other modifications and changes may be made therein without departing from the broader spirit and scope set forth in the claims. Therefore, while specific disclosed embodiments have been described, they are not intended to be limiting. Various modifications and equivalents are within the scope of the following claims.

[0241] In the context of describing the disclosed embodiments (particularly in the context of the following claims), the terms “a,” “an,” and “the,” and similar designations, are to be interpreted as covering both singular and plural, unless otherwise indicated herein or obviously contradicted by the context. Unless otherwise stated, the terms “comprising,” “having,” “including,” and “containing” are to be interpreted as open-ended terms (i.e., meaning “including but not limited to”). The term “connected” should be interpreted as partially or wholly contained in, attached to, or joined together, even if something exists in between. Unless otherwise indicated herein, the enumeration of value ranges herein is intended only as a shorthand method for individually referencing each individual value falling within that range, and each individual value is incorporated into the specification as if it were individually enumerated herein. Unless otherwise indicated herein or obviously contradicted by the context, all methods described herein can be performed in any suitable order. The use of any and all examples or exemplary language (e.g., “such as”) provided herein is intended only to better illustrate embodiments of this disclosure and does not constitute a limitation on the scope of this disclosure, unless otherwise stated. Nothing in the specification should be construed as indicating that any unclaimed element is essential to the practice of this disclosure.

[0242] Disjunctive language, such as the phrase “at least one of X, Y, or Z”, unless otherwise explicitly stated, is intended to be understood in the context generally used to represent items, terms, etc., and may be X, Y, or Z, or any combination thereof (e.g., X, Y, and / or Z). Therefore, such disjunctive language is generally not intended to, and should not, imply that some embodiments require the presence of at least one of X, at least one of Y, or at least one of Z, each individually.

[0243] This document describes preferred embodiments of the present disclosure, including the best modes known to the inventors for carrying out the present disclosure. Variations of those preferred embodiments will become apparent to those skilled in the art upon reading the foregoing description. The inventors expect that those skilled in the art should be able to suitably employ such variations and that the present disclosure can be practiced in ways other than those specifically described herein. Accordingly, the present disclosure includes all modifications and equivalents to the subject matter recited in the appended claims, where permitted by applicable law. Furthermore, unless otherwise indicated herein, or unless obviously contradicted by the context, the present disclosure includes any combination of the foregoing elements in all its possible variations.

[0244] All references cited in this article, including publications, patent applications and patents, are incorporated into this article by reference to the same extent as if each reference individually and specifically indicated to be incorporated by reference and elaborated in full in this article.

[0245] In the foregoing specification, various aspects of this disclosure have been described with reference to specific embodiments thereof, but those skilled in the art will recognize that this disclosure is not limited thereto. The various features and aspects of the foregoing disclosure may be used individually or in combination. Furthermore, embodiments may be used in any number of settings and applications other than those described herein without departing from the broader spirit and scope of this specification. Accordingly, this specification and the accompanying drawings should be considered illustrative rather than restrictive.

Claims

1. A method comprising: A first request is received by a multi-cloud gateway (MCG) implemented in the first cloud environment to perform a first operation in the second cloud environment; In response to receiving the first request, the MCG generates a first API call pointing to the second cloud environment; The MCG causes the first API call to be transmitted to the second cloud environment; The MCG receives a second request to perform a second operation in a third cloud environment; In response to receiving the second request, the MCG generates a second API call pointing to the third cloud environment; as well as The MCG causes the second API call to be transmitted to the third cloud environment, wherein each of the first cloud environment, the second cloud environment, and the third cloud environment is provided by a different cloud service provider.

2. The method of claim 1, wherein the first API call is directed to a first endpoint associated with a service provided by the second cloud environment, and the second API call is directed to a second endpoint associated with another service provided by the third cloud environment, the first endpoint being different from the second endpoint.

3. The method according to claim 1, further comprising: The software development kit (SDK) component implemented in the first cloud environment receives the first request to perform the first operation in the second cloud environment; The code generator included in the SDK component generates a modified first request by transforming the first request into a common format. as well as The modified first request is transmitted to the MCG by the SDK component.

4. The method according to claim 3, further comprising: The SDK component obtains the API specifications of the second cloud environment that provides the first service associated with the first operation; as well as Based on the API specifications of the second cloud environment, the modified first request is generated in the general format.

5. The method according to claim 3, wherein, The modified first request includes metadata identifying at least the following items: The cloud service provider for the second cloud environment The type of the first operation, The first endpoint in the second cloud environment to which the first request is directed, and Cloud link information associated with the first request.

6. The method according to claim 5, wherein, The cloud link information includes a mapping between a user's rental in the first cloud environment and the user's account in the second cloud environment.

7. The method according to claim 1, further comprising: The user who issued the first request is verified by the MCG; In response to successful user verification, cloud link information about the first cloud environment and the second cloud environment is obtained; as well as The first request is transmitted to the second cloud environment.

8. The method of claim 1, wherein the first API call to the first cloud environment and the second API call to the second cloud environment are each REST API calls.

9. The method according to claim 1, further comprising: The user's identity in the first cloud environment is mapped to another identity of the user in the second cloud environment via a token exchange service.

10. The method of claim 9, further comprising: The MCG receives a response to the first request from the second cloud environment, wherein the second cloud environment is configured to determine, based on the token exchange service, whether the user is permitted to issue the first request requesting the first operation.

11. A computing device, comprising: processor; as well as A memory containing instructions that, when executed by a processor, cause a computing device deployed in the first cloud environment to at least: Receive the first request to perform the first operation in the second cloud environment; In response to receiving the first request, a first API call is generated pointing to the second cloud environment; This causes the first API call to be transmitted to the second cloud environment; Receive a second request to perform a second operation in a third cloud environment; In response to receiving the second request, a second API call is generated pointing to the third cloud environment; as well as The second API call is transmitted to the third cloud environment, wherein each of the first cloud environment, the second cloud environment, and the third cloud environment is provided by a different cloud service provider.

12. The computing device of claim 11, wherein the first API call is directed to a first endpoint associated with a service provided by the second cloud environment, and the second API call is directed to a second endpoint associated with another service provided by the third cloud environment, the first endpoint being different from the second endpoint.

13. The computing device according to claim 11, wherein, The computing device is also configured to: The software development kit (SDK) component implemented in the first cloud environment receives the first request to perform the first operation in the second cloud environment; The modified first request is generated by the code generator included in the SDK component by transforming the first request into a common format. as well as The modified first request is transmitted to the computing device via the SDK component.

14. The computing device according to claim 13, wherein, The computing device is also configured to: The API specifications of the second cloud environment that provides the first service associated with the first operation are obtained through the SDK component; and Based on the API specifications of the second cloud environment, the modified first request is generated in the general format.

15. The computing device according to claim 13, wherein, The modified first request includes metadata identifying at least the following items: The cloud service provider for the second cloud environment The type of the first operation, The first endpoint in the second cloud environment to which the first request is directed, and Cloud link information associated with the first request.

16. The computing device according to claim 15, wherein, The cloud link information includes a mapping between a user's rental in the first cloud environment and the user's account in the second cloud environment.

17. The computing device according to claim 11, wherein, The computing device is also configured to: Verify the user who issued the first request; In response to successful user verification, cloud link information about the first cloud environment and the second cloud environment is obtained; as well as The first request is transmitted to the second cloud environment.

18. A non-transitory computer-readable medium storing specific computer-executable instructions, which, when executed by a processor, cause a computer system to perform a method, the method comprising: A first request is received by a multi-cloud gateway (MCG) implemented in the first cloud environment to perform a first operation in the second cloud environment; In response to receiving the first request, the MCG generates a first API call pointing to the second cloud environment; The MCG causes the first API call to be transmitted to the second cloud environment; The MCG receives a second request to perform a second operation in a third cloud environment; In response to receiving the second request, the MCG generates a second API call pointing to the third cloud environment; as well as The MCG causes the second API call to be transmitted to the third cloud environment, wherein each of the first cloud environment, the second cloud environment, and the third cloud environment is provided by a different cloud service provider.

19. The non-transitory computer-readable medium of claim 18, wherein the first API call is directed to a first endpoint associated with a service provided by the second cloud environment, and the second API call is directed to a second endpoint associated with another service provided by the third cloud environment, the first endpoint being different from the second endpoint.

20. The non-transitory computer-readable medium of claim 18, further comprising: The software development kit (SDK) component implemented in the first cloud environment receives the first request to perform the first operation in the second cloud environment; The code generator included in the SDK component generates a modified first request by transforming the first request into a common format. as well as The modified first request is transmitted to the MCG by the SDK component.

Images