Identity Management for Virtual Private Label Clouds

Abstract

Novel techniques are disclosed for enabling identity cloud services for virtual private label clouds (vPLCs). A vPLC is created for a cloud service provider (CSP) reseller using a CSP-provided infrastructure within a region so that the CSP reseller can offer one or more reseller-supplied cloud services to the reseller's customers. In some embodiments, identity management can be configured using either a shared identity cloud services (IDCS) stack model or an independent IDCS stack model. In one embodiment, a two-tier vPLC-aware identity management function is implemented for the CSP reseller and the reseller's customers.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application is a nonprovisional application of U.S. Provisional Patent Application No. 63 / 407,571, filed September 16, 2022, entitled "VIRTUAL PRIVATE LABEL CLOUD," and claims the benefit of and priority under 35 U.S.C. § 119(e), which is incorporated herein by reference in its entirety for all purposes.

[0002] This disclosure is also related to the following patent applications, the entire contents of each of which are incorporated herein by reference for all purposes: (1) Non-provisional application Ser. No. 18 / 368,881 (Attorney Docket No. 088325-1311130 (344800US)), entitled "Virtual Private Label Cloud," filed concurrently with this application; (2) Non-provisional application Ser. No. 18 / 368,863 (Attorney Docket No. 088325-1311132 (344900US)), entitled "CONSOLE CUSTOMIZATION FOR VIRTUAL PRIVATE LABEL CLOUDS," filed concurrently herewith; (3) Non-provisional application Ser. No. 18 / 368,870 (Attorney Docket No. 088325-1311140 (US 345000)), entitled "RESOURCE ALLOCATION FOR VIRTUAL PRIVATE LABEL CLOUDS," filed concurrently herewith; (4) Non-provisional application Ser. No. 18 / 368,877, entitled "ENDPOINTS FOR VIRTUAL PRIVATE LABEL CLOUDS," filed concurrently herewith (Attorney Docket No. 088325-1311138 (345100US)); (5) Non-provisional application Ser. No. 18 / 468,024 (Attorney Docket No. 088325-1311136 (345300US)), entitled "METADATA CUSTOMIZATION FOR VIRTUAL PRIVATE LABEL CLOUDS," filed concurrently herewith; (6) Non-provisional application Ser. No. 18 / 468,037, entitled "REMOTE DATA PLANES FOR VIRTUAL PRIVATE LABEL CLOUDS," filed concurrently herewith (Attorney Docket No. 088325-1311141 (345400US)); (7) Non-provisional application Ser. No. 18 / 468,238, entitled "CONNECTIVITY FOR VIRTUAL PRIVATE LABEL CLOUDS," filed concurrently herewith (Attorney Docket No. 088325-1311142 (345500US)); (8) Non-provisional application Ser. No. 18 / 468,047 (Attorney Docket No. 088325-1315390 (346900US)), entitled "RESOURCE USAGE MONITORING, BILLING AND ENFORCEMENT FOR VIRTUAL PRIVATE LABEL CLOUDS," filed concurrently herewith; and (9) Non-provisional application Ser. No. 18 / 468,044 (Attorney Docket No. 088325-1315384 (346800US)), entitled "CLOUD INFRASTRUCTURE-BASED ONLINE PUBLISHING PLATFORMS FOR VIRTUAL PRIVATE LABEL CLOUDS," filed concurrently herewith.

[0003] Field The present disclosure generally relates to techniques for providing cloud services. More specifically, novel techniques are disclosed for enabling identity cloud services for virtual private label clouds (vPLCs). A vPLC can be created for a cloud service provider (CSP) reseller using CSP-provided infrastructure within a region and can be used to provide one or more reseller-supplied cloud services to the reseller's customers. [Background technology]

[0004] background Cloud computing, due to its ubiquitous presence and ease of access to essential data, is gradually becoming a part of modern life and plays a vital role in everyday activities. However, just like in the early telecommunications era, there are only a few companies in the cloud infrastructure industry. Therefore, there is a need to make cloud infrastructure services more widely available. Summary of the Invention

[0005] A brief overview

[0001] The present disclosure generally relates to techniques for providing cloud services. More specifically, a novel technique is disclosed for enabling identity cloud services for a virtual private label cloud (vPLC). A vPLC is created for a cloud service provider (CSP) reseller using a CSP-provided infrastructure within a region so that the CSP reseller can offer one or more reseller-supplied cloud services to the reseller's customers. Various embodiments are described herein, including methods, systems, non-transitory computer-readable storage media storing programs, code, or instructions executable by one or more processors, etc.

[0006]

[0001] The present disclosure generally relates to techniques for providing cloud services. More particularly, novel techniques are disclosed that enable cloud infrastructure in a region provided by a cloud service provider (CSP) to be used to create one or more virtual private clouds (referred to herein as virtual private label clouds or "vPLCs") for providing cloud services supplied by the CSP to customers of the CSP. Various embodiments are described herein, including methods, systems, non-transitory computer-readable storage media storing programs, code, or instructions executable by one or more processors, and the like.

[0007] A vPLC created in accordance with the various techniques described herein can be used for a variety of purposes. For example, in one use case, a vPLC may be created for a reseller who is a customer of a CSP and used by the reseller to provide one or more reseller-supplied cloud services to the reseller's customers. As another use case, a vPLC may be used as a virtual data center and associated with a realm different from the realm associated with cloud infrastructure in a region provided by the CSP.

[0008] In certain embodiments, a vPLC represents a virtual cloud that includes a set(s) of resources that have been allocated to the vPLC from CSP-provided infrastructure within a region. One or more vPLCs can be created using the CSP-provided infrastructure within a region.

[0009] In certain embodiments, regional infrastructure provided by a CSP in a region can be used both to provide cloud services to the CSP's customers and to provide reseller-supplied cloud services to customers of resellers who are customers of the CSP. This is achieved by creating one or more vPLCs for one or more resellers from the CSP-provided regional infrastructure, where a vPLC created for a particular reseller can be used to provide reseller-supplied and reseller-branded cloud services to that particular reseller's customers. This allows the reseller to become a provider of reseller-branded cloud services to its customers without having to invest in the infrastructure required to provide those cloud services. Instead, the reseller uses the infrastructure provided by the CSP to provide reseller-branded cloud services to the reseller's customers. In this manner, a first portion of the infrastructure provided by the CSP within the region may be used to provide CSP-supplied cloud services to customers of the CSP, a second portion of the infrastructure provided by the CSP within the region and allocated to a first vPLC created for a first reseller may be used to provide first reseller-supplied cloud services to customers of the first reseller, a third portion of the infrastructure provided by the CSP within the region and allocated to a second vPLC created for a second reseller may be used to provide second reseller-supplied cloud services to customers of the second reseller, and so on.

[0010] In an embodiment, a method includes: using a first portion of a cloud service provider (CSP)-provided infrastructure in a first region to provide one or more CSP-supplied cloud services to one or more customers of the CSP; allocating a second portion of the CSP-provided infrastructure to a first virtual private label cloud (vPLC); creating a first vPLC for a first reseller based on the CSP-provided infrastructure; providing one or more first reseller-supplied cloud services to one or more customers of the first reseller using the first vPLC; and creating a CS based on the CSP-provided infrastructure. P, configuring identity management for a first reseller based on a CSP-provided infrastructure in a region, creating identity information associated with a customer of the CSP in a first namespace, creating identity information associated with the first reseller in a second namespace, performing identity management functions for the customer of the CSP using the identity information associated with the customer of the CSP, and performing identity management functions for a user of the first reseller using the identity information associated with the first reseller.

[0011] In yet another embodiment, identity management functions for customers of the CSP are performed by a first identity services stack that includes a first set of CSP-provided infrastructure resources.

[0012] In yet another embodiment, identity management functions for users of the first reseller are performed by a first identity services stack.

[0013] In yet another embodiment, identity management functions for users of the first reseller are performed by a second identity services stack that includes a second set of resources of the CSP-provided infrastructure.

[0014] In yet another embodiment, the second identity service stack is a clone of the first identity service stack.

[0015] In yet another embodiment, creating the identity information associated with the first reseller is performed by a first identity service stack.

[0016] In yet another embodiment, creating the identity information associated with the first reseller is performed by a second identity services stack.

[0017] In yet another embodiment, the method further includes configuring identity management for a second reseller of the CSP based on the CSP-provided infrastructure and creating identity information associated with the second reseller in a third namespace.

[0018] In yet another embodiment, the identity information associated with the first reseller includes identity information for users of the first reseller and users of customers of the first reseller.

[0019] In various embodiments, a system is provided that includes one or more data processors and a non-transitory computer-readable medium that contains instructions that, when executed on the one or more data processors, cause the one or more data processors to perform some or all of one or more of the methods disclosed herein.

[0020] In various embodiments, a non-transitory computer-readable medium storing computer-executable instructions that, when executed by one or more processors, cause one or more processors of a computer system to perform one or more methods disclosed herein.

[0021] In various embodiments, a computer program product comprising computer programs / instructions that, when executed by a processor, cause the processor to perform any of the methods disclosed herein.

[0022] The techniques described above and below can be implemented in multiple ways and in multiple contexts. Some exemplary implementations and contexts are provided with reference to the accompanying drawings, as described in more detail below. However, the following implementations and contexts are only a few of many. [Brief explanation of the drawings]

[0023] [Figure 1] FIG. 1 is a high-level diagram of a distributed environment illustrating a virtual or overlay cloud network hosted by a cloud service provider infrastructure according to an embodiment. [Figure 2] FIG. 2 is a simplified architectural diagram of physical components within a physical network within CSPI according to one embodiment. [Figure 3] FIG. 1 illustrates an exemplary configuration within CSPI in which a host machine is connected to multiple network virtualization devices (NVDs), according to one embodiment. [Figure 4] A diagram illustrating connectivity between host machines and NVDs to provide I / O virtualization to support multi-tenancy functionality in one embodiment. [Figure 5] FIG. 2 is a simplified block diagram of a physical network provided by CSPI according to one embodiment. [Figure 6]1 is a block diagram of a distributed environment illustrating an example of a virtual private label cloud (vPLC) hosted by a CSP-provided infrastructure within a region, according to an embodiment. [Figure 7] FIG. 1 illustrates relationships between a CSP, the CSP's direct and reseller customers and their associated users, and individual reseller customers and their associated users, according to an embodiment. [Figure 8] FIG. 1 illustrates the relationship between resources allocated to tenancies for a CSP's direct customers, tenancies for the CSP's resellers, and tenancies for the individual resellers' customers, according to an embodiment. [Figure 9A] FIG. 1 illustrates a mechanism for storing information within a tenancy for a customer of an individual reseller of a CSP, according to an embodiment. [Figure 9B] FIG. 1 illustrates a mechanism for storing information within a tenancy for a customer of an individual reseller of a CSP, according to an embodiment. [Figure 10] FIG. 1 is a flow diagram illustrating an example of a vPLC setup process for a CSP reseller, according to an embodiment. [Figure 11] 1 is a block diagram of a distributed environment illustrating an example of a virtual private label cloud (vPLC) hosted by a CSP-provided infrastructure within a region, according to an embodiment. [Figure 12] FIG. 1 is a block diagram illustrating a vPLC segmentation model for resource partitioning, according to an embodiment. [Figure 13] FIG. 1 is a block diagram illustrating a vPLC nested model for resource partitioning, according to an embodiment. [Figure 14] FIG. 1 is a flow diagram illustrating a sign-up process for customers of a reseller associated with a vPLC, according to an embodiment. [Figure 15] FIG. 10 is a flow diagram illustrating a login process for a user associated with a customer of a reseller associated with a vPLC, according to an embodiment. [Figure 16] FIG. 1 is a flow diagram illustrating a resource allocation and provisioning process for a reseller's customer associated with a vPLC, according to an embodiment. [Figure 17] FIG. 1 is a flow diagram illustrating a resource allocation and provisioning process for a reseller's customer associated with a vPLC, according to an embodiment. [Figure 18] FIG. 1 is a simplified diagram illustrating a use case in which a vPLC is hosted by infrastructure in a first realm but associated with a second, different realm, according to an embodiment. [Figure 19] FIG. 1 is a flow diagram illustrating an exemplary method for creating a vPLC, where the vPLC is hosted by a CSP-provided infrastructure in a particular region in a particular realm, but is virtually associated with different regions in different realms, according to an embodiment. [Figure 20] FIG. 1 illustrates a shared IDCS stack model, according to an embodiment. [Figure 21] FIG. 1 illustrates another embodiment of a shared IDCS stack model, according to an embodiment. [Figure 22] FIG. 1 illustrates an independent IDCS stack model, according to an embodiment. [Figure 23] FIG. 10 is a flow diagram illustrating an identity management configuration process at a reseller level, according to an embodiment. [Figure 24] FIG. 10 is a flow diagram illustrating another embodiment of an identity management configuration process at a reseller level, according to an embodiment. [Figure 25] FIG. 1 is a flow diagram illustrating an identity management configuration process at a reseller customer level, according to an embodiment. [Figure 26] FIG. 1 illustrates a mechanism for storing identity information within a customer tenancy of a vPLC, according to an embodiment. [Figure 27] FIG. 1 illustrates a mechanism for storing identity information within a customer tenancy of a vPLC, according to an embodiment. [Figure 28] FIG. 1 is a flow diagram illustrating a process for using IDCS to perform identity management functions for a reseller's users, according to an embodiment. [Figure 29] FIG. 1 is a flow diagram illustrating a process of using IDCS to perform identity management functions for users of a reseller's customers, according to an embodiment. [Figure 30] FIG. 1 is a block diagram illustrating one pattern for implementing a cloud infrastructure as a service system, according to at least one embodiment. [Figure 31] FIG. 1 is a block diagram illustrating another pattern for implementing a cloud infrastructure as a service system, according to at least one embodiment. [Figure 32] FIG. 1 is a block diagram illustrating another pattern for implementing a cloud infrastructure as a service system, according to at least one embodiment. [Figure 33] FIG. 1 is a block diagram illustrating another pattern for implementing a cloud infrastructure as a service system, according to at least one embodiment. [Figure 34] FIG. 1 is a block diagram illustrating an exemplary computer system according to at least one embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0024] Detailed Description In the following description, for purposes of explanation, specific details are set forth in order to provide a thorough understanding of aspects of certain embodiments. It will be understood, however, that various embodiments may be practiced without these specific details. The drawings and descriptions are not intended to be limiting. The word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any embodiment or design described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments or designs.

[0025]

[0001] The present disclosure generally relates to techniques for providing cloud services. More particularly, novel techniques are disclosed that enable cloud infrastructure in a region provided by a cloud service provider (CSP) to be used to create one or more virtual private clouds (referred to herein as virtual private label clouds or "vPLCs") for providing cloud services supplied by the CSP to customers of the CSP. Various embodiments are described herein, including methods, systems, non-transitory computer-readable storage media storing programs, code, or instructions executable by one or more processors, and the like.

[0026] A vPLC created in accordance with the various techniques described herein can be used for a variety of purposes. For example, in one use case, a vPLC may be created for a reseller (reseller) who is a customer of a CSP and used by the reseller to provide one or more reseller-supplied cloud services to the reseller's customers. As another use case, a vPLC may be used as a virtual data center and associated with a realm different from the realm associated with cloud infrastructure in a region provided by the CSP.

[0027] In certain embodiments, a vPLC represents a virtual cloud that includes a set(s) of resources that are allocated to the vPLC from a CSP-provided infrastructure within a region. One or more vPLCs can be created using the CSP-provided infrastructure within a region. In some embodiments, a virtual private label cloud (vPLC) refers to a model and architecture provided by a CSP within a CSP-provided infrastructure within a region that enables and facilitates reseller-supplied and reseller-branded cloud services.

[0028] Just as a CSP-provided infrastructure in a region provides cloud resources that can be accessed by the CSP's customers, a vPLC created for a reseller provides a set of resources that can be accessed by the reseller's customers. From the perspective of the reseller's customers, a vPLC is like a reseller-provided infrastructure in a region that provides resources that can be accessed by the reseller's customers. A vPLC is similar to a data center in a region provided by a reseller that delivers reseller-supplied cloud services to the reseller's customers.

[0029] In certain embodiments, regional infrastructure provided by a CSP in a region can be used both to provide cloud services to the CSP's customers and to provide reseller-supplied cloud services to customers of resellers who are customers of the CSP. This is achieved by creating one or more vPLCs for one or more resellers from the CSP-provided regional infrastructure, where a vPLC created for a particular reseller can be used to provide reseller-supplied and reseller-branded cloud services to that particular reseller's customers. This allows the reseller to become a provider of reseller-branded cloud services to its customers without having to invest in the infrastructure required to provide those cloud services. Instead, the reseller uses the infrastructure provided by the CSP to provide reseller-branded cloud services to the reseller's customers.

[0030] In a reseller use case, a vPLC is created for a reseller using cloud infrastructure within a region provided by a cloud service provider (CSP). The vPLC created for the reseller can be used by the reseller to provide one or more reseller-supplied cloud services to the reseller's customers. In one embodiment, the CSP-provided regional infrastructure providing the services used by the reseller to provide one or more reseller-supplied services is partitioned into securely isolated portions, and the isolated portions (or partitions) are allocated to different resellers, such that the partitions allocated to one reseller are isolated from other resellers. In this manner, a first portion of the infrastructure provided by the CSP within a region may be used to provide CSP-supplied cloud services to the CSP's customers, a second portion of the infrastructure provided by the CSP within its region and allocated to a first vPLC created for a first reseller may be used to provide first reseller-supplied cloud services to the first reseller's customers, a third portion of the infrastructure provided by the CSP within its region and allocated to a second vPLC created for a second reseller may be used to provide second reseller-supplied cloud services to the second reseller's customers, and so on. Each portion of the infrastructure allocated to a reseller may also be further divided into a second level of securely isolated portions for providing multi-tenant cloud infrastructure services to the reseller's customers.

[0031] A reseller can be an entity such as a company or business entity or an individual. Using the techniques described herein enables a reseller entity to quickly become a cloud service provider while avoiding all of the barriers to entry discussed in the background section of this disclosure. For example, a reseller does not need to procure the infrastructure used to provide the cloud services because the offered cloud services use infrastructure provided by a CSP. To the reseller's customers, this appears as if the reseller is providing reseller-supplied cloud services. Thus, from the perspective of the reseller's customers, the reseller is a cloud service provider whose services the customers subscribe to. The reseller's customers may not even know or recognize the CSP whose infrastructure the reseller supplies and uses to provide the reseller's branded cloud services.

[0032] The ability to provide vPLC may be offered as a service by a CSP, to which one or more customers can subscribe. For example, an entity that wants to provide cloud services to its customers but does not want to invest in procuring the infrastructure required to provide the service can subscribe to this CSP-supplied vPLC service. Upon subscribing to the vPLC service, the entity is a customer of the CSP, but a "special" customer because the CSP-provided infrastructure allocated to the entity in the form of a vPLC is used to provide the entity-supplied and branded services to the entity's customers. Thus, this customer is both a customer of the CSP and a reseller of cloud services that use the vPLC created for the entity. Therefore, this entity is referred to as a "reseller" to distinguish it from the CSP's actual direct customers. A CSP's direct customer is an entity that subscribes to and consumes one or more CSP-supplied and CSP-branded services, but does not use the CSP infrastructure to sell cloud services to its customers. For purposes of this disclosure, a CSP's direct customers are also referred to as "non-reseller customers" to distinguish them from resellers, for which, as part of the vPLC service, the CSP provides infrastructure to the reseller in the form of a vPLC, which is used by the reseller to deliver the reseller's branded cloud services to the reseller's customers.

[0033] A vPLC can be used for a variety of other purposes that do not involve a reseller. As an example, a vPLC can be created and associated with a realm that is different from the realm associated with the CSP-provided regional infrastructure used to create the vPLC. For example, the CSP-provided regional infrastructure may be associated with a first realm, and the vPLC may be associated with a second realm that is different from the first realm. In such a scenario, the infrastructure corresponding to the vPLC is hosted by the first realm but is considered to belong to the second realm. This can be used for a variety of different purposes. For example, because the vPLC is associated with the second realm, it can communicate with other infrastructure (e.g., other data centers) in the second realm. Thus, data centers in the second realm can communicate with the vPLC because both the data centers and the vPLC are in the same realm and share the trust and identity profile of the second realm.

[0034] Creating and managing a vPLC is performed using infrastructure and services provided by the CSP and is a technically very complex task. When a vPLC is created for a reseller, technical functionality is provided to enable the reseller to use the vPLC to offer reseller-supplied cloud services to the reseller's customers. This includes the ability to segregate traffic between resellers, segregate traffic of the reseller's different customers, dynamically manage the allocation of CSP-provided resources to the vPLC, manage the allocation of resources allocated to the vPLC among the vPLC's different customers, meter usage of the vPLC allocated to the reseller and perform related billing functions, meter usage of the vPLC allocated among the vPLC's different customers and perform related billing functions, enable a marketplace for different resellers, and perform identity management functions for resellers and their customers. This disclosure describes various embodiments depicting various architectures and corresponding methods for implementing and enabling vPLC-related functions.

[0035] Resellers can use the vPLC created for them to offer reseller-supplied and reseller-branded cloud services and provide specialized services in the reseller's areas of expertise. Reseller-supplied services can be tailored to various segments of customers. In some embodiments, reseller-supplied cloud services may be different from cloud services supplied by the CSP. In some other embodiments, reseller-supplied cloud services may be based on cloud services supplied by the CSP; for example, the reseller-supplied services may be customized versions of CSP-supplied cloud services.

[0036] For example, a first vPLC may be created from a CSP-provided infrastructure in a first region for a first reseller that specializes in providing financial services. The first reseller's customers may be banks and other financial institutions. Thus, the first reseller can use the first vPLC to provide customized, first reseller-branded financial cloud services to its customers. A second vPLC may be created from the same CSP-provided infrastructure in the first region for a second reseller that specializes in providing telecommunications services. The second reseller's customers may be users of telecommunications services. The second reseller can use the second vPLC to provide customized, second reseller-branded telecommunications cloud services to its customers. In addition to the first and second vPLCs, the CSP-provided infrastructure in the first region may be used by the CSP to provide CSP-supplied and CSP-branded cloud services to one or more of the CSP's direct (non-reseller) customers. In this way, the same CSP-provided infrastructure within a region is used to provide CSP-supplied and CSP-branded cloud services to one or more of the CSP's direct (non-reseller) customers, to provide customized first reseller-branded financial cloud services to the first reseller's customers, and to provide customized second reseller-branded telecommunications cloud services to the second reseller's customers.

[0037] Identity and Access Management (IAM) is an essential part of cloud services. As cloud computing becomes more prevalent, customers and their users can access and share data from anywhere in the "cloud." However, IAM plays a critical role in ensuring the right people have access to the data they need and have the appropriate security clearances.

[0038] As demand for cloud services grows, CSPs are exploring ways to expand their cloud service presence, including by having resellers provide more specialized services. However, increased demand for cloud services can lead to increased complexity in identity and access management. For example, cloud service provisioning may become two-tiered (e.g., providing services to the CSP's resellers and also the resellers' customers) instead of one-tiered (e.g., providing services only to the CSP's non-reseller customers). As a result, the number of users accessing the services increases exponentially. Therefore, as the level and number of users increase, challenges such as scalability, manageability, complexity, and cost issues must be addressed.

[0039] Identity and Access Management (IAM) may be referred to herein as identity management, which may include identity cloud services (IDCS). Identity management applies to various platforms, while identity cloud services are based on cloud platforms. Identity management may perform identity management functions (also referred to herein as identity checks) such as authentication, authorization, user management, etc.

[0040] The present disclosure generally relates to techniques for providing cloud services. More specifically, a novel technique is disclosed for enabling identity cloud services for virtual private label clouds (vPLCs). A vPLC is created for a cloud service provider (CSP) reseller using a CSP-provided infrastructure within a region so that the CSP reseller can offer one or more reseller-supplied cloud services to the reseller's customers.

[0041] The novel techniques described herein enable identity cloud services with flexible identity management configurations that use either a shared identity cloud service (IDCS) stack model or an independent IDCS stack model (i.e., an IDCS stack for each vPLC) based on the reseller's preference. The techniques can provide both cost savings and scalability. In other words, the techniques described in this disclosure can use vPLC IDs to determine which resellers and their customers are within a particular namespace inside an aggregated IDCS, rather than having one identity stack for each vPLC. As a result, the techniques can reduce the overhead associated with running multiple instances in exchange for manageability and scalability.

[0042] Additionally, the techniques described in this disclosure address complexity and manageability issues by enabling two-tier vPLC-aware identity management functions for a CSP's resellers and the reseller's customers by organizing and storing identity information according to multi-tier tenancy identification and performing identity management functions based on vPLC ID and customer tenancy ID associated with the user. Thus, an architecture involving the techniques of this disclosure is scalable and improves manageability.

[0043] The techniques described in this disclosure can configure identity management for different vPLCs associated with different resellers to perform identity management functions using a shared Identity Cloud Service (IDCS) stack or a dedicated IDCS stack for each vPLC.

[0044] In a shared IDCS stack model, a shared IDCS stack can perform identity management functions for a CSP's non-reseller customers and resellers associated with different vPLCs. The shared IDCS stack performs identity management functions for each vPLC using vPLC-specific identity information stored in a namespace associated with that vPLC.

[0045] An existing shared IDCS stack, which provides identity management capabilities for the CSP's non-reseller customers, may create a namespace that stores identity information for resellers associated with that vPLC.

[0046] In the independent IDCS stack model, each dedicated IDCS stack for a vPLC (also referred to herein as a vPLC IDCS stack) has its set of resources (e.g., compute nodes, memory, databases, etc.) that can operate independently and perform identity management functions for each vPLC using vPLC-specific identity information. In one embodiment, the vPLC IDCS stack for a vPLC may be a clone of an existing IDCS stack for a non-reseller customer of the CSP.

[0047] In an independent IDCS stack model, a vPLC IDCS stack may create a namespace that stores identity information for resellers associated with that vPLC. In an alternative embodiment, an existing IDCS stack may create a namespace that stores identity information for resellers associated with a vPLC before cloning it to become the vPLC IDCS stack for that vPLC to perform identity management functions.

[0048] In one embodiment, the identity information of a reseller associated with a vPLC may include identity information of the reseller and its associated users, as well as identity information of the reseller's customers and their associated users. This multi-level identity information may be organized and stored in a distributed manner (also known as a per-vPLC manner) or a centralized manner (also known as a vPLC ID as a partitioned manner).

[0049] The identity management functionality is vPLC-aware, in other words, it is implemented for each group of users, either the reseller's users or the reseller's customers' users, that are associated with a vPLC by identifying the vPLC ID and customer tenancy ID to access the appropriate level of identity information.

[0050] Figures 1-5 and the associated description provided in the "Exemplary Virtual Networking Architecture" section below describe networking concepts, including network virtualization, foundation networks, overlay networks, VNICs, etc., and provide example environments in which certain embodiments described in this disclosure may be implemented. Figures 6-29 describe examples and embodiments related to vPLC architectures that configure identity and access management for vPLC and perform identity management functions for users, as described in this disclosure. Figures 30-33 depict example architectures for implementing a cloud infrastructure for providing one or more cloud services, which infrastructure may incorporate teachings described herein. Figure 34 depicts a block diagram illustrating an exemplary computer system or device, according to at least one embodiment.

[0051] Exemplary Virtual Networking Architecture The term cloud services is generally used to refer to services made available by a cloud service provider (CSP) to users or customers on demand (e.g., via a subscription model) using systems and infrastructure (cloud infrastructure) provided by the CSP. Typically, the servers and systems that make up the CSP's infrastructure are separate from the customer's own on-premises servers and systems. Thus, customers can use cloud services provided by the CSP without having to purchase separate hardware and software resources for the service. Cloud services are designed to provide subscribing customers with easy and scalable access to applications and computing resources without the customer having to invest in procuring the infrastructure used to provide the service.

[0052] There are several cloud service providers that offer different types of cloud services. There are various different types or models of cloud services, including Software as a Service (SaaS), Platform as a Service (PaaS), Infrastructure as a Service (IaaS), etc.

[0053] A customer can subscribe to one or more cloud services offered by a CSP. A customer can be any entity, such as an individual, an organization, a business, etc. When a customer subscribes or registers for a service offered by a CSP, a tenancy or account is created for the customer. The customer can then access one or more subscribed cloud resources associated with the account through this account.

[0054] As mentioned above, Infrastructure as a Service (IaaS) is one specific type of cloud computing service. In the IaaS model, a CSP provides infrastructure (referred to as Cloud Service Provider Infrastructure or CSPI) that customers can use to build their own customizable networks and deploy customer resources. Therefore, customer resources and networks are hosted in a distributed environment by the infrastructure provided by the CSP. This differs from traditional computing, where customer resources and networks are hosted by the infrastructure provided by the customer.

[0055] CSPI can include interconnected high-performance computing resources, including various host machines, memory resources, and network resources, forming a physical network, also referred to as an infrastructure network or underlay network. Resources within a CSPI can be distributed across one or more data centers, which can be geographically distributed across one or more geographic regions. These physical resources can run virtualization software to provide a virtualized distributed environment. Virtualization creates an overlay network (also known as a software-based network, software-defined network, or virtual network) on top of the physical network. The CSPI physical network provides the underlying infrastructure for creating one or more overlay or virtual networks on top of the physical network. The physical network (or infrastructure network or underlay network) includes physical network devices such as physical switches, routers, computers, and host machines. An overlay network is a logical (or virtual) network that operates on top of the physical infrastructure network. A given physical network can support one or more overlay networks. Overlay networks typically use encapsulation techniques to distinguish traffic belonging to different overlay networks. A virtual or overlay network is also referred to as a virtual cloud network (VCN). Virtual networks are implemented using software virtualization technologies (e.g., hypervisors, virtualization functions implemented by network virtualization devices (NVDs) (e.g., smartNICs), top-of-rack (TOR) switches, smart TORs that implement one or more functions performed by NVDs, and other mechanisms) to create a layer of network abstraction that can operate on top of a physical network. Virtual networks can take many forms, including peer-to-peer networks, IP networks, etc. Virtual networks are typically either layer-3 IP networks or layer-2 VLANs.This virtual or overlay networking method is often referred to as virtual or overlay Layer 3 networking. Examples of protocols deployed for virtual networks include IP-in-IP (or Generic Routing Encapsulation (GRE)), Virtual Extensible LAN (VXLAN - IETF RFC7348), Virtual Private Networks (VANs) (e.g., MPLS Layer 3 Virtual Private Networks (RFC4364)), VMware's NSX, GENEVE (Generic Network Virtualization Encapsulation), etc.

[0056] In IaaS, the infrastructure (CSPI) provided by the CSP can be configured to provide virtualized computing resources over a public network (e.g., the Internet). In the IaaS model, a cloud computing service provider can host infrastructure components (e.g., servers, storage devices, network nodes (e.g., hardware), deployment software, platform virtualization (e.g., hypervisor layer), etc.). In some cases, the IaaS provider can also provide various services (e.g., billing, monitoring, logging, security, load balancing, clustering, etc.) to accompany those infrastructure components. Accordingly, these services can be policy-driven, allowing IaaS users to implement policies to drive load balancing to maintain application availability and performance. The CSPI provides infrastructure and a set of complementary cloud services that enable customers to build and run diverse applications and services in a highly available, hosted, distributed environment. The CSPI delivers high-performance compute resources and capabilities, as well as storage capacity, within a flexible virtual network that is securely accessible from various networked locations, such as the customer's on-premises network. When a customer subscribes or registers for an IaaS service offered by a CSP, the tenancy created for that customer is a secure, isolated division within the CSP where the customer can create, organize, and manage their cloud resources.

[0057] Customers can build their own virtual networks using the compute, memory, and networking resources provided by CSPI. They can deploy one or more customer resources or workloads, such as compute instances, on these virtual networks. For example, customers can build one or more customizable private virtual networks, referred to as virtual cloud networks (VCNs), using resources provided by CSPI. Customers can deploy one or more customer resources, such as compute instances, on the customer VCN. Compute instances can take the form of virtual machines, bare metal instances, and so on. CSPI therefore provides an infrastructure and a set of complementary cloud services that enable customers to build and run diverse applications and services within a highly available, virtual host environment. While customers do not manage or control the underlying physical resources provided by CSPI, they can control the operating system, storage, and deployed applications, and in some cases, have limited control over the selection of networking components (e.g., firewalls).

[0058] The CSP can provide a console that enables customers and network administrators to configure, access, and manage resources deployed in the cloud using CSPI resources. In some embodiments, the console provides a web-based user interface that can be used to access and manage the CSPI. In some embodiments, the console is a web-based application provided by the CSP.

[0059] CSPI can support single-tenancy or multi-tenancy architectures. In a single-tenancy architecture, software (e.g., applications, databases) or hardware components (e.g., host machines or servers) serve a single customer or tenant. In a multi-tenancy architecture, software or hardware components serve multiple customers or tenants. Thus, in a multi-tenancy architecture, CSPI resources are shared among multiple customers or tenants. In a multi-tenancy situation, precautions are taken and safeguards are implemented within CSPI to ensure that each tenant's data remains isolated and invisible to other tenants.

[0060] In a physical network, a network endpoint (“endpoint”) refers to a computing device or system that is connected to the physical network and communicates to and from the connected network. Network endpoints in a physical network may be connected to a local area network (LAN), a wide area network (WAN), or other types of physical networks. Examples of traditional endpoints in a physical network include modems, hubs, bridges, switches, routers, and other networking devices, physical computers (or host machines), etc. Each physical device in a physical network has a fixed network address that can be used to communicate with the device. This fixed network address can be a Layer 2 address (e.g., a MAC address), a fixed Layer 3 address (e.g., an IP address), etc. In a virtualized environment or virtual network, endpoints can include various virtual endpoints, such as virtual machines hosted by components of the physical network (e.g., hosted by a physical host machine). These endpoints in the virtual network are addressed by overlay addresses, such as overlay Layer 2 addresses (e.g., an overlay MAC address) and overlay Layer 3 addresses (e.g., an overlay IP address). Network overlays enable flexibility by allowing network managers to move overlay addresses associated with network endpoints around using software management (e.g., via software implementing the virtual network's control plane). Thus, unlike physical networks, in virtual networks, overlay addresses (e.g., overlay IP addresses) can be moved from one endpoint to another using network management software. Because virtual networks are built on top of physical networks, communication between components in a virtual network involves both the virtual network and the underlying physical network.To facilitate such communications, CSPI components are configured to learn and store mappings that map overlay addresses in the virtual network to actual physical addresses in the underlying network, and vice versa. These mappings are then used to facilitate communications. Customer traffic is encapsulated to facilitate routing within the virtual network.

[0061] Thus, a physical address (e.g., a physical IP address) is associated with an element in a physical network, and an overlay address (e.g., an overlay IP address) is associated with an entity in a virtual or overlay network. A physical IP address is an IP address associated with a physical device (e.g., a network device) in an underlying or physical network. For example, each NVD has an associated physical IP address. An overlay IP address is an overlay address associated with an entity in an overlay network, such as a compute instance in a customer's virtual cloud network (VCN). Two different customers or tenants, each with their own private VCN, can potentially use the same overlay IP address in that VCN without knowing anything about each other. Both physical IP addresses and overlay IP addresses are types of real IP addresses. They are separate from virtual IP addresses. A virtual IP address is typically a single IP address that represents or maps to multiple real IP addresses. A virtual IP address provides a one-to-many mapping between a virtual IP address and multiple real IP addresses. For example, a load balancer may use a VIP to map to or represent multiple servers, each with its own real IP address.

[0062] A cloud infrastructure or CSPI is physically hosted in one or more data centers within one or more regions around the world. The CSPI can include components within a physical or underlying network and virtualized components (e.g., virtual networks, compute instances, virtual machines, etc.) within a virtual network built on top of the physical network components. In one embodiment, the CSPI is organized and hosted in realms, regions, and availability domains. A region is typically a local geographic area that includes one or more data centers. Regions are generally independent of each other and can be separated by large distances, for example, across countries or even continents. For example, a first region may be in Australia, another region may be in Japan, yet another region may be in India, etc. CSPI resources are divided among multiple regions, so that each region has its own independent subset of CSPI resources. Each region can provide a set of essential infrastructure services and resources, such as compute resources (e.g., bare metal servers, virtual machines, containers and related infrastructure, etc.), storage resources (e.g., block volume storage, file storage, object storage, archive storage), networking resources (e.g., virtual cloud networks (VCNs), load balancing resources, connectivity to on-premises networks), database resources, edge networking resources (e.g., DNS), and access management and monitoring resources, etc. Each region typically has multiple paths connecting it to other regions within the realm.

[0063] Because using nearby resources is faster than using distant resources, applications are typically deployed in the region where they are most heavily used (i.e., on the infrastructure associated with that region). Applications may also be deployed in different regions for a variety of reasons, such as redundancy to mitigate the risk of regional events such as large-scale weather systems or earthquakes, to meet the varying requirements of jurisdictions, tax areas, and other business or societal standards, etc.

[0064] Data centers within a region can also be further organized and subdivided into availability domains (ADs). An availability domain can correspond to one or more data centers located within a region. A region can consist of one or more availability domains. In such a distributed environment, CSPI resources are either region-specific, such as a virtual cloud network (VCN), or availability domain-specific, such as a compute instance.

[0065] ADs within a region are isolated from each other, fault-tolerant, and configured to be highly unlikely to fail simultaneously. This is achieved by ADs not sharing critical infrastructure resources such as networking, physical cables, cable routes, cable entry points, etc., so that a failure in one AD within a region is unlikely to affect the availability of other ADs in the same region. ADs within the same region can be connected to each other by low-latency, high-bandwidth networks, which provides highly available connectivity to other networks (e.g., the Internet, customer on-premises networks, etc.) and makes it possible to build replicated systems within multiple ADs for both high availability and disaster recovery. Cloud services use multiple ADs to ensure high availability and protect against resource failures. As the infrastructure provided by IaaS grows, more regions and ADs are added to add capacity. Traffic between availability domains is typically encrypted.

[0066] In one embodiment, regions are grouped into realms. A realm is a logical collection of multiple regions. Realms are isolated from each other and do not share any data. Regions within the same realm can communicate with each other, but regions in different realms cannot communicate with each other. A customer's tenancy and account with a CSP exist within a single realm and can be distributed across one or more regions belonging to that realm. Typically, when a customer subscribes to an IaaS service, their tenancy and account are created in a customer-specified region within the realm (referred to as the "home" region). The customer can extend their tenancy across one or more other regions within the realm. A customer cannot access regions that are not within the realm in which their tenancy resides.

[0067] An IaaS provider may offer multiple realms, each serving a particular set of customers or users. For example, a commercial realm may be offered to commercial customers. As another example, a realm for a particular country may be offered to customers within that country. As yet another example, a government realm may be offered to a government, and so on. For example, a government realm may be offered to a particular government and may have a higher level of security than a commercial realm. For example, Oracle Cloud Infrastructure (OCI) currently offers one realm in a commercial region and two realms in a government cloud region (e.g., FedRAMP-authorized and IL5-authorized).

[0068] In one embodiment, an AD can be subdivided into one or more fault domains. A fault domain is a grouping of infrastructure resources within an AD to provide anti-affinity. Fault domains allow for the distribution of compute instances so that instances are not on the same physical hardware within a single AD. This is known as anti-affinity. A fault domain refers to a set of hardware components (computers, switches, etc.) that share a single point of failure. A compute pool is logically divided into multiple fault domains. Due to this, a hardware failure or compute hardware maintenance event affecting one fault domain does not affect instances in other fault domains. Depending on the embodiment, the number of fault domains in each AD may vary. For example, in one embodiment, each AD includes three fault domains. Fault domains act as logical data centers within an AD.

[0069] When a customer subscribes to an IaaS service, resources from CSPI are provisioned to the customer and associated with the customer's tenancy. The customer can use these provisioned resources to build private networks and deploy resources on these networks. A customer network hosted in the cloud by CSPI is referred to as a virtual cloud network (VCN). A customer can set up one or more virtual cloud networks (VCNs) using the CSPI resources allocated to the customer. A VCN is a virtual or software-defined private network. Customer resources deployed within a customer's VCN can include compute instances (e.g., virtual machines, bare metal instances) and other resources. These compute instances can represent various customer workloads such as applications, load balancers, databases, etc. Compute instances deployed on a VCN can communicate with publicly accessible endpoints ("public endpoints") on public networks such as the Internet, other instances in the same VCN or other VCNs (e.g., other VCNs of the customer or VCNs not belonging to the customer), the customer's on-premises data center or network, and service endpoints and other types of endpoints.

[0070] CSPs can offer various services using CSPI. In some cases, customers of a CSPI themselves may act as service providers and offer services using CSPI resources. Service providers can expose service endpoints characterized by identifying information (e.g., IP addresses, DNS names, and ports). Customer resources (e.g., compute instances) can consume a particular service by accessing the service endpoint exposed by the service for that particular service. These service endpoints are generally publicly accessible by users via a public communications network, such as the Internet, using a public IP address associated with the endpoint. Publicly accessible network endpoints are sometimes referred to as public endpoints.

[0071] In some embodiments, a service provider may expose a service through an endpoint for that service (sometimes referred to as a service endpoint). Customers of the service may then access the service using this service endpoint. In some embodiments, a service endpoint provided for a service may be accessed by multiple customers who intend to consume the service. In other embodiments, a dedicated service endpoint may be provided for a customer, so that only that customer can access the service using that dedicated service endpoint.

[0072] In one embodiment, when a VCN is created, it is associated with a private overlay Classless Inter-Domain Routing (CIDR) address space, which is a range of private overlay IP addresses (e.g., 10.0 / 16) that are assigned to the VCN. A VCN includes associated subnets, routing tables, and gateways. A VCN exists within a single region but can span one, more, or all of the region's availability domains. Gateways are virtual interfaces configured for a VCN that enable traffic to and from the VCN to one or more endpoints outside the VCN. One or more different types of gateways may be configured for a VCN to enable communication to and from different types of endpoints.

[0073] A VCN can be subdivided into one or more subnetworks, such as one or more subnets. A subnet is thus a building block or subdivision that can be created within a VCN. A VCN can have one or more subnets. Each subnet in a VCN is associated with a contiguous range of overlay IP addresses (e.g., 10.0.0.0 / 24 and 10.0.1.0 / 24) that do not overlap with other subnets in that VCN and represent a subset of address space within the VCN's address space.

[0074] Each compute instance is associated with a virtual network interface card (VNIC) that enables the compute instance to participate in a subnet of a VCN. A VNIC is a logical representation of a physical network interface card (NIC). Generally, a VNIC is an interface between an entity (e.g., a compute instance, a service) and a virtual network. A VNIC exists within a subnet and has one or more associated IP addresses and associated security rules or policies. A VNIC is equivalent to a Layer 2 port on a switch. A VNIC is attached to a compute instance and a subnet within a VCN. A VNIC associated with a compute instance enables the compute instance to be part of a subnet of a VCN and enables the compute instance to communicate (e.g., send and receive packets) with endpoints on the same subnet as the compute instance, endpoints in a different subnet within the VCN, or endpoints outside the VCN. Thus, a VNIC associated with a compute instance determines how the compute instance connects with endpoints inside and outside the VCN. When a compute instance is created and added to a subnet within a VCN, a VNIC for the compute instance is created and associated with the compute instance. For a subnet that includes a set of compute instances, the subnet includes VNICs that correspond to the set of compute instances, with each VNIC attached to a compute instance in the set of compute instances.

[0075] Each compute instance is assigned a private overlay IP address via the VNIC associated with the compute instance. This private overlay IP address is assigned to the VNIC associated with the compute instance when the compute instance is created and is used to route traffic to and from the compute instance. All VNICs within a given subnet use the same routing table, security lists, and DHCP options. As described above, each subnet within a VCN is associated with a contiguous range of overlay IP addresses (e.g., 10.0.0.0 / 24 and 10.0.1.0 / 24) that do not overlap with other subnets within that VCN and represent a subset of address space within the VCN's address space. For a VNIC on a particular subnet of a VCN, the private overlay IP address assigned to the VNIC is from the contiguous range of overlay IP addresses allocated to the subnet.

[0076] In some embodiments, a compute instance may optionally be assigned additional overlay IP addresses in addition to its private overlay IP address, such as one or more public IP addresses if it is in a public subnet. These multiple addresses are assigned on the same VNIC or across multiple VNICs associated with the compute instance. However, each instance has a primary VNIC created during instance launch and associated with the overlay private IP address assigned to the instance, and this primary VNIC cannot be deleted. Additional VNICs, referred to as secondary VNICs, can be added to an existing instance in the same availability domain as the primary VNIC. All VNICs are in the same availability domain as the instance. The secondary VNIC can be in a subnet in the same VCN as the primary VNIC or in a different subnet, either in the same VCN or a different VCN.

[0077] Compute instances may optionally be assigned public IP addresses if they are in a public subnet. Subnets can be designed as either public or private subnets when the subnet is created. A private subnet means that resources (e.g., compute instances) and associated VNICs within the subnet cannot have public overlay IP addresses. A public subnet means that resources and associated VNICs within the subnet can have public IP addresses. Customers can specify a subnet to exist within a single availability domain or across multiple availability domains within a region or realm.

[0078] As described above, a VCN can be subdivided into one or more subnets. In one embodiment, a virtual router (VR) configured for a VCN (referred to as a VCN VR or simply VR) enables communication between subnets in the VCN. For a subnet in a VCN, the VR represents the logical gateway for that subnet, allowing the subnet (i.e., the compute instances on that subnet) to communicate with endpoints on other subnets in the VCN and with other endpoints outside the VCN. A VCN VR is a logical entity configured to route traffic between VNICs in a VCN and a virtual gateway ("gateway") associated with the VCN. Gateways are further described below with respect to FIG. 1. A VCN VR is a Layer 3 / IP layer concept. In one embodiment, there is one VCN VR for a VCN, and the VCN VR has a potentially infinite number of ports addressed by IP addresses, one port for each subnet in the VCN. In this way, the VCN VR has a different IP address for each subnet in the VCN to which the VCN VR is attached. The VRs are also connected to various gateways configured for the VCN. In some embodiments, a specific overlay IP address from a subnet's overlay IP address range is reserved for ports in that subnet's VCN VR. For example, consider a VCN with two subnets, each with associated address ranges 10.0 / 16 and 10.1 / 16. For the first subnet in the VCN with address range 10.0 / 16, an address from this range is reserved for ports in that subnet's VCN VR. In some cases, the first IP address from that range may be reserved for a VCN VR. For example, for a subnet with overlay IP address range 10.0 / 16, IP address 10.0.0.1 may be reserved for ports in that subnet's VCN VR.For a second subnet in the same VCN with address range 10.1 / 16, the VCN VR may have a port in that second subnet with IP address 10.1.0.1. The VCN VR has a different IP address for each of the subnets in the VCN.

[0079] In some other embodiments, each subnet within a VCN can have its own associated VR that is addressable by the subnet using a reserved or default IP address associated with the VR. The reserved or default IP address may, for example, be the first IP address from a range of IP addresses associated with the subnet. VNICs within a subnet can use this default or reserved IP address to communicate (e.g., send and receive packets) with the VR associated with the subnet. In such embodiments, a VR is the ingress / egress point for that subnet. A VR associated with a subnet within a VCN can communicate with other VRs associated with other subnets in the VCN. The VR can also communicate with a gateway associated with the VCN. The VR functions of a subnet are running on or performed by one or more NVDs that are performing the VNIC functions of VNICs within the subnet.

[0080] Routing tables, security rules, and DHCP options can be configured for a VCN. A routing table is a virtual routing table for a VCN that contains rules for routing traffic from subnets within the VCN to destinations outside the VCN by gateways or specially configured instances. A VCN's routing table can be customized to control how packets are forwarded / routed into and out of the VCN. DHCP options are configuration information that is automatically provided to instances when they launch.

[0081] Security rules configured for a VCN represent the overlay firewall rules for the VCN. Security rules can include ingress and egress rules and can specify the type of traffic (based on protocol and port) that is allowed in and out of instances within the VCN. Customers can choose whether a given rule is stateful or stateless. For example, a customer can allow SSH traffic from anywhere to enter a set of instances by setting up a stateful ingress rule with source CIDR 0.0.0.0 / 0 and destination TCP port 22. Security rules can be implemented using network security groups or security lists. A network security group consists of a set of security rules that apply only to resources within that group. A security list, on the other hand, contains rules that apply to all resources within any subnet that uses the security list. A VCN can be provided with a default security list with default security rules. DHCP options configured for a VCN provide configuration information that is automatically provided to instances within the VCN when the instances launch.

[0082] In one embodiment, configuration information for a VCN is determined and stored by a VCN control plane. The VCN configuration information may include, for example, information about address ranges associated with the VCN, subnets and associated information within the VCN, one or more VRs associated with the VCN, compute instances and associated VNICs within the VCN, NVDs performing various virtualized network functions (e.g., VNICs, VRs, gateways) associated with the VCN, VCN state information, and other VCN-related information. In one embodiment, a VCN distribution service publishes the configuration information stored by the VCN control plane, or portions thereof, to the NVD. The distributed information can be used to update information (e.g., forwarding tables, routing tables, etc.) stored and used by the NVD to forward packets to and from compute instances within the VCN.

[0083] In one embodiment, VCN and subnet creation is handled by a VCN control plane (CP), and compute instance launching is handled by the compute control plane. The compute control plane is responsible for allocating physical resources for the compute instances and then calls the VCN control plane to create and attach VNICs to the compute instances. The VCN CP also sends VCN data mappings to the VCN data plane, which is configured to perform packet forwarding and routing functions. In one embodiment, the VCN CP provides a distribution service that is responsible for providing updates to the VCN data plane. Examples of VCN control planes are also shown in Figures 1, 2, 3, and 4 (see reference numbers 116, 216, 316, and 416) and described below.

[0084] Customers can create one or more VCNs using resources hosted by CSPI. Compute instances deployed on a customer VCN can communicate with multiple different endpoints. These endpoints can include endpoints hosted by CSPI and endpoints external to CSPI.

[0085] Various different architectures for implementing cloud-based services using CSPI are shown in Figures 1, 2, 3, 4, and 5 and described below. Figure 1 is a high-level diagram of a distributed environment 100 illustrating an overlay or customer VCN hosted by CSPI according to one embodiment. The distributed environment shown in Figure 1 includes multiple components within an overlay network. The distributed environment 100 shown in Figure 1 is illustrative only and is not intended to unduly limit the scope of the claimed embodiments. Many variations, alternatives, and modifications are possible. For example, in some other embodiments, the distributed environment shown in Figure 1 may have more or fewer systems or components than those shown in Figure 1, may combine two or more systems, or may have a different configuration or arrangement of systems.

[0086] As shown in the example depicted in FIG. 1 , a distributed environment 100 includes a CSPI 101 that provides services and resources that customers can subscribe to and use to build their virtual cloud networks (VCNs). In one embodiment, CSPI 101 provides IaaS services to subscribing customers. Data centers within CSPI 101 can be organized into one or more regions. One exemplary region, “Region US” 102, is shown in FIG. 1 . A customer configures a customer VCN 104 in region 102. A customer can deploy various compute instances on VCN 104, where the compute instances may include virtual machines or bare metal instances. Example instances include applications, databases, load balancers, etc.

[0087] In the embodiment shown in FIG. 1 , customer VCN 104 includes two subnets, "Subnet 1" and "Subnet 2," each with its own CIDR IP address range. In FIG. 1 , Subnet 1's overlay IP address range is 10.0 / 16, and Subnet 2's address range is 10.1 / 16. VCN virtual router 105 represents the VCN's logical gateway, enabling communication between subnets in VCN 104 and with other endpoints outside the VCN. VCN VR 105 is configured to route traffic between VNICs in VCN 104 and the gateway associated with VCN 104. VCN VR 105 provides a port for each subnet in VCN 104. For example, VR 105 may provide a port with IP address 10.0.0.1 for Subnet 1 and a port with IP address 10.1.0.1 for Subnet 2.

[0088] Multiple compute instances may be deployed on each subnet, where the compute instances may be virtual machine instances and / or bare metal instances. The compute instances within a subnet may be hosted by one or more host machines within CSPI 101. A compute instance joins a subnet via a VNIC associated with the compute instance. For example, as shown in FIG. 1, compute instance C1 is part of subnet 1 via a VNIC associated with the compute instance. Similarly, compute instance C2 is part of subnet 1 via a VNIC associated with C2. Similarly, multiple compute instances, which may be virtual machine instances or bare metal instances, may be part of subnet 1. Through its associated VNIC, each compute instance is assigned a private overlay IP address and MAC address. For example, in FIG. 1, compute instance C1 has an overlay IP address of 10.0.0.2 and a MAC address of M1, while compute instance C2 has a private overlay IP address of 10.0.0.3 and a MAC address of M2. Each compute instance in Subnet 1, including compute instances C1 and C2, has a default routing to VCN VR 105 using IP address 10.0.0.1, which is the IP address of a port in VCN VR 105 in Subnet 1.

[0089] Subnet2 may have multiple compute instances deployed thereon, including virtual machine instances and / or bare metal instances. For example, as shown in FIG. 1, compute instances D1 and D2 are part of Subnet2 via VNICs associated with the respective compute instances. In the embodiment shown in FIG. 1, compute instance D1 has an overlay IP address of 10.1.0.2 and a MAC address of MM1, while compute instance D2 has a private overlay IP address of 10.1.0.3 and a MAC address of MM2. Each compute instance in Subnet2, including compute instances D1 and D2, has default routing to VCN VR 105 using IP address 10.1.0.1, which is the IP address of a port in VCN VR 105 in Subnet2.

[0090] VCN A 104 may also include one or more load balancers. For example, a load balancer may be provided for a subnet and configured to load balance traffic across multiple compute instances on the subnet. A load balancer may also be provided to load balance traffic across subnets within the VCN.

[0091] A particular compute instance deployed on VCN 104 can communicate with a variety of different endpoints. These endpoints may include endpoints hosted by CSPI 200 and endpoints outside of CSPI 200. Endpoints hosted by CSPI 101 may include endpoints on the same subnet as the particular compute instance (e.g., communication between two compute instances in Subnet 1), endpoints on a different subnet but within the same VCN (e.g., communication between a compute instance in Subnet 1 and a compute instance in Subnet 2), endpoints in a different VCN within the same region (e.g., communication between a compute instance in Subnet 1 and an endpoint in a VCN in the same region 106 or 110, communication between a compute instance in Subnet 1 and an endpoint in a service network 110 in the same region), or endpoints in a VCN in a different region (e.g., communication between a compute instance in Subnet 1 and an endpoint in a VCN in a different region 108). Compute instances in a subnet hosted by CSPI 101 may also communicate with endpoints not hosted by CSPI 101 (i.e., outside of CSPI 101). These external endpoints include endpoints within the customer's on-premise network 116, endpoints within other remote cloud host networks 118, public endpoints 114 accessible via public networks such as the Internet, and other endpoints.

[0092] Communication between compute instances on the same subnet is facilitated using VNICs associated with the source and destination compute instances. For example, compute instance C1 in Subnet 1 may desire to send a packet to compute instance C2 in Subnet 1. For a packet originating from a source compute instance and destined for another compute instance in the same subnet, the packet is first processed by the VNIC associated with the source compute instance. The processing performed by the VNIC associated with the source compute instance may include determining the packet's destination information from the packet header, identifying any policies (e.g., security lists) configured for the VNIC associated with the source compute instance, determining the packet's next hop, performing any encapsulation / decapsulation as needed, and then forwarding / routing the packet to the next hop with the goal of facilitating communication of the packet to its intended destination. When the destination compute instance is in the same subnet as the source compute instance, the VNIC associated with the source compute instance is configured to identify the VNIC associated with the destination compute instance and forward the packet to that VNIC for processing. At this time, the VNIC associated with the destination compute instance is executed and forwards the packet to the destination compute instance.

[0093] For packets to be communicated from a compute instance in a subnet to an endpoint in a different subnet within the same VCN, communication is facilitated by VNICs and VCN VRs associated with the source and destination compute instances. For example, if compute instance C1 in Subnet 1 of FIG. 1 wants to send a packet to compute instance D1 in Subnet 2, the packet is first processed by the VNIC associated with compute instance C1. The VNIC associated with compute instance C1 is configured to route the packet to VCN VR 105 using the VCN VR's default routing or port 10.0.0.1. VCN VR 105 is configured to route the packet to Subnet 2 using port 10.1.0.1. The packet is then received and processed by the VNIC associated with D1, which forwards the packet to compute instance D1.

[0094] For packets to be communicated from a compute instance within VCN 104 to an endpoint outside VCN 104, the communication is facilitated by a VNIC associated with the source compute instance, VCN VR 105, and a gateway associated with VCN 104. One or more types of gateways may be associated with VCN 104. A gateway is an interface between a VCN and another endpoint, where the other endpoint is external to the VCN. A gateway is a Layer 3 / IP layer concept that allows a VCN to communicate with endpoints outside the VCN. Thus, a gateway facilitates traffic flow between a VCN and other VCNs or networks. Various different types of gateways may be configured for a VCN to facilitate different types of communication with different types of endpoints. Depending on the gateway, the communication may be over a public network (e.g., the Internet) or a private network. Various communication protocols may be used for these communications.

[0095] For example, compute instance C1 may desire to communicate with an endpoint outside VCN 104. The packet may first be processed by a VNIC associated with source compute instance C1. The VNIC processing determines that the packet's destination is outside Subnet 1 of C1. The VNIC associated with C1 may forward the packet to VCN VR 105 of VCN 104. VCN VR 105 then processes the packet and, as part of the processing, determines a particular gateway associated with VCN 104 as the packet's next hop based on the packet's destination. VCN VR 105 may then forward the packet to the particular identified gateway. For example, if the destination is an endpoint within a customer's on-premises network, the packet may be forwarded by VCN VR 105 to a dynamic routing gateway (DRG) gateway 122 configured for VCN 104. The packet may then be forwarded from the gateway to the next hop to facilitate communication of the packet to its final intended destination.

[0096] Various different types of gateways may be configured for a VCN. Examples of gateways that may be configured for a VCN are shown in FIG. 1 and described below. Examples of gateways associated with VCNs are also shown in FIGS. 1, 2, 3, and 4 (e.g., gateways referenced by reference numbers 134, 136, 138, 234, 236, 238, 334, 336, 338, 434, 436, and 438) and described below. As shown in the embodiment shown in FIG. 1, a dynamic routing gateway (DRG) 122 can be added to or associated with a customer VCN 104 to provide a path for private network traffic communication between the customer VCN 104 and another endpoint, where the other endpoint can be the customer's on-premises network 116, a VCN 108 in a different region of CSPI 101, or another remote cloud network 118 not hosted by CSPI 101. The customer on-premises network 116 may be a customer network or customer data center built using the customer's resources. Access to the customer on-premises network 116 is generally very restricted. For a customer that has both a customer on-premises network 116 and one or more VCNs 104 deployed or hosted in the cloud by CSPI 101, the customer may want their on-premises network 116 and their cloud-based VCNs 104 to be able to communicate with each other. This allows the customer to build an extended hybrid environment that encompasses the customer's VCN 104 hosted by CSPI 101 and their on-premises network 116. The DRG 122 enables this communication. To enable such communication, a communication channel 124 is set up, where one endpoint of the channel resides in the customer on-premises network 116 and the other endpoint resides in CSPI 101 and is connected to the customer VCN 104. The communication channel 124 may be over a public or private communication network, such as the Internet.A variety of different communication protocols may be used, such as IPsec VPN technology over a public communication network such as the Internet, or Oracle's FastConnect technology, which uses a private network instead of a public network. A device or equipment in the customer on-premises network 116 that forms one endpoint of the communication channel 124 is referred to as customer premises equipment (CPE), such as CPE 126 shown in Figure 1. On the CSPI 101 side, the endpoint may be a host machine running DRG 122.

[0097] In some embodiments, remote peering connections (RPCs) can be added to a DRG, allowing customers to peer one VCN with another VCN in a different region. Using such RPCs, a customer VCN 104 can connect with a VCN 108 in another region using the DRG 122. The DRG 122 can also be used to communicate with other remote cloud networks 118 not hosted by CSPI 101, such as the Microsoft Azure cloud, the Amazon AWS cloud, etc.

[0098] 1, an internet gateway (IGW) 120 can be configured for a customer VCN 104 that enables compute instances on VCN 104 to communicate with public endpoints 114 accessible over a public network, such as the Internet. The IGW 120 is a gateway that connects a VCN to a public network, such as the Internet. The IGW 120 enables public subnets in a VCN, such as VCN 104 (where resources in the public subnet have public overlay IP addresses) to directly access public endpoints 112 on the public network 114, such as the Internet. The IGW 120 can be used to initiate connections from subnets in VCN 104 or from the Internet.

[0099] A network address translation (NAT) gateway 128 can be configured for a customer's VCN 104 to enable cloud resources in the customer's VCN that do not have dedicated public overlay IP addresses to access the Internet, without exposing those resources to a direct inbound Internet connection (e.g., an L4-L7 connection). This allows private subnets in the VCN, such as Private Subnet 1 in VCN 104, to have private access to public endpoints on the Internet. With a NAT gateway, connections can only be initiated from the private subnet to the public Internet, not from the Internet to the private subnet.

[0100] In one embodiment, a service gateway (SGW) 126 can be configured for customer VCN 104 and provides a pathway for private network traffic between VCN 104 and supported service endpoints in service network 110. In one embodiment, service network 110 can be provided by a CSP and can offer a variety of services. One example of such a service network is Oracle's service network, which offers a variety of services that can be used by customers. For example, a compute instance (e.g., a database system) in a private subnet of customer VCN 104 can back up data to a service endpoint (e.g., object storage) without requiring a public IP address or access to the Internet. In one embodiment, a VCN can have only one SGW, and connections can be initiated only from subnets within the VCN, not from service network 110. If a VCN is peered with another VCN, resources in the other VCN cannot access the SGW. Resources in an on-premises network connected to a VCN by FastConnect or VPN Connect can also use a service gateway configured for that VCN.

[0101] In one embodiment, SGW 126 uses the concept of a service classless inter-domain routing (CIDR) label, which is a string that represents all regional public IP address ranges for a service or group of services of interest. A customer uses the service CIDR label when configuring the SGW and associated routing rules to control traffic to the service. A customer can optionally utilize the service CIDR label when configuring security rules without having to adjust those security rules if the service's public IP addresses change in the future.

[0102] A local peering gateway (LPG) 132 is a gateway that can be added to a customer VCN 104, allowing the VCN 104 to peer with another VCN in the same region. Peering means that the VCNs communicate using private IP addresses without the traffic traversing a public network such as the Internet or routing the traffic through the customer's on-premises network 116. In a preferred embodiment, a VCN has a separate LPG for each peering it establishes. Local peering or VCN peering is a common practice used to establish network connectivity between different applications or infrastructure management functions.

[0103] A service provider, such as a provider of a service in service network 110, can provide access to the service using different access models. According to a public access model, the service may be exposed as a public endpoint publicly accessible by a compute instance in the customer VCN over a public network such as the Internet, and / or may be privately accessible through SGW 126. According to a specific private access model, the service is made accessible as a private IP endpoint in a private subnet in the customer's VCN. This is referred to as private endpoint (PE) access and allows service providers to expose their services as instances in the customer's private network. Private endpoint resources represent services in the customer's VCN. Each PE appears as a VNIC (referred to as a PE-VNIC with one or more private IPs) in a subnet chosen by the customer in the customer's VCN. Thus, the PE provides a way to present the service in the private customer VCN subnet using a VNIC. Because the endpoint is exposed as a VNIC, all features associated with the VNIC, such as routing rules, security lists, etc., are now made available to the PE VNIC.

[0104] Service providers can register their services to enable access through the PE. Providers can associate policies with services, which constrain the visibility of the service to a customer tenancy. Providers can register multiple services under a single virtual IP address (VIP), especially for multi-tenant services. There may be multiple such private endpoints (in multiple VCNs) representing the same service.

[0105] Compute instances in the private subnet can then access the service using the private IP address or service DNS name of the PE VNIC. Compute instances in the customer VCN can access the service by sending traffic to the private IP address of the PE in the customer VCN. A private access gateway (PAGW) 130 is a gateway resource that can be attached to a service provider VCN (e.g., a VCN in service network 110) that acts as the ingress / egress point for all traffic from / to customer subnet private endpoints. The PAGW 130 allows providers to scale the number of PE connections without utilizing their internal IP address resources. A provider only needs to configure one PAGW for any number of services registered in a single VCN. A provider can represent services as private endpoints in multiple VCNs for one or more customers. From the customer's perspective, the PE VNIC appears to be attached to the service with which the customer wants to interact, instead of being attached to the customer's instance. Traffic destined for the private endpoint is routed to the service through the PAGW 130. These are referred to as Customer-to-Service Private Connections (C2S Connections).

[0106] The PE concept can also be used to extend private access for services to a customer's on-premises network and data center by allowing traffic to flow through a FastConnect / IPsec link and a private endpoint within the customer VCN. Private access for services can also be extended to a customer's peered VCN by allowing traffic to flow between LPG 132 and a PE in the customer VCN.

[0107] Customers can control routing in their VCNs at the subnet level; thus, they can specify which subnets of their VCNs, such as VCN 104, use each gateway. A VCN's routing tables are used to determine whether traffic is allowed to exit a VCN through a particular gateway. For example, in a particular case, the routing table for a public subnet in customer VCN 104 can send non-local traffic through IGW 120. The routing table for a private subnet in the same customer VCN 104 can send traffic destined for CSP services through SGW 126. All remaining traffic can be sent through NAT gateway 128. The routing tables control only traffic that exits a VCN.

[0108] Security lists associated with a VCN are used to control traffic entering the VCN through the gateway via incoming connections. All resources within a subnet use the same routing table and security lists. Security lists can be used to control specific types of traffic allowed into and out of instances within a subnet of the VCN. Security list rules can include ingress (inbound) and egress (outbound) rules. For example, ingress rules can specify allowed source address ranges, while egress rules can specify allowed destination address ranges. Security rules can specify specific protocols (e.g., TCP, ICMP), specific ports (e.g., 22 for SSH, 3389 for Windows RDP), etc. In some embodiments, the instance's operating system can enforce its own firewall rules that are aligned with the security list rules. Rules can be stateful (e.g., connections are tracked and responses are automatically allowed without explicit security list rules for the response traffic) or stateless.

[0109] Access from a customer VCN (i.e., by resources or compute instances deployed on VCN 104) can be categorized as public access, private access, or private access. Public access refers to an access model in which public IP addresses or NATs are used to access public endpoints. Private access allows customer workloads in VCN 104 with private IP addresses (e.g., resources in a private subnet) to access services without traversing a public network such as the Internet. In one embodiment, CSPI 101 enables customer VCN workloads with private IP addresses to access services (public service endpoints) using a service gateway. Thus, the service gateway provides a private access model by establishing a virtual link between the customer VCN and the service's endpoints, which reside outside the customer's private network.

[0110] Additionally, CSPI can provide dedicated public access using technologies such as FastConnect public peering, which allows customer on-premises instances to use a FastConnect connection to access one or more services in the customer VCN without traversing a public network such as the Internet. CSPI can also provide dedicated private access using FastConnect private peering, which allows customer on-premises instances with private IP addresses to access customer VCN workloads using a FastConnect connection. FastConnect is a network connection alternative to using the public Internet to connect a customer's on-premises network to CSPI and its services. FastConnect provides an easy, elastic, and economical way to create a dedicated private connection, with higher bandwidth options and a more reliable and consistent networking experience compared to Internet-based connections.

[0111] FIG. 1 and the accompanying discussion above describe various virtualized components in an exemplary virtual network. As noted above, a virtual network is built on an underlying physical or infrastructure network. FIG. 2 shows a simplified architectural diagram of physical components in a physical network within CSPI 200 that provides the underlying layer of the virtual network, according to one embodiment. As shown, CSPI 200 provides a distributed environment including components and resources (e.g., compute, memory, and networking resources) provided by a cloud service provider (CSP). These components and resources are used to provide cloud services (e.g., IaaS services) to subscribing customers, i.e., customers who subscribe to one or more services offered by the CSP. Based on the services to which the customer subscribes, a subset of CSPI 200's resources (e.g., compute, memory, and networking resources) is provisioned to the customer. The customer can then build their own cloud-based (i.e., CSPI-hosted), customizable private virtual network using the physical compute, memory, and networking resources provided by CSPI 200. As previously indicated, these customer networks are referred to as virtual cloud networks (VCNs). Customers can deploy one or more customer resources, such as compute instances, on these customer VCNs. The compute instances can be virtual machines, bare metal instances, etc. CSPI 200 provides the infrastructure and a set of complementary cloud services that enable customers to build and run a wide variety of applications and services in a highly available hosted environment.

[0112] In the exemplary embodiment shown in FIG. 2, the physical components of CSPI 200 include one or more physical host machines or servers (e.g., 202, 206, 208), network virtualization devices (NVDs) (e.g., 210, 212), top-of-rack (TOR) switches (e.g., 214, 216), and a physical network (e.g., 218), as well as switches within physical network 218. The physical host machines or servers can host and execute various compute instances that participate in one or more subnets of a VCN. The compute instances may include virtual machine instances or bare metal instances. For example, the various compute instances shown in FIG. 1 may be hosted by the physical host machines shown in FIG. 2. The virtual machine compute instances in a VCN may be executed by a single host machine or by multiple different host machines. The physical host machines may also host virtual host machines, container-based hosts or functions, etc. The VNICs and VCN VRs shown in FIG. 1 may be executed by the NVDs shown in FIG. 2. The gateway shown in FIG. 1 may be implemented by a host machine and / or NVD shown in FIG.

[0113] A host machine or server may run a hypervisor (also referred to as a virtual machine monitor or VMM), which creates and enables a virtualized environment on the host machine. Virtualization or virtualized environments facilitate cloud-based computing. One or more compute instances may be created, run, and managed on the host machine by the hypervisor on the host machine. The hypervisor on the host machine enables the host machine's physical computing resources (e.g., compute, memory, and networking resources) to be shared among the various compute instances executed by the host machine.

[0114] For example, as shown in FIG. 2, host machines 202 and 208 execute hypervisors 260 and 266, respectively. These hypervisors may be implemented using software, firmware, or hardware, or a combination thereof. Typically, a hypervisor is a process or software layer that runs on the host machine's hardware processor and resides above the host machine's operating system (OS). A hypervisor provides a virtualized environment by allowing the host machine's physical computing resources (e.g., processing resources such as processors / cores, memory resources, and networking resources) to be shared among various virtual machine computing instances executed by the host machine. For example, in FIG. 2, hypervisor 260 can reside above the OS of host machine 202 and allow the host machine's computing resources (e.g., processing, memory, and networking resources) to be shared among computing instances (e.g., virtual machines) executed by host machine 202. A virtual machine can have its own operating system (referred to as a guest operating system), which may be the same as or different from the host machine's OS. The operating system of a virtual machine executed by a host machine may be the same as or different from the operating system of another virtual machine executed by the same host machine. Thus, the hypervisor allows multiple operating systems to run side by side with each other while sharing the same computing resources of the host machine. The host machines shown in Figure 2 may have the same or different types of hypervisors.

[0115] A compute instance can be a virtual machine instance or a bare metal instance. In Figure 2, compute instance 268 on host machine 202 and compute instance 274 on host machine 208 are examples of virtual machine instances. Host machine 206 is an example of a bare metal instance provided to a customer.

[0116] In some cases, an entire host machine may be provisioned to a single customer, and one or more compute instances (either virtual machines or bare metal instances) hosted by that host machine all belong to that same customer. In other cases, a host machine may be shared among multiple customers (i.e., multiple tenants). In such a multi-tenancy scenario, a host machine may host virtual machine compute instances belonging to different customers. These compute instances may be members of different VCNs for different customers. In some embodiments, bare metal compute instances are hosted by bare metal servers without a hypervisor. When a bare metal compute instance is provisioned, a single customer or tenant maintains control of the physical CPU, memory, and network interfaces of the host machine that hosts the bare metal instance, and the host machine is not shared with other customers or tenants.

[0117] As previously described, each compute instance that is part of a VCN is associated with a VNIC that enables the compute instance to be a member of a subnet of the VCN. A VNIC associated with a compute instance facilitates communication of packets or frames to and from the compute instance. When a compute instance is created, a VNIC is associated with the compute instance. In one embodiment, for a compute instance executed by a host machine, the VNIC associated with the compute instance is executed by an NVD connected to the host machine. For example, in FIG. 2, host machine 202 executes virtual machine compute instance 268 that is associated with VNIC 276, which is executed by NVD 210 connected to host machine 202. As another example, bare metal instance 272 hosted by host machine 206 is associated with VNIC 280, which is executed by NVD 212 connected to host machine 206. As yet another example, VNIC 284 is associated with compute instance 274 executed by host machine 208, which is executed by NVD 212 connected to host machine 208.

[0118] For compute instances hosted by a host machine, the NVD connected to that host machine also executes VCN VRs corresponding to the VCNs of which the compute instance is a member. For example, in the embodiment shown in Figure 2, NVD 210 executes VCN VR 277 corresponding to the VCN of which compute instance 268 is a member. NVD 212 may also execute one or more VCN VRs 283 corresponding to the VCNs corresponding to the compute instances hosted by host machines 206 and 208.

[0119] A host machine may include one or more network interface cards (NICs) that allow the host machine to be connected to other devices. The NICs on a host machine may provide one or more ports (or interfaces) that allow the host machine to be communicatively connected to another device. For example, a host machine may be connected to an NVD using one or more ports (or interfaces) provided on the host machine and the NVD. A host machine may also be connected to other devices, such as another host machine.

[0120] 2, host machine 202 is connected to NVD 210 using link 220 extending between port 234 provided by NIC 232 of host machine 202 and port 236 of NVD 210. Host machine 206 is connected to NVD 212 using link 224 extending between port 246 provided by NIC 244 of host machine 206 and port 248 of NVD 212. Host machine 208 is connected to NVD 212 using link 226 extending between port 252 provided by NIC 250 of host machine 208 and port 254 of NVD 212.

[0121] The NVDs are then connected via communication links to top-of-rack (TOR) switches, which are connected to a physical network 218 (also referred to as a switch fabric). In one embodiment, the links between the host machines and the NVDs and between the NVDs and the TOR switches are Ethernet links. For example, in Figure 2, NVDs 210 and 212 are connected to TOR switches 214 and 216 using links 228 and 230, respectively. In one embodiment, links 220, 224, 226, 228, and 230 are Ethernet links. The collection of host machines and NVDs connected to a TOR may be referred to as a rack.

[0122] The physical network 218 provides a communications fabric that allows the TOR switches to communicate with each other. The physical network 218 can be a multi-tiered network. In one embodiment, the physical network 218 is a multi-tiered Clos network of switches, with the TOR switches 214 and 216 representing leaf-level nodes of the multi-tiered, multi-node physical switching network 218. Different Clos network configurations are possible, including, but not limited to, two-tier networks, three-tier networks, four-tier networks, five-tier networks, and generally "n"-tier networks. An example of a Clos network is shown in FIG. 5 and described below.

[0123] A variety of different connection configurations between host machines and NVDs are possible, such as one-to-one configurations, many-to-one configurations, one-to-many configurations, etc. In a one-to-one configuration embodiment, each host machine is connected to its own separate NVD. For example, in Figure 2, host machine 202 is connected to NVD 210 via NIC 232 of host machine 202. In a many-to-one configuration, multiple host machines are connected to one NVD. For example, in Figure 2, host machines 206 and 208 are connected to the same NVD 212 via NICs 244 and 250, respectively.

[0124] In a one-to-many configuration, one host machine is connected to multiple NVDs. Figure 3 shows an example of a CSPI 300 in which a host machine is connected to multiple NVDs. As shown in Figure 3, host machine 302 includes a network interface card (NIC) 304 that includes multiple ports 306 and 308. Host machine 300 is connected to a first NVD 310 via port 306 and link 320, and to a second NVD 312 via port 308 and link 322. Ports 306 and 308 may be Ethernet ports, and links 320 and 322 between host machine 302 and NVDs 310 and 312 may be Ethernet links. NVD 310 is then connected to a first TOR switch 314, and NVD 312 is connected to a second TOR switch 316. The links between NVDs 310 and 312 and TOR switches 314 and 316 may be Ethernet links. TOR switches 314 and 316 represent tier 0 switching devices within a multi-tiered physical network 318 .

[0125] 3 provides two separate physical network paths from the physical switch network 318 to the host machine 302 and from the host machine 302 to the physical switch network 318: a first path traversing the TOR switch 314-NVD 310-host machine 302, and a second path traversing the TOR switch 316-NVD 312-host machine 302. These separate paths provide increased availability (referred to as high availability) of the host machine 302. If there is a problem with one of the paths (e.g., one link in the path fails) or one of the devices (e.g., a particular NVD is not functioning), the other path may be used for communications to / from the host machine 302.

[0126] In the configuration shown in Figure 3, the host machine is connected to two different NVDs using two different ports provided by the host machine's NIC. In other embodiments, the host machine may include multiple NICs that enable the host machine's connectivity to multiple NVDs.

[0127] Referring back to Figure 2, an NVD is a physical device or component that implements one or more network and / or storage virtualization functions. An NVD may be any device that has one or more processing units (e.g., a CPU, a network processing unit (NPU), an FPGA, a packet processing pipeline, etc.), memory including cache, and ports. Various virtualization functions may be implemented by software / firmware executed by the one or more processing units of the NVD.

[0128] The NVD may be implemented in a variety of different forms. For example, in one embodiment, the NVD is implemented as an interface card, referred to as a smartNIC or intelligent NIC, that has an on-board processor. A smartNIC is a separate device from the NIC on the host machine. In Figure 2, NVDs 210 and 212 may be implemented as smartNICs connected to host machine 202 and host machines 206 and 208, respectively.

[0129] However, smartNIC is merely one example of an NVD implementation. Various other implementations are possible. For example, in some other embodiments, the NVD or one or more functions performed by the NVD may be incorporated into or performed by one or more host machines, one or more TOR switches, and other components of CSPI 200. For example, the NVD may be embodied within a host machine, where the functions performed by the NVD are performed by the host machine. As another example, the NVD may be part of a TOR switch, or a TOR switch may be configured to perform the functions performed by the NVD that enable the TOR switch to perform various complex packet transformations used in public clouds. A TOR that performs the functions of an NVD may be referred to as a smart TOR. In yet other embodiments where customers are provided with virtual machine (VM) instances rather than bare metal (BM) instances, the functions performed by the NVD may be implemented inside the hypervisor of the host machine. In some other embodiments, some of the NVD's functions may be offloaded to a centralized service running on a fleet of host machines.

[0130] In some embodiments, such as when implemented as a smartNIC as shown in FIG. 2, an NVD may comprise multiple physical ports that allow it to be connected to one or more host machines and one or more TOR switches. Ports on an NVD can be classified as host-facing ports (also referred to as "south ports") or network-facing or TOR-facing ports (also referred to as "north ports"). A host-facing port of an NVD is a port used to connect the NVD to a host machine. Examples of host-facing ports in FIG. 2 include port 236 on NVD 210 and ports 248 and 254 on NVD 212. A network-facing port of an NVD is a port used to connect the NVD to a TOR switch. Examples of network-facing ports in FIG. 2 include port 256 on NVD 210 and port 258 on NVD 212. As shown in FIG. 2, NVD 210 is connected to TOR switch 214 using link 228 extending from port 256 of NVD 210 to TOR switch 214. Similarly, the NVD 212 is connected to the TOR switch 216 using a link 230 that extends from a port 258 of the NVD 212 to the TOR switch 216 .

[0131] The NVD can receive packets and frames from a host machine (e.g., packets and frames generated by a compute instance hosted by the host machine) via a host-side port, perform any necessary packet processing, and then forward the packets and frames to a TOR switch via a network-side port of the NVD. The NVD can receive packets and frames from a TOR switch via a network-side port of the NVD, and perform any necessary packet processing, and then forward the packets and frames to a host machine via a host-side port of the NVD.

[0132] In some embodiments, there may be multiple ports and associated links between the NVD and the TOR switch. These ports and links can be aggregated to form a link aggregator group (referred to as a LAG) of multiple ports or links. Link aggregation allows multiple physical links between two endpoints (e.g., between the NVD and the TOR switch) to be treated as a single logical link. All physical links within a given LAG may operate at the same speed and in full-duplex mode. LAGs help increase the bandwidth and reliability of the connection between two endpoints. If one of the physical links in the LAG fails, traffic is dynamically and transparently reassigned to one of the other physical links in the LAG. The aggregated physical link delivers higher bandwidth than each individual link. Multiple ports associated with a LAG are treated as a single logical port. Traffic can be load-balanced across the multiple physical links of a LAG. One or more LAGs can be configured between two endpoints. The two endpoints may be between the NVD and the TOR switch, between a host machine and the NVD, etc.

[0133] The NVD implements or performs network virtualization functions. These functions are performed by software / firmware executed by the NVD. Examples of network virtualization functions include, but are not limited to, packet encapsulation and decapsulation functions, functions for creating VCN networks, functions for implementing network policies such as VCN security list (firewall) functions, functions for facilitating routing and forwarding of packets to and from compute instances within the VCN, etc. In one embodiment, when a packet is received, the NVD is configured to execute a packet processing pipeline to process the packet and determine how the packet is to be forwarded or routed. As part of this packet processing pipeline, the NVD can execute one or more virtual functions associated with the overlay network, such as running VNICs associated with compute instances within the VCN, running virtual routers (VRs) associated with the VCN, encapsulating and decapsulating packets to facilitate forwarding or routing within the virtual network, running certain gateways (e.g., local peering gateways), implementing security lists, network security groups, network address translation (NAT) functions (e.g., translating per-host public IPs to private IPs), throttling functions, and other functions.

[0134] In one embodiment, the packet processing data path within the NVD may include multiple packet pipelines, each consisting of a series of packet transformation stages. In one embodiment, when a packet is received, it is parsed and classified into a single pipeline. The packet is then processed linearly from one stage to another until the packet is dropped or sent out on an interface of the NVD. These stages provide basic functional packet processing building blocks (e.g., header validation, throttling enforcement, insertion of new Layer 2 headers, L4 firewall enforcement, VCN encapsulation / decapsulation, etc.), such that new pipelines can be constructed by composing existing stages, and new functionality can be added by creating new stages and inserting them into existing pipelines.

[0135] The NVD can implement both control plane and data plane functions corresponding to the control and data planes of a VCN. Examples of a VCN control plane are also shown in Figures 1, 2, 3, and 4 (see references 116, 216, 316, and 416) and are described below. Examples of a VCN data plane are also shown in Figures 1, 2, 3, and 4 (see references 118, 218, 318, and 418) and are described below. The control plane functions include functions used to configure the network (e.g., setting up routes and routing tables, configuring VNICs, etc.) that control how data should be forwarded. In one embodiment, a VCN control plane is provided that centrally computes all overlay-to-foundation mappings and publishes them to the NVD and virtual network edge devices such as DRGs, SGWs, IGWs, etc. Firewall rules can also be published using the same mechanism. In one embodiment, the NVD obtains only mappings that are relevant to that NVD. The data plane functions include functions for the actual routing / forwarding of packets based on the configuration set up using the control plane. The VCN data plane is implemented by encapsulating customer network packets before they traverse the underlying network. The encapsulation / decapsulation functionality is implemented on the NVD. In one embodiment, the NVD is configured to intercept all network packets entering and leaving the host machine and perform network virtualization functions.

[0136] As indicated above, the NVD performs various virtualization functions, including VNICs and VCN VRs. The NVD can execute VNICs associated with compute instances hosted by one or more host machines connected to the VNICs. For example, as shown in FIG. 2, NVD 210 executes the functions of VNIC 276 associated with compute instance 268 hosted by host machine 202 connected to NVD 210. As another example, NVD 212 executes VNIC 280 associated with bare metal compute instance 272 hosted by host machine 206 and VNIC 284 associated with compute instance 274 hosted by host machine 208. The host machines can host compute instances that belong to different VCNs that belong to different customers, and the NVDs connected to the host machines can execute VNICs (i.e., perform VNIC-related functions) corresponding to the compute instances.

[0137] NVDs also run VCN virtual routers corresponding to the VCNs of the compute instances. For example, in the embodiment shown in FIG. 2, NVD 210 runs VCN VR 277 corresponding to the VCN to which compute instance 268 belongs. NVD 212 runs one or more VCN VRs 283 corresponding to one or more VCNs to which compute instances hosted by host machines 206 and 208 belong. In one embodiment, a VCN VR corresponding to that VCN is run by all NVDs connected to a host machine that hosts at least one compute instance belonging to that VCN. If a host machine hosts compute instances that belong to different VCNs, the NVDs connected to that host machine may run VCN VRs corresponding to those different VCNs.

[0138] In addition to VNICs and VCN VRs, an NVD may include one or more hardware components that run various software (e.g., daemons) and facilitate various network virtualization functions performed by the NVD. For simplicity, these various components are grouped together as a “packet processing component” shown in FIG. 2 . For example, NVD 210 includes packet processing component 286, and NVD 212 includes packet processing component 288. For example, the packet processing component of an NVD may include a packet processor configured to interact with the NVD's ports and hardware interfaces to monitor all packets received by and communicated using the NVD and store network information. The network information may include, for example, network flow information that identifies different network flows handled by the NVD per flow information (e.g., per flow statistics). In one embodiment, the network flow information may be stored per VNIC. The packet processor may perform per-packet operations and implement a stateful NAT and an L4 firewall (FW). As another example, the packet processing component may include a replication agent configured to replicate information stored by the NVD to one or more different replication target stores. As yet another example, the packet processing component may include a logging agent configured to perform logging functions for the NVD. The packet processing component may also include software for monitoring the performance and health of the NVD, and possibly also the status and health of other components connected to the NVD.

[0139] FIG. 1 illustrates components of an exemplary virtual or overlay network, including a VCN, subnets within the VCN, compute instances deployed on the subnets, VNICs associated with the compute instances, VRs for the VCN, and a set of gateways configured for the VCN. The overlay components illustrated in FIG. 1 may be executed or hosted by one or more of the physical components illustrated in FIG. 2. For example, compute instances in a VCN may be executed or hosted by one or more host machines illustrated in FIG. 2. For compute instances hosted by a host machine, the VNICs associated with the compute instance are typically executed by the NVD connected to the host machine (i.e., the VNIC functionality is provided by the NVD connected to the host machine). The VCN VR functionality for a VCN is performed by all NVDs connected to the host machines that host or execute compute instances that are part of the VCN. The gateways associated with a VCN may be executed by one or more different types of NVDs. For example, some gateways may be executed by smartNICs, while other gateways may be executed by one or more host machines or other implementations of NVDs.

[0140] As described above, a compute instance in a customer VCN can communicate with a variety of different endpoints, where the endpoints can be in the same subnet as the source compute instance, in a different subnet than the source compute instance but in the same VCN, or the endpoints can be outside the VCN of the source compute instance. These communications are facilitated using VNICs associated with the compute instance, VCN VRs, and gateways associated with the VCN.

[0141] For communication between two compute instances on the same subnet within a VCN, the communication is facilitated using VNICs associated with the source and destination compute instances. The source and destination compute instances may be hosted by the same host machine or may be hosted by different host machines. A packet originating from a source compute instance can be forwarded from the host machine hosting the source compute instance to an NVD connected to that host machine. On the NVD, the packet is processed using a packet processing pipeline, which may include running the VNIC associated with the source compute instance. Because the packet's destination endpoint is within the same subnet, running the VNIC associated with the source compute instance results in the packet being forwarded to an NVD running the VNIC associated with the destination compute instance, which then processes the packet and forwards it to the destination compute instance. The VNICs associated with the source and destination compute instances may run on the same NVD (e.g., when both the source and destination compute instances are hosted by the same host machine) or on different NVDs (e.g., when the source and destination compute instances are hosted by different host machines connected to different NVDs). The VNIC can use the routing / forwarding tables stored by the NVD to determine the next hop for the packet.

[0142] To allow packets to be communicated from a compute instance in a subnet to an endpoint in a different subnet within the same VCN, packets originating from a source compute instance are communicated from a host machine hosting the source compute instance to an NVD connected to that host machine. On the NVD, the packets are processed using a packet processing pipeline that may include the execution of one or more VNICs and VRs associated with the VCN. For example, as part of the packet processing pipeline, the NVD executes or invokes functions corresponding to the VNICs associated with the source compute instance (also referred to as executing VNICs). The functions performed by the VNICs may include looking at the VLAN tag on the packet. Because the packet destination is outside the subnet, VCN VR functions are then invoked and executed by the NVD. The VCN VR then routes the packet to an NVD that executes the VNIC associated with the destination compute instance. The VNIC associated with the destination compute instance then processes the packet and forwards it to the destination compute instance. The VNICs associated with the source and destination compute instances may run on the same NVD (e.g., when both the source and destination compute instances are hosted by the same host machine) or may run on different NVDs (e.g., when the source and destination compute instances are hosted by different host machines connected to different NVDs).

[0143] If the packet's destination is outside the VCN of the source compute instance, the packet originating from the source compute instance is communicated from the host machine hosting the source compute instance to the NVD connected to that host machine. The NVD runs a VNIC associated with the source compute instance. Because the packet's destination endpoint is outside the VCN, the packet is then processed by the VCN VR for that VCN. The NVD invokes a VCN VR function, which results in the packet being forwarded to an NVD running the appropriate gateway associated with the VCN. For example, if the destination is an endpoint in a customer's on-premises network, the packet can be forwarded by the VCN VR to an NVD running a DRG gateway configured for the VCN. The VCN VR may be run on the same NVD as the NVD running the VNIC associated with the source compute instance or by a different NVD. The gateway may be run by the NVD, which may be a smartNIC, a host machine, or another NVD implementation. The packet may then be processed by the gateway and forwarded to the next hop, which facilitates communication of the packet to its final intended destination. 2, a packet originating from compute instance 268 may be communicated from host machine 202 over link 220 (using NIC 232) to NVD 210. On NVD 210, VNIC 276 is called out because it is the VNIC associated with source compute instance 268. VNIC 276 is configured to examine encapsulation information within the packet, determine a next hop to forward the packet to with the goal of facilitating communication of the packet to its intended destination endpoint, and then forward the packet to the determined next hop.

[0144] Compute instances deployed on a VCN can communicate with a variety of different endpoints. These endpoints may include endpoints hosted by CSPI 200 and endpoints external to CSPI 200. Endpoints hosted by CSPI 200 may include instances in the same VCN or another VCN, which may be a customer VCN or a VCN not belonging to the customer. Communication between endpoints hosted by CSPI 200 may be conducted over physical network 218. Compute instances may also communicate with endpoints not hosted by CSPI 200 or external to CSPI 200. Examples of these endpoints include endpoints in a customer's on-premises network or data center, or public endpoints accessible over a public network such as the Internet. Communication with endpoints external to CSPI 200 may be conducted over a public network (e.g., the Internet) (not shown in FIG. 2) or a private network (not shown in FIG. 2) using various communication protocols.

[0145] The architecture of CSPI 200 shown in FIG. 2 is illustrative only and is not intended to be limiting. Variations, substitutions, and modifications are possible in alternative embodiments. For example, in some other embodiments, CSPI 200 may have more or fewer systems or components than those shown in FIG. 2, may combine two or more systems, or may have a different configuration or arrangement of systems. The systems, subsystems, and other components shown in FIG. 2 may be implemented in software (e.g., code, instructions, programs) executed by one or more processing units (e.g., processors, cores) of the respective systems, using hardware, or a combination thereof. The software may be stored on a non-transitory storage medium (e.g., on a memory device).

[0146] FIG. 4 illustrates connectivity between host machines and an NVD to provide I / O virtualization to support multitenancy functionality according to one embodiment. As shown in FIG. 4, a host machine 402 runs a hypervisor 404 that provides a virtualized environment. The host machine 402 runs two virtual machine instances: VM1 406 belonging to customer / tenant #1 and VM2 408 belonging to customer / tenant #2. The host machine 402 includes a physical NIC 410 connected to an NVD 412 via link 414. Each of the compute instances is attached to a VNIC implemented by the NVD 412. In the embodiment of FIG. 4, VM1 406 is attached to VNIC-VM1 420, and VM2 408 is attached to VNIC-VM2 422.

[0147] 4, NIC 410 includes two logical NICs: logical NIC A 416 and logical NIC B 418. Each virtual machine is attached to and configured to work with its own logical NIC. For example, VM1 406 is attached to logical NIC A 416, and VM2 408 is attached to NIC B 418. Although host machine 402 includes only one physical NIC 410 shared by multiple tenants, due to the logical NICs, each tenant's virtual machine believes it has its own host machine and NIC.

[0148] In one embodiment, each logical NIC is assigned its own VLAN ID. Thus, a particular VLAN ID is assigned to logical NIC A 416 for tenant #1, and a separate VLAN ID is assigned to logical NIC B 418 for tenant #2. When a packet is communicated from VM1 406, a tag assigned to tenant #1 is attached to the packet by the hypervisor, and the packet is then communicated from host machine 402 to NVD 412 over link 414. Similarly, when a packet is communicated from VM2 408, a tag assigned to tenant #2 is attached to the packet by the hypervisor, and the packet is then communicated from host machine 402 to NVD 412 over link 414. Thus, a packet 424 communicated from host machine 402 to NVD 412 has an associated tag 426 that identifies the particular tenant and associated VM. On the NVD, for a packet 424 received from host machine 402, a tag 426 associated with the packet is used to determine whether the packet should be processed by VNIC-VM1 420 or VNIC-VM2 422. The packet is then processed by the corresponding VNIC. The configuration shown in Figure 4 allows each tenant's compute instances to think that they own their own host machine and NIC. The setup shown in Figure 4 provides I / O virtualization to support multi-tenancy.

[0149] FIG. 5 illustrates a simplified block diagram of a physical network 500 according to one embodiment. The embodiment illustrated in FIG. 5 is structured as a Clos network. A Clos network is a specific type of network topology designed to provide connection redundancy while maintaining high bisection bandwidth and maximum resource utilization. A Clos network is a type of non-blocking, multi-stage or multi-tier switching network, and the number of stages or tiers can be 2, 3, 4, 5, etc. The embodiment illustrated in FIG. 5 is a three-tier network with tiers 1, 2, and 3. TOR switch 504 represents a tier 0 switch in the Clos network. One or more NVDs are connected to the TOR switch. A tier 0 switch is also referred to as an edge device of the physical network. A tier 0 switch is connected to a tier 1 switch, also referred to as a leaf switch. In the embodiment illustrated in FIG. 5, a set of “n” tier 0 TOR switches is connected to a set of “n” tier 1 switches, together forming a pod. Each tier 0 switch in a pod is interconnected to all tier 1 switches within the pod, but there is no switch connectivity between pods. In one embodiment, two pods are referred to as blocks. Each block is served by or connected to a set of “n” tier-2 switches (sometimes referred to as spine switches). There may be several blocks in a physical network topology. The tier-2 switches are then connected to “n” tier-3 switches (sometimes referred to as super-spine switches). Communication of packets over the physical network 500 is typically performed using one or more layer-3 communication protocols. Typically, all layers of the physical network except the TOR layer are n-way redundant, thus enabling high availability. Policies can be specified for pods and blocks to control the visibility of switches in the physical network to each other to enable scaling of the physical network.

[0150] A feature of Clos networks is that the maximum hop count required to reach from one tier-0 switch to another tier-0 switch (or from an NVD connected to a tier-0 switch to another NVD connected to a tier-0 switch) is fixed. For example, in a three-tier Clos network, a packet requires a maximum of seven hops to reach from one NVD to another, where the source and destination NVDs are connected to the leaf tiers of the Clos network. Similarly, in a four-tier Clos network, a packet requires a maximum of nine hops to reach from one NVD to another, where the source and destination NVDs are connected to the leaf tiers of the Clos network. Therefore, the Clos network architecture maintains consistent latency throughout the network, which is important for communication within and between data centers. Clos topologies are horizontally scalable and cost-efficient. The network's bandwidth / throughput capacity can be easily increased by adding more switches (e.g., more leaf and spine switches) to various tiers and by increasing the number of links between switches in adjacent tiers.

[0151] In one embodiment, each resource in CSPI is assigned a unique identifier called a Cloud Identifier (CID). This identifier is included as part of the resource's information and can be used to manage the resource, for example, via a console or through an API. An exemplary syntax for a CID is as follows:

[0152] ocid1.<RESOURCE TYPE> . <realm>.[REGION][.FUTURE USE].<UNIQUE ID> where: ocid1: A string indicating the version of the CID.

[0153] resource type: The type of resource (for example, instance, volume, VCN, subnet, user, group, etc.).

[0154] realm: The realm the resource is in. Example values ​​are "c1" for a commercial realm, "c2" for a government cloud realm, or "c3" for a federal cloud realm. Each realm can have its own domain name.

[0155] region: The region the resource is in. This part can be blank if no region is applicable to the resource.

[0156] future use: Reserved for future use. unique ID: The unique part of the ID. The format may vary depending on the type of resource or service.

[0157] Virtual Private Label Cloud (vPLC) Techniques are described herein that enable cloud infrastructure within a region provided by a cloud service provider (CSP) to be used to create one or more virtual private clouds (referred to herein as virtual private label clouds or "vPLCs") for providing cloud services supplied by the CSP to customers of the CSP. A vPLC created according to the various techniques described herein can be used for a variety of purposes. For example, in one use case, a vPLC may be created for a reseller who is a customer of the CSP and used by the reseller to provide one or more reseller-supplied cloud services to the reseller's customers. As another use case, a vPLC may be used as a virtual data center and associated with a realm different from the realm associated with cloud infrastructure within a region provided by the CSP.

[0158] A cloud service provider (CSP) provides infrastructure (referred to as cloud service provider infrastructure or CSPI or CSP-delivered infrastructure) used to deliver one or more cloud services to the CSP's customers. CSP customers can include the CSP's direct customers and / or resellers. CSPI can include physical infrastructure components and virtual infrastructure components. Physical CSPI can include interconnected high-performance computing resources, including racks, servers and host machines, memory resources, and network resources used to form a physical foundation or underlay network, network virtualization devices, and other resources. Virtual CSPI components may include components in an overlay network, such as virtual machines, virtual routers and gateways, and other virtual components.

[0159] A CSP may provide various types of cloud services. These may include various types of services, including software as a service (SaaS), platform as a service (PaaS), infrastructure as a service (IaaS), etc. In one embodiment, as described in this disclosure, a CSP may provide a "vPLC cloud service." A customer of a CSP may subscribe to one or more cloud services offered by the CSP. A customer may be any entity, such as an individual, an organization, a business, etc.

[0160] Infrastructure as a Service (IaaS) is a specific type of cloud computing service. In the IaaS model, a CSP provides infrastructure that customers can use to build their own customizable networks and deploy customer resources. Therefore, customer resources and networks are hosted in a distributed environment by infrastructure provided by the CSP. This differs from traditional computing, in which customer resources and networks are hosted by infrastructure provided by the customer. In one embodiment, a vPLC as a Service is a type of IaaS service in which infrastructure is allocated to create a virtual private label cloud for a customer. For example, if a CSP customer is a reseller, the vPLC created for the reseller can be used to provide a reseller-supplied cloud service.

[0161] When a customer subscribes or registers for a cloud service offered by a CSP, a tenancy (or account) is created for that customer. The customer can then access one or more subscribed cloud resources associated with the account / tenancy through this account / tenancy. Each tenancy is identified by a tenancy identifier (T_ID) that is assigned to the tenancy and uniquely identifies the tenancy. The tenancy identifier is generated and associated with the tenancy when the tenancy is created. For example, when a direct customer of a CSP subscribes to a service offered by the CSP, a tenancy is created for the customer, and a tenancy identifier that uniquely identifies the tenancy is created for the direct customer. When a reseller subscribes to a vPLC service, a tenancy is created for the reseller, and a tenancy identifier that uniquely identifies the tenancy is created for the reseller. A tenancy can be a logical and secure partition that contains all resources and services utilized by a customer associated with the tenancy.

[0162] Resources within a CSP-provided infrastructure are typically identified using a unique resource identifier (resource ID or RID). Because these resources are provided within the cloud and used to provide cloud services, resource IDs are also referred to as cloud resource identifiers (or cloud IDs or CIDs). For example, in the cloud environment provided by Oracle Corporation, resources are identified using an "ocid" or "Oracle cloud identifier." In some embodiments, a resource ID is a globally unique identifier string that identifies a cloud resource. In some embodiments, a tenancy ID is one type of cloud resource ID (or referred to as a cloud identifier (CID)).

[0163] A resource or cloud ID (CID) may be represented in the following format, as an example:

[0164] RID_version. <resourcetype> . <realm>.[REGION][.FUTURE USE].<UNIQUE ID> where: RID_version - Refers to the version of the resource ID.

[0165] "resource type" - Identifies the particular type of resource that the resource ID refers to. Examples of different resource types include instances, virtual network subnets, tenancies, etc. For example, an "instance RID" identifies a compute instance.

[0166] "realm" and "region" - Indicates the location (realm and region) of the resource.

[0167] "unique ID" - The unique part of the ID (e.g., a unique string to identify a specific resource). Thus, the tenancy ID can identify a unique ID within a CID's tenancy resource type.

[0168] CIDs are generated for resources when those resources are created. Virtual machines, accounts, tenancies, authentication policies, virtual network subnets, access control lists (ACLs), and load balancers are examples of cloud resources and can therefore be referenced by their respective CIDs. A tenancy is a logical container used to group other cloud resources that belong to the same cloud customer account.

[0169] When an entity becomes a customer of a CSP and subscribes to a vPLC service, a vPLC is created for the customer and associated with the customer's tenancy. For example, if the customer is a reseller, a vPLC is created for the reseller and associated with the reseller's tenancy. A vPLC created for a reseller is treated as a resource and assigned a unique resource identifier, referred to as a vPLC identifier (vPLC ID), and the vPLC ID is associated with the reseller's tenancy. As described herein, for a vPLC created for a reseller, the vPLC ID of the vPLC is used to identify all resources and requests related to the vPLC. Each vPLC ID uniquely identifies a vPLC.

[0170] For example, in one embodiment, all resources allocated to a vPLC are tagged with the vPLC identifier of that vPLC. In one embodiment, a vPLC ID field is inserted into the representation of the CID described above. When a resource is associated with a vPLC, the vPLC ID corresponding to that vPLC may be provided in this field of the CID; otherwise, if the resource is not associated with a vPLC, the field may be left blank.

[0171] For the purposes of this disclosure, the following terminology will be used for the sake of clarity. CSP.Cn (or simply Cn) - identifies a direct, non-reseller customer of the CSP. For example, "CSP.C1" refers to the CSP's non-reseller direct customer C1, and "CSP.C2" refers to the CSP's non-reseller direct customer C2. CSP.Rn (or simply Rn) - identifies a CSP's reseller customer. For example, "CSP.R1" refers to CSP's reseller customer R1, and "CSP.R2" refers to CSP's reseller customer R2. Rn.Cm - Resellers can have their own customers. This identifies a specific customer of a specific reseller. For example, "R1.C1" refers to customer C1 of reseller R1, and "R2.C1" refers to customer C1 of reseller R2. T.Cn - Identifies the tenancy for a CSP's reseller customer. For example, "T.C1" refers to the tenancy for direct non-reseller customer C1. T.Rn - Identifies the tenancy for the reseller. For example, "T.R1" refers to the tenancy for reseller R1. T.Rn.Cm - Each reseller's customer has their own tenancy. This identifies the tenancy associated with a particular customer of a particular reseller. For example, "T.R1.C1" refers to the tenancy associated with reseller R1's customer C1, and "T.R2.C1" refers to the tenancy associated with reseller R2's customer C1. T_ID.T.Cn - Identifies the tenancy identifier that identifies the tenancy for a CSP's non-reseller customer. For example, "T_ID.T.C1" refers to the tenancy ID of the tenancy for direct non-reseller customer C1. T_ID.T.Rn - Identifies the tenancy identifier of the tenancy for the reseller. For example, "T_ID.T.R1" refers to the tenancy identifier of the tenancy for reseller R1. T_ID.Rn.Cm - Identifies the tenancy identifier of the tenancy associated with a particular customer of a particular reseller. For example, "T_ID.T.R1.C1" refers to the tenancy identifier of the tenancy associated with customer C1 of reseller R1, and "T_ID.T.R2.C1" refers to the tenancy identifier of the tenancy associated with customer C1 of reseller R2. vPLC.Rn - Identifies a vPLC being created for reseller Rn. For example, "vPLC.R1" identifies the tenancy being created for and associated with reseller R1. This identifies a specific vPLC being created for and associated with the reseller's tenancy. vPLC_ID.vPLC.Rn - Identifies the vPLC identifier of a vPLC being created for a specific reseller. For example, "vPLC_ID.vPLC.R1" refers to the vPLC identifier of a vPLC being created for reseller R1.

[0172] As described herein, a vPLC is created for a reseller, who is a customer of a CSP. The vPLC is associated with the reseller's tenancy. In some embodiments, a vPLC ID, which identifies the vPLC, may be associated with a tenancy ID, which identifies the reseller's tenancy. Thus, given a vPLC ID, the tenancy and tenancy identifier for the reseller can be included to identify the corresponding vPLC and the reseller for whom the vPLC is being created.

[0173] CSPI or CSP-provided infrastructure may be organized into realms, regions, and data centers. A region refers to a local geographic area that includes one or more connected data centers. Regions are independent from other regions and can be separated by large distances, for example, across countries or even continents. A set of application programming interfaces (APIs) is provided for the CSP-provided infrastructure in a region, and the APIs are used to perform operations using the infrastructure in that region. Operations that can be performed using the infrastructure in a region may include creating resources in the infrastructure in that region, accessing resources in the infrastructure in the region, performing operations involving resources in the infrastructure in the region, deleting resources from the infrastructure in the region, and performing other operations involving the infrastructure in the region.

[0174] The set of APIs is region-specific. Thus, the CSP-provided infrastructure in a region is characterized by the set of APIs that can be used with the infrastructure in that region. CSP-provided infrastructure in different regions has different API sets provided for the different regions. Therefore, for purposes of this application, the phrase “CSP-provided infrastructure in a region” or “CSP-provided infrastructure in a region” means that the infrastructure is characterized by the specific API set provided for that region. Thus, a first API set may be provided for a first region, a second API set may be provided for a second region different from the first region, a third API set may be provided for a third region different from the first and second regions, and so on. Thus, a specific API set identifies the CSP-provided infrastructure in a particular region. In other words, CSP-provided infrastructure in two separate regions will have two separate API sets. Thus, a region is characterized by the infrastructure in the region and the set of APIs associated with the region for performing operations involving the region infrastructure. CSP-provided infrastructure within a region is also referred to as "region infrastructure" or "region-specific infrastructure."

[0175] For example, a CSP may provide a first infrastructure in Seattle and a second infrastructure in Portland. A first set of APIs may be provided for the first infrastructure in Seattle, and these APIs may be characterized by the following format:

[0176] "<service_identifier> .us-seattle.oci.Oraclecloud.com” A second set of APIs may be provided for a second infrastructure in Portland, and these APIs may be characterized by the following format:

[0177] "<service_identifier> .us-portland.oci.Oraclecloud.com” Note that Seattle and Portland are used here only as examples. The geographic location of the regions can vary. For example, it is possible to have two or more regions within the same geographic city. For example, it is possible to have a first region in a first building within a city and a second region in a second, separate building within the same city. As yet another example, it is possible to have a first region on one floor within a building and a second region on another floor of the building. As yet another example, for infrastructure provided by a CSP on a floor, a first portion of the infrastructure may belong to a first region associated with a first set of APIs and a second portion of the infrastructure may belong to a second region associated with a second set of APIs that is different from the first set of APIs.

[0178] CSP-provided infrastructure within a region may be organized into one or more datacenters. For example, in a particular region where a CSP may have two buildings, each hosting infrastructure used to provide cloud services, the infrastructure in the first building may be organized into a first datacenter and referred to as the first datacenter, and the infrastructure in the second building may be organized into a second datacenter and referred to as the second datacenter. As another example, in a particular region where a CSP may have a single building with multiple floors, each floor hosting infrastructure used to provide cloud services, the infrastructure on the first floor may be organized into a first datacenter and referred to as the first datacenter, and the infrastructure on the second floor may be organized into a second datacenter and referred to as the second datacenter. As yet another example, in a particular region where a CSP may have infrastructure used to provide cloud services, a first portion of the infrastructure may be organized into a first datacenter and referred to as the first datacenter, and a second separate portion of the infrastructure may be organized into a second datacenter and referred to as the second datacenter.

[0179] A region may have one or more data centers. In some embodiments, all regional infrastructure may be contained within a single data center, while in other embodiments, the infrastructure within a region may be organized into multiple data centers. Each data center may include infrastructure resources, such as compute, storage, and networking resources provided by a CSP. Within a region, the data centers within the region may be organized into one or more availability domains (ADs). Availability domains are isolated from each other, fault-tolerant, and highly unlikely to fail simultaneously. ADs are configured so that a failure in one AD within a region is unlikely to affect the availability of other ADs in the same region.

[0180] A realm refers to a logical collection of one or more regions. A realm can contain one or more regions. Realms are typically isolated from one another and do not share data. Each realm has its own identity and trust profile (e.g., passwords, credentials, etc.). The realm's identity and trust profile are configured so that cross-realm communication is not enabled. In this manner, each realm represents an isolated domain. For example, one realm may be created for a government agency that is a customer of the CSP, another realm may be created for a commercial customer of the CSP, yet another realm may be created for a private organization customer of the CSP, and so on. For regions within a realm, the region infrastructure can communicate with each other, but the region infrastructure associated with a first realm is not allowed to communicate with the region infrastructure associated with a second, different realm.

[0181] A cloud service reseller can be an entity such as a corporation (eg, a telecommunications company), a systems integrator, a government agency, an IT department of a large corporation, a school, and the like.

[0182] FIG. 6 is a simplified block diagram of a distributed environment 600 illustrating an example of a virtual private label cloud (vPLC) that includes a CSP-provided infrastructure within a region and is hosted by the CSP-provided infrastructure, according to an embodiment. The distributed environment 600 illustrated in FIG. 6 is merely exemplary and is not intended to unduly limit the scope of the claimed embodiments. Many variations, alternatives, and modifications are possible. For example, in some other embodiments, the distributed environment 600 may have more or fewer systems or components than those shown in FIG. 6, may combine two or more systems, or may have a different configuration or arrangement of systems. The systems, subsystems, and other components illustrated in FIG. 6 may be implemented in software (e.g., code, instructions, programs) executed by one or more processing units (e.g., processors, cores) of the respective systems, using hardware, or a combination thereof. The software may be stored on a non-transitory storage medium (e.g., on a memory device).

[0183] Figure 6 depicts a CSP-provided infrastructure within a region 601. The infrastructure 601 may be organized into one or more data centers. The infrastructure 601 may include memory or storage resources 622, computational resources 620, networking resources (not shown), and other physical and logical resources provided by the CSP. The storage resources 622 may include one or more volatile or non-volatile memory resources. The computational resources 620 may include one or more racks, each rack including one or more servers, each server hosting one or more operating systems.

[0184] The CSP-provided region infrastructure 601 may be used by the CSP to offer one or more CSP-supplied cloud services to one or more of the CSP's direct customers, and can also be used to create one or more vPLCs. In the example depicted in FIG. 6, two vPLCs, vPLC.R1 604 and vPLC.R2 606, have been created for resellers R1 and R2, respectively. vPLC.R1 604 is used to offer cloud services supplied by one or more resellers R1 and branded by R1 to one or more customers 642 of R1. vPLC.R2 606 is used to offer cloud services supplied by one or more resellers R2 and branded by R2 to one or more customers of R2 644.

[0185] As shown, infrastructure 601 can be communicatively coupled to computing devices used by users associated with direct customers 640 and users associated with resellers' customers (e.g., users associated with reseller R1's customers 642 and users associated with reseller R2's customers 644) via communications network 652. Communications network 652 can be of various types and can include one or more communications networks. Examples of communications network 652 include, without limitation, the Internet, a wide area network (WAN), a local area network (LAN), an Ethernet network, a public or private network, a wired network, a wireless network, etc., and combinations thereof. Different communications protocols may be used to facilitate communications, including both wired and wireless protocols such as the IEEE 802.XX protocol suite, TCP / IP, IPX, SAN, AppleTalk®, Bluetooth®, and other protocols. In general, communications network 652 may include any infrastructure that facilitates communications with infrastructure 601.

[0186] 6, infrastructure 601 is divided into three securely isolated portions: (1) a first portion 602 used to provide CSP-supplied and CSP-branded cloud services to one or more of the CSP's direct (or non-reseller) customers 640; (2) a second portion allocated to vPLC.R1 604 created for reseller R1; and (3) a third portion allocated to vPLC.R2 606 created for reseller R2. In one embodiment, these three portions are securely separated from each other.

[0187] The partitioning of resources includes partitioning of storage resources 622 and computational resources 620. For example, as shown in Figure 6, storage resources 622 include a memory portion 622a included in first portion 602 that is used to provide CSP-delivered cloud services to the CSP's direct customers. Storage resources 622 also include a memory portion 622b allocated to vPLC.R1 604 and a memory portion 622c allocated to vPLC.R2 606. The memory or memory portion allocated to a vPLC may include one or more volatile and / or non-volatile memory resources.

[0188] 6, the computational resources 620 are also partitioned. The computational resources 620 include a computational portion 620a included in the first portion 602 that is used to provide the CSP-delivered cloud services to the CSP's direct customers. The computational resources 620 also include a portion 620b allocated to vPLC.R1 604 and a portion 620c allocated to vPLC.R2 606.

[0189] As described above, a set of APIs are provided for accessing and implementing functions related to the CSP-provided infrastructure within the region. These APIs can be accessed by users associated with the CSP's direct customers 640 using a set of endpoints 610-1 provided by the CSP. For example, one or more of the endpoints 610-1 may be used by users to access and / or use compute resources 620a or memory resources 622a allocated to the infrastructure used to provide CSP-supplied cloud services to the CSP's customers. The 610-1 endpoints may point to URLs with associated fully qualified domain names (FQDNs) used to access cloud services provided by the infrastructure 601. Examples of these endpoints are:

[0190] "createVM.compute.us-seattle.oci.Oraclecloud.com" - for creating VMs, "deleteVM.compute.us-seattle.oci.Oraclecloud.com" - for deleting a VM, "storeObject.storage.us-seattle.oci.Oraclecloud.com" - for storing data, etc.

[0191] Users associated with the CSP's direct customers 640 and connected to the infrastructure 601 via a communications network may use one or more of the endpoints 610-1.

[0192] In one embodiment, a set of vPLC-specific endpoints is created for each vPLC. For example, as depicted in FIG. 6 , a set of endpoints 612-1 is created for vPLC.R1 and a set of endpoints 614-1 is created for vPLC.R2. Endpoint 612-1 can be used by users associated with customer 642 of reseller R1 to access resources associated with vPLC.R1 and to perform one or more functions or operations involving resources allocated to vPLC.R1. Endpoint 614-1 can be used by users associated with customer 644 of reseller R2 to access resources associated with vPLC.R2 and to perform one or more functions or operations involving resources allocated to vPLC.R2.

[0193] In certain embodiments, a set of vPLC-specific endpoints for a vPLC are created based on endpoints provided by the CSP for its direct customers. For example, in FIG. 6, endpoint 612-1 for vPLC.R1 is created based on endpoint 610-1. Endpoints created for a vPLC may include an identifier that specifically identifies the vPLC for which the endpoint is created. For example, given the exemplary endpoints provided by the CSP for its direct customers, endpoint 612-1 for vPLC.R1 is: "createVM.compute.vplcR1.us-seattle.oci.Oraclecloud.com" - for creating a VM, "deleteVM.compute.vplcR1.us-seattle.oci.Oraclecloud.com" - for deleting a VM, "storeObject.storage.vplcR1.us-seattle.oci.Oraclecloud.com" - for storing data, etc., where "vplcR1" uniquely identifies the vPLC corresponding to the endpoint.

[0194] An endpoint 614-1 for vPLC.R2 may also be created based on endpoint 610-1. For example, endpoint 614-1 for vPLC.R2 may be created based on endpoint 610-1. "createVM.compute.vplcR2.us-seattle.oci.Oraclecloud.com" - for creating VMs, "deleteVM.compute.vplcR2.us-seattle.oci.Oraclecloud.com" - for deleting a VM, "storeObject.storage.vplcR2.us-seattle.oci.Oraclecloud.com" - for storing data, etc., where "vplcR2" uniquely identifies the vPLC corresponding to the endpoint.

[0195] Because the endpoints associated with a vPLC include an identifier that identifies the particular vPLC, when an API call is received by infrastructure 601, the endpoint call can be analyzed to determine whether the call is for a direct customer of the CSP or for a reseller. Furthermore, because the endpoints use unique identifiers for different vPLCs, the particular vPLC and associated reseller can also be identified from the endpoint call. In this manner, based on the results of the analysis of the endpoint call, the endpoint request can be appropriately forwarded for processing.

[0196] The CSP also provides a console 610-2 for use by its direct customers 640. Users associated with the CSP's direct customers 640 can use the console 610-2 to configure, access, and manage the first portion 602 of the CSP-provided region infrastructure. In some embodiments, the console 610-2 can provide a set of web-based graphical user interfaces (GUIs) that can be used to access and manage the CSP-provided cloud infrastructure in the region. In some embodiments, the console is a web-based application.

[0197] In one embodiment, a console is also provided for each vPLC. For example, as depicted in FIG. 6 , console 612-2 is provided for vPLC.R1 and console 614-2 is provided for vPLC.R2. The console 612-2 provided for vPLC.R1 can be used by a user associated with a customer 642 of reseller R1 to configure, access, and manage a second portion of the CSP-provided region infrastructure allocated to vPLC.R1 604 created for reseller R1. The console 614-2 provided for vPLC.R2 can be used by a user associated with a customer 644 of reseller R2 to configure, access, and manage a third portion of the CSP-provided region infrastructure allocated to vPLC.R2 606 created for reseller R2.

[0198] In some embodiments, a reseller may join two or more vPLCs, but each vPLC may still have an endpoint set and a console. For example, if a reseller joins two vPLCs, vPLC1 and vPLC2, vPLC1 may have an endpoint set (e.g., 612-1) and a console (e.g., 612-2), and vPLC2 (e.g., 614-1) may have another endpoint set and a console (e.g., 614-2). Both vPLC1 (e.g., 604) and vPLC2 (606) may be associated with the same reseller.

[0199] The infrastructure 601 may have networking resources that enable communication to and from the infrastructure 601. These network resources may include, for example, a network gateway 628 that enables traffic to and from the CSP provided region infrastructure 601 to be communicated to other destinations, which may be communicatively coupled to the infrastructure 601 via a public (e.g., the Internet) or private network, which may be within an on-premises network, within another cloud network, etc. The gateway 628 may be implemented as a physical network device (e.g., a router), a logical networking component (e.g., a virtual router), or some combination of physical and logical components.

[0200] The infrastructure 601 may include a control plane (CP) 624 and a management plane (MP) 626 provided by the CSP. In some embodiments, the CP 624 may be responsible for providing management, deployment, and orchestration functions within the cloud environment provided by the CSP. For example, the CP may receive requests from the CSP's direct customers, reseller R1's customers, and reseller R2's customers through endpoints and consoles, collaborate with other components in the infrastructure, such as resource managers and management planes, determine whether the requests can be fulfilled, and respond accordingly to the CSP's direct customers, reseller R1's customers, and reseller R2's customers. In other words, the CSP CP 624 may be responsible for performing vPLC-related functions. In some embodiments, the CSP may also provide a vPLC-specific control plane that is specifically configured to perform vPLC operations. In some embodiments, there may be one vPLC control plane that can perform processing for multiple vPLCs. In other embodiments, each vPLC may be provided with its own CP.

[0201] In some embodiments, MP 626 may be a collection of software components responsible for provisioning all the processes and workflows required for a requested operation. For example, for a service request to access a cloud service, the MP may provision a workflow with an identity check on the requestor that checks DNS records to obtain an IP address for accessing the service backend. The MP ensures that various tasks are performed appropriately and in the correct order by the data plane.

[0202] In one embodiment, the computing resources 620 may include one or more racks, each rack including one or more servers, each server hosting one or more operating systems. The division of the computing resources 620 into resources 620a allocated to infrastructure for serving the CSP's direct customers 640 and resources 620b and 620c allocated to vPLCs may occur along various levels or boundaries. The division of computing resources at a particular level may also be referred to herein as a segmentation level. In one embodiment, the division may be performed at the rack level (i.e., segmented at the rack level), with one or more individual racks allocated to 620a, 620b, and 620c. In this embodiment, the rack is allocated exclusively to a particular vPLC and is not used by other vPLCs or by the CSP to provide CSP-delivered services to its direct customers 640.

[0203] In another embodiment, the partitioning may be performed at the server level (i.e., segmentation may be performed at the server level), with one or more individual servers from the same or different racks being allocated to 620a, 620b, and 620c. In this embodiment, the servers are allocated exclusively to a particular vPLC and are not used by other vPLCs or by a CSP to provide CSP-supplied services to its direct customers 640. A rack may have multiple servers, so that one server on the rack may be allocated to 620a, another server to 620b, and a third server to 620c. Thus, a rack may be shared between two vPLCs, or may be shared between a vPLC and infrastructure used by a CSP to provide CSP-supplied services to its direct customers 640.

[0204] In some embodiments, the partitioning may be performed at the hypervisor level (i.e., segmentation may be performed at the hypervisor level), with one or more individual hypervisors from the same or different servers in the same or different racks being allocated to 620a, 620b, and 620c. In this embodiment, the hypervisors are allocated exclusively to a particular vPLC and are not used by other vPLCs or by a CSP to provide CSP-supplied services to its direct customers 640. Because a server can host multiple hypervisors, one hypervisor on a server may be allocated to 620a, another hypervisor on the same server may be allocated to 620b, and a third hypervisor on the same server may be allocated to 620c. Thus, a server may be shared between two vPLCs or may be shared between a vPLC and infrastructure used by a CSP to provide CSP-supplied services to its direct customers 640.

[0205] A hypervisor on a server on a rack can manage and run multiple virtual machines (VMs). In some embodiments, the division may be performed at the VM level (i.e., segmentation may be performed at the VM level), with VMs running by the same hypervisor being allocated to 620a, 620b, and 620c. In this embodiment, a VM is allocated exclusively to a particular vPLC and is not used by other vPLCs or by a CSP to provide CSP-supplied services to its direct customers 640. Because the hypervisor can support multiple VMs, one VM may be allocated to 620a, another VM may be allocated to 620b, and a third VM on the same hypervisor may be allocated to 620c. Thus, a hypervisor may be shared between two vPLCs or may be shared between a vPLC and infrastructure used by a CSP to provide CSP-supplied services to its direct customers 640.

[0206] In still other embodiments, a combination of the above partitioning techniques may be used. Regardless of the partitioning technique used, the partitioning is done in a secure and safe manner that ensures that traffic and storage intended for a particular vPLC for a particular reseller cannot be seen or accessed by other resellers or their customers, or other direct customers of the CSP. Also, traffic and storage intended for a particular customer of a reseller cannot be seen or accessed by other customers of that reseller.

[0207] Figure 7 illustrates the relationships between a CSP, the CSP's direct and reseller customers and their associated users, and individual reseller customers and their associated users, according to one embodiment. As shown in Figure 7, CSP 710 is at the top or root of the hierarchy. CSP 710 can supply one or more cloud services to which the CSP's customers may subscribe. In Figure 7, the customers of CSP 710 are represented by 702. A tenancy, identified by a tenancy ID, is created for each subscribing customer of CSP 710.

[0208] Customers 702 of a CSP 710 may include direct or non-reseller customers and reseller customers. A reseller customer is an entity that subscribes to a vPLC service offered by a CSP, and the vPLC created for the reseller as a result of subscribing to the vPLC service is used to provide reseller-supplied and reseller-branded cloud services to the reseller's customers. A reseller's customers may be different from the CSP's customers. A direct or non-reseller customer of a CSP is an entity that subscribes to one or more CSP-supplied cloud services, but does not utilize the CSP's infrastructure to provide any cloud services of its own to its customers.

[0209] 7, the direct or non-reseller customers of the CSP 710 include customer C1 726 with associated tenancy T.CSP.C1 and customer C2 728 with associated tenancy T.CSP.C2. There may be multiple such direct customers of the CSP 710. One or more users may be each direct customer of the CSP, and the users may use one or more of the CSP-supplied cloud services to which the corresponding direct customer subscribes. For example, a user associated with a customer of the CSP may be an employee or agent of the customer.

[0210] As shown in Figure 7, reseller customers of a CSP 710 include reseller R1 720 with associated tenancy T.CSP.R1 and reseller R2 722 with associated tenancy T.CSP.R2. There may be multiple such reseller customers of a CSP 710. One or more users (not shown in Figure 7) may be each reseller customer of the CSP.

[0211] A reseller may have its own customers who subscribe to one or more reseller-supplied and reseller-branded cloud services, which are delivered using a vPLC created by the CSP for the reseller. In Figure 7, customers of reseller R1 720 are illustrated using reference number 704, and customers of reseller R2 722 are illustrated using reference number 706.

[0212] A reseller's customers may have their own tenancies and associated sets of users. For example, as shown in FIG. 7 , reseller R1 may have multiple customers, including customer R1.C1 740 with tenancy T.R1.C1 and associated users, customer R1.C2 742 with tenancy T.R1.C2 and associated users, and so on. Reseller R1 may supply a set of R1-supplied and R1-branded cloud services, and R1's customers may subscribe to one or more of the R1-supplied services. The set of one or more R1-supplied services to which R1.C1 740 subscribes may be the same as or different from the set of one or more R1-supplied services to which R1.C2 742 subscribes. The cloud services supplied by R1 722 may be different from the cloud services supplied by reseller R2 722 and the cloud services supplied by CSP 710.

[0213] Reseller R2 722 may have multiple customers of its own who subscribe to R2-supplied and R2-branded cloud services provided using a vPLC created for R2. As shown in FIG. 7 , R2 722's customers include customer R2.C1 744 with tenancy T.R2.C1 and associated users, customer R2.C2 746 with tenancy T.R2.C2 and associated users, etc. The cloud services supplied by R2 722 may be different from the cloud services supplied by reseller R1 720 and the cloud services supplied by CSP 710. R2's customers may subscribe to one or more of the R2-supplied services. The set of one or more R2-supplied services to which R2.C1 744 subscribes may be the same as or different from the set of one or more R2-supplied services to which R2.C2 746 subscribes.

[0214] As shown in FIG. 7, there are two levels of tenancies that come into play when a vPLC is used by a reseller to sell reseller-supplied cloud services to the reseller's customers. The first level of tenancies corresponds to tenancies associated with the CSP's 710 customers, including direct or non-reseller customers and reseller customers. These tenancies may also be referred to as CSP customer tenancies (or CoCT). In FIG. 7, the CoCT includes tenancies associated with reseller R1 720, reseller R2 722, direct customer C1 726, and direct customer C2 728. The second level of tenancies includes tenancies associated with the reseller's customers. These tenancies may also be referred to as reseller customer tenancies (or CoRT). In Figure 7, CoRT includes tenancies associated with R1.C1 740, R1.C2 742, R2.C1 744, R2.C2 746, etc.

[0215] 8 is a diagram illustrating the relationship between resources allocated to tenancies for a CSP's direct customers, tenancies for the CSP's resellers, and tenancies for each reseller's customer, according to one embodiment. Referring to FIG. 8, in one embodiment, a CSP-provided infrastructure (e.g., data center) 810 within a region may be divided into several securely isolated portions for the CSP's resellers, such as resources allocated to vPLC.R1 820 created for reseller R1 and associated with reseller R1's tenancy (T.R1), resources allocated to vPLC.R2 822 created for reseller R2 and associated with reseller R2's tenancy (T.R2), etc. Resource partitions may also include securely isolated partitions for the CSP's non-reseller direct customers C1 and C2, such as resources allocated to non-reseller customer CSP.C1's tenancy (T.CSP.C1) 826 and resources allocated to non-reseller customer CSP.C2's tenancy (T.CSP.C2) 828. The partitions of the resource CSP-provided regional infrastructure allocated to reseller R1's tenancy 820, reseller R2's 822, customer CSP.C1's tenancy, and customer CSP.C2's tenancy may be referred to as first-level resource partitions.

[0216] Each first-level resource partition or securely isolated portion of resources associated with a reseller's tenancy (i.e., the R1 tenancy and the R2 tenancy) may be assigned or tagged with a vPLC ID (i.e., resource ID), such as vPLC_ID.vPLC.R1 820 and vPLC_ID.vPLC.R2 822. Each of these partitions for the CSP's non-reseller direct customers may be tagged with a resource ID. Each tenancy of the CSP's non-reseller direct customers can be accessed by their respective users, who can use one or more of the CSP-supplied cloud services to which their corresponding direct customers subscribe.

[0217] Within each first-level resource partition (e.g., or portion) for a reseller, there may be additional second-level resource partitions allocated to the tenancies of the reseller's customers. For example, securely isolated resource partition (vPLC.R1.C1) 840 may be allocated to the tenancy (T.R1.C1) associated with reseller R1's customer C1, and securely isolated resource partition (vPLC.R1.C2) 842 may be allocated to the tenancy (T.R1.C2) associated with reseller R1's customer C2. Isolation partition vPLC.R1.C1 may be tagged with a customer resource ID that identifies the resources allocated to T.R1.C1, and isolation partition vPLC.R1.C2 may be tagged with a customer resource ID that identifies the resources allocated to T.R1.C1.

[0218] Similarly, securely isolated resource partition (vPLC.R2.C1) 844 may be allocated to a tenancy (T.R2.C1) associated with reseller R2's customer C1, and securely isolated resource partition (vPLC.R2.C2) 846 may be allocated to a tenancy (T.R2.C2) associated with reseller R2's customer C2. Isolation partition vPLC.R2.C1 may be tagged with a customer resource ID that identifies the resources allocated to T.R2.C1, and isolation partition vPLC.R2.C2 may be tagged with a customer resource ID that identifies the resources allocated to T.R1.C1. Each tenancy associated with a reseller's customer (e.g., 840 for R1.C1, 842 for R1.C2, 844 for R2.C1, or 846 for R2.C2) can be accessed by one or more users associated with the customer, who can use one or more of the reseller-supplied cloud services to which the corresponding customer subscribes.

[0219] 9A and 9B illustrate a mechanism for storing information within tenancies for customers of individual resellers of a CSP, according to one embodiment. In some embodiments, the vPLC ID and tenancy ID (sometimes referred to as customer tenancy ID or CT ID) for the tenancy associated with the reseller may be associated in two ways: 1) a distributed manner (or record space per vPLC) as shown in FIG. 9A; and 7) a centralized manner (or vPLC ID as the partition key) as shown in FIG. 9B. While FIGS. 9A and 9B illustrate storing information in a table format, in some embodiments, the information may be stored in a different format, such as an array, a key-value store, XML, JSON, etc.

[0220] FIG. 9A illustrates the first approach, i.e., per-vPLC record space. In some embodiments, per-vPLC record space can be an approach of storing tenancy-related information in a vPLC-specific table based on each reseller's vPLC ID. In other words, the information can be stored in a distributed manner. Each table is assigned a vPLC ID. Within each table, one row can be designated for one customer of the reseller, another row can be designated for another customer of the reseller, and so on. For example, in FIG. 9A , there are two tables for storing information such as identity information (e.g., user logins, associated authentication information, and certificates) or billing information (e.g., resource usage, pricing). Table 910 is assigned a vPLC ID (vPLC_ID.vPLC.R1) for vPLC.R1 associated with reseller R1, and table 920 is assigned a vPLC ID (vPLC_ID.vPLC.R2) for vPLC.R2 associated with reseller R2. Within table 910, there are three records for three customers C1, C2, and C3 of reseller R1. The first record / row may include a tenancy ID (T_ID.T.R1.C1) for the tenancy associated with reseller R1's customer C1 and associated information for customer C1, e.g., identity, billing, etc. The second record / row may include a tenancy ID (T_ID.T.R1.C2) for the tenancy associated with reseller R1's customer C2 and associated information for customer C2. The third record / row may include a tenancy ID (T_ID.T.R1.C3) for the tenancy associated with reseller R1's customer C3 and associated information for customer C3.

[0221] The same structure may be used for table 920, storing three records for reseller R2's three customers C1, C2, and C3. The first record / row may include a tenancy ID (T_ID.T.R2.C1) for the tenancy associated with reseller R2's customer C1 and associated information for customer C1, e.g., identity, billing, etc. The second record / row may include a tenancy ID (T_ID.T.R2.C2) for the tenancy associated with reseller R2's customer C2 and associated information for customer C2. The third record / row may include a tenancy ID (T_ID.T.R2.C3) for the tenancy associated with reseller R2's customer C3 and associated information for customer C3.

[0222] FIG. 9B shows a second approach using the vPLC ID as a partition key. In FIG. 9B, the second approach stores information in a table with both the customer tenancy ID and the vPLC ID as two table columns for searching by using the vPLC ID as the partition key. In other words, information can be stored within a table in a centralized manner. For example, in FIG. 9B, reseller R1, who is associated with a vPLC ID (vPLC_ID.vPLC.R1), has three customers C1, C2, and C3, who are associated with tenancy IDs T_ID.T.R1.C1, T_ID.T.R1.C2, and T_ID.T.R1.C3, respectively, that identify the tenancies for these three customers. The first three rows of table 930, based on the vPLC ID column, can then be marked as vPLC_ID.vPLC.R1, which is the vPLC ID for vPLC.R1 created for reseller R1. The customer tenancy ID column of the first row can be marked as the tenancy ID for the tenancy associated with reseller R1's customer C1 (T_ID.T.R1.C1), while the tenancy ID column of the second row can be marked as the tenancy ID for the tenancy associated with reseller R1's customer C2 (T_ID.T.R1.C2), and the tenancy ID column of the third row can be marked as the tenancy ID for the tenancy associated with reseller R1's customer C3 (T_ID.T.R1.C3). The information stored in the first three rows belongs to reseller R1's customers C1, C2, and C3.

[0223] The same structure may also be used to store records for customers C1, C2, and C3 of reseller R2, e.g., in rows 4, 5, and 6, within the same table 930. For example, the last three rows of table 930 based on the vPLC ID column may be marked as vPLC_ID.vPLC.R2, which is the vPLC ID for vPLC.R2 created for reseller R2. The customer tenancy ID column of the fourth row can be marked as the tenancy ID for the tenancy associated with reseller R2's customer C1 (T_ID.T.R2.C1), while the tenancy ID column of the fifth row can be marked as the tenancy ID for the tenancy associated with reseller R2's customer C2 (T_ID.T.R2.C2), and the tenancy ID column of the sixth row can be marked as the tenancy ID for the tenancy associated with reseller R2's customer C3 (T_ID.T.R2.C3). The information stored in the last three rows belongs to reseller R2's customers C1, C2, and C3.

[0224] FIG. 10 is a flow diagram illustrating an example of a vPLC setup process, according to an embodiment. The process illustrated in FIG. 10 may be implemented in software (e.g., code, instructions, programs) executed by one or more processing units (e.g., processors, cores) of the respective systems, using hardware, or a combination thereof. The software may be stored on a non-transitory storage medium (e.g., on a memory device). The method presented in FIG. 10 and described below is intended to be exemplary and non-limiting. While FIG. 10 depicts various processing steps occurring in a particular sequence or order, this is not intended to be limiting. In an alternative embodiment, the process may be performed in some different order, or some steps may also be performed in parallel. It should be appreciated that in alternative embodiments, the process illustrated in FIG. 10 may include more or fewer steps than depicted in FIG. 10.

[0225] 10 may be performed by a CSP. Some steps may be performed by components within the CSP-provided region infrastructure (e.g., a control plane (CP), a resource manager (RM), and a management plane (MP), etc.) or one or more CSP-provided cloud services, e.g., an endpoint management service (EMS), an identity management service, etc.

[0226] As previously described, a vPLC may be created when a reseller subscribes to a CSP-provided vPLC service. In step 1002, a signal to create a vPLC may be received. In one embodiment, the signal may be a request received when a reseller subscribes to a CSP-provided vPLC service. The request may be an agreement between the CSP and the reseller that lists the virtual model (e.g., nested or segmented model), database, resource placement configuration, customization details, etc. In other embodiments, a signal may be received from some component indicating that a vPLC is being created.

[0227] In step 1004, configuration information for the vPLC to be created may be received. In some embodiments, the configuration information may include tenancy information (e.g., a tenancy identifier) ​​for which the vPLC is being created and information to identify the realm to be associated with the vPLC. As described above, a tenancy is created for the reseller, and a tenancy identifier that uniquely identifies the tenancy is also created for the reseller to be associated with the vPLC.

[0228] As previously mentioned, in use cases where a vPLC may be used as a virtual data center and associated with a different realm than the realm associated with the cloud infrastructure in the region provided by the CSP, identity and trust profile information (e.g., passwords, credentials, etc.) of the different realm needs to be associated with the vPLC. Association of identity and trust profile information of the different realm may be achieved by creating a mapping (or "shadow copy") of the identity and trust profile information in the vPLC. Further details are discussed in FIG. 18.

[0229] Step 1006, covering steps 1008-1018, may implement a process for creating a vPLC for a reseller. In step 1008, a vPLC identifier may be created for the vPLC. A vPLC created for a reseller is treated as a resource and assigned a unique resource identifier, referred to as a vPLC identifier (vPLC ID), which is associated with the reseller's tenancy.

[0230] In step 1010, a set of resources from the CSP-provided infrastructure in the region may be allocated to the vPLC. The set of resources allocated to the vPLC may be selected based on an agreement between the reseller and the CSP. For example, reseller R may subscribe to one or more cloud services offered by the CSP, such as compute services and object storage services. Thus, the set of resources may include compute resources and object storage resources.

[0231] In step 1012, a namespace may be reserved for vPLC. A CSP's resource allocation service (or resource manager (RM)) may create and reserve a namespace for grouping multiple different types of resources that can support vPLC. A namespace may refer to a logical grouping or virtual boundary that aids in the organization and management of resources and prevents naming conflicts when multiple instances of similar resources are created. Thus, a namespace provides a structured way to organize and group related resources. In some embodiments, namespace 0 is reserved for CSPs. Namespaces 1 and beyond are used for vPLC. In one embodiment, a range of Internet Protocol (IP) addresses may be reserved and associated with a namespace for vPLC for traffic separation purposes.

[0232] In step 1014, a console may be created for the vPLC that allows reseller R, the reseller's customers, and users associated with the reseller's customers to configure, access, and manage resources deployed within the vPLC and visualize, access, and interact with reseller-supplied cloud services. Additionally, reseller R may configure the console to provide different customized experiences to different customers of the reseller.

[0233] In step 1016, a set of endpoints for the vPLC may be created. In some embodiments, some of the set of endpoints may be default endpoints (e.g., identity endpoints) that are always available for use, and some endpoints are determined based on which cloud services the reseller subscribes to. For example, the reseller may subscribe to one or more CSP-supplied cloud services, such as compute, storage, virtual cloud network (VCN), database, etc. As a result, in addition to a default identity endpoint for accessing identity services, endpoints for accessing subscribed services may also be created.

[0234] In step 1018, vPLC-related information, such as the vPLC ID, the reseller's identity information, and other information facilitating the reseller-supplied cloud service, may be stored in a namespace for the vPLC. In step 1020, the CSP may send a response and other information to the reseller R indicating the created vPLC. The reseller can begin managing the vPLC for the reseller's customer, such as creating a customer account, configuring the reseller-supplied cloud service, etc.

[0235] Figure 11 is a block diagram of a distributed environment illustrating an example of a virtual private label cloud (vPLC) hosted by a CSP-provided infrastructure within a region, according to an embodiment. Figure 11 may be similar to Figure 6, and both figures include securely isolated portions (or partitions) of the CSP-provided region infrastructure for the CSP's direct customers (e.g., first portion 602, second portion 604, and third portion 606 of Figure 6), vPLC.R1 associated with reseller R1, and vPLC.R2 associated with reseller R2. The CSP's direct customers, customers of reseller R1, and customers of reseller R2 can access their respective partitions through one or more endpoints and consoles. The CSP-provided region infrastructure 602 may also have a control plane, management plane, and gateways, as described in Figure 6.

[0236] However, Figure 11 shows a more detailed view of the interconnections between resources within the CSP-provided region infrastructure, such as compute, storage, gateways, CPs, and the management plane. Additionally, the CSP's direct non-reseller customers, customers of reseller R1, and customers of reseller R2 may access the CSP-provided region infrastructure from either the Internet or a corporate network. For example, a gateway may be used to access the Internet or a corporate network from a Real Application Cluster (RAC) within the CSP-provided region infrastructure.

[0237] FIG. 11 shows three cloud partitions running on the same CSP-provided region infrastructure 1102: the CSP's native cloud 1120 and two vPLCs for resellers R1 and R2: vPLC.R1 1122 and vPLC.R2 1124. The CSP-provided region infrastructure 1102 is operated by the CSP. The CSP-provided region infrastructure 1102 may also include a control plane 1104 and a management plane 1105. As mentioned above, the CP 1104 may be responsible for providing management, deployment, and orchestration functions with the cloud environment provided by the CSP. The MP 1105 may be a collection of software components responsible for provisioning all processes and workflows required for requested operation. Additionally, software implementing cloud services may originate from a deployment system 1106 and enter data centers within the region through the management plane 1105. Software updates (e.g., new services) for the control plane 1104 can be delivered by a deployment system 1106 through the management plane 1105. A CSP administrator or reseller operator 1107 deploying the software / services can also access the CSP-provided region infrastructure through the management plane 1105.

[0238] Computing resources, namely, three Real Application Clusters (RACs) 1140, 1142, and 1144, that share the same storage 1150 are partitioned and allocated to the CSP's native cloud 1120, vPLC.R1 1122, and vPLC.R2 1124, respectively. For example, RAC 1140 is dedicated to the CSP's direct customers, RAC 1142 is for vPLC.R1, and RAC 1144 is for vPLC.R2.

[0239] Network resources may also be divided or shared among the CSP's direct customers and resellers, allowing their computing resources to connect to the Internet or corporate networks. For example, each RAC 1140, 1142, and 1144 uses its network gateways 1160, 1162, and 1164 to access the Internet 1108 or corporate network 1109. In some embodiments, two RACs may share the same network interface. For example, RAC 1142 and RAC 1144 may share the same gateway 1162 to access the Internet 1108.

[0240] The storage in Figure 11 is simplified. Storage 1150 may include multiple different types of storage, such as global storage, dedicated storage, and block storage and object storage. In Figure 11, vPLC.R1 and vPLC.R2 have their dedicated storage 1152 and 1154, respectively. Global storage 1150 belongs to the CSP.

[0241] In FIG. 11 , a CSP's native cloud 1120 can provide CSP-supplied cloud services to the CSP's direct non-reseller customers (e.g., CSP.C1-n), such as corporations. Users associated with the reseller's customers can access the services through dedicated service endpoints that are part of their respective namespaces, including fully qualified domain names (FQDNs), IP addresses, console experiences, identities, etc. The CSP can provide individual consoles for each reseller, and the consoles provide the services through the endpoints. The endpoints associated with the consoles can differ based on which services are provided to that reseller's customers based on the reseller's subscription. For example, an individual reseller's customer may submit an access request through a dedicated endpoint or via the console.

[0242] Users can use service endpoints 1170 and 1172 exposed by the CSP (two endpoints are shown, but more are possible), which have DNS names. The native cloud 1120 may also include a console 1174 with a look and feel. The CSP's direct customers on the Internet 1108 can access the CSP-provided regional infrastructure 1102 through public endpoints 1170 and 1172 to request instance launch or termination. Reseller R1's customers on the Internet 1108 can also access the CSP-provided regional infrastructure 1102 through public endpoints 1180 and 1182 to request instance launch or termination. Reseller R2's customers on the corporate network 1109 can access the CSP-provided regional infrastructure 1102 through private endpoints 1190 and 1192 to request instance launch or termination. The control plane 1104 can then launch instances, terminate instances, configure connectivity for those instances, and orchestrate services within the data center. Network interfaces 1160 and 1162 (e.g., Internet Gateways (IGW)) for CSP and vPLC.R1, respectively, can provide network interconnection to the Internet 1108. Network interface 1164 (e.g., Dynamic Routing Gateway (DRG)) for vPLC.R2 can provide network interconnection to the enterprise network 1109.

[0243] In FIG. 11 , vPLC.R1 1122 and vPLC.R2 1124 share the same physical infrastructure. Their endpoints are specific to each reseller, such as endpoints 1180 and 1182 for reseller R1 and endpoints 1190 and 1192 for reseller R2. Consoles 1184 and 1194 are customized for reseller R1 and reseller R2, respectively. The vPLC endpoints and consoles can connect to the Internet or their corporate networks depending on their needs. For example, in FIG. 11 , vPLC.R1 connects to the Internet 1108 through a network gateway 1162 (e.g., IGW). However, vPLC.R2 does not connect to the Internet and therefore instead connects to the corporate network 1109 through a secured network 1164 (e.g., FastConnect). 11, gateway 1162 can be sliced ​​and shared by vPLC.R1 and vPLC.R2 and can be part of an orchestration performed by a CSP. In this example, some of vPLC.R2's data can go to the Internet 1108 and some can go to its corporate network 1109.

[0244] If the request comes in through console 1194, it can go to control plane 604. The control plane can determine that the request came from vPLC.R2 1124 and can find and provision an available resource (e.g., a computer) from the RAC specified for vPLC.R2 (i.e., 1144 in this case), then launch this resource. Control plane 604 can then respond to reseller R2's customer through console 1194 and provide an IP address for their network configuration (e.g., VCN). Control plane 1104 can orchestrate gateway 1164 depending on how it is configured toward either enterprise network 1109 or the Internet 1108. The control plane can recognize not only the request, but also the source of the request, whether it is a direct customer of the CSP, a customer of reseller R1, or a customer of reseller R2.

[0245] As mentioned above, the division of the CSP-provided region infrastructure can be implemented for different vPLCs at various predetermined infrastructure levels, such as the rack level, the server level, the hypervisor level, or the VM level. To further illustrate, for division at the rack level, a vPLC may have its own dedicated rack and resources therein. For division at the server level, multiple vPLCs may share a rack, but each vPLC has its dedicated server. In yet another example, for division at the hypervisor level, multiple vPLCs may share a server, but each vPLC has its dedicated hypervisor. For division at the VM level, multiple vPLCs may share a hypervisor, but each vPLC has its dedicated VM. Therefore, to achieve a multi-vPLC approach, there may be two main vPLC infrastructure division models (or opposite ends of a spectrum): a segmented model and a nested model. The segmented model may refer to resource division at the rack level. The nested model may refer to resource division at the VM level. However, other variations between them are possible, taking into account operational efficiency (e.g., control plane sharing) and fragmentation (i.e., resource division).

[0246] Figure 12 is a block diagram illustrating a vPLC segmentation model for resource partitioning, according to an embodiment. In this model, which represents one end of the partitioning spectrum, each vPLC may have its dedicated resources, including compute and network resources, at the rack level. In other words, a rack may be allocated exclusively to a particular vPLC and may not be used by other vPLCs or by a CSP to provide CSP-supplied services to its direct customers. The segmentation model implements resource and usage tracking at the physical resource level based on affinity tagging (e.g., vPLC ID and customer tenancy ID). There may be a default behavior for each segment, but each segment can be customized by the reseller associated with the vPLC in that segment.

[0247] In FIG. 12 , multiple vPLCs (e.g., vPLC.R1 1210, vPLC.R2 1220, and vPLC.Rn 1230) may exist within a CSP-provided infrastructure (e.g., a data center) within a region. In an embodiment, in a segmentation model, there may be physical and logical partitions, as shown in FIG. 12 . For example, for a physical partition, a vPLC is assigned its dedicated hardware resources, such as, for example, a rack containing one or more servers, with each server hosting one or more operating systems. In FIG. 12 , a first rack is assigned to the vPLC.R1 segment 1210, a second rack is assigned to the vPLC.R2 segment 1220, and a third rack is assigned to the vPLC.Rn segment 1230. Each vPLC may also have dedicated networking switches (also referred to as gateways), such as, for example, gateway 1219 for vPLC.R1 1210, gateway 1229 for vPLC.R2 1220, and gateway 1239 for vPLC.Rn 1230, that connect to the Internet / enterprise network 1260. Thus, in Figure 12, vPLC.R1 1210 has dedicated servers 1214a-b, a hypervisor 1216 running on server 1214b and hosting VMs 1218a-n, and a gateway (e.g., IGW) 1219. Similarly, vPLC.R2 1220 has dedicated servers 1224a-b, a hypervisor 1226 running on server 1224b and hosting VMs 1228a-n, and a gateway 1229. The vPLC.Rn 1230 includes dedicated servers 1234a-b, a hypervisor 1236 running on server 1234b and hosting VMs 1238a-n, and a gateway 1239.

[0248] For logical partitions, there is a common control plane 1250 and resource manager 1252 managed and operated by the CSP for all vPLCs. However, the data plane may be dedicated to each vPLC in a segmented model. For example, each vPLC.R1, vPLC.R2, and vPLC.Rn has its own data plane 1212, 1222, and 1232, respectively. In some embodiments, the resource manager 1252 may be part of the management plane. In other embodiments, the resource manager 1252 may be separate from the management plane.

[0249] In some embodiments, a user associated with a customer of reseller R1 1280 can send a request to launch a VM instance or a bare metal server through one of vPLC.R1 endpoints 1270. Assume the request is to launch a new VM instance. In that case, control plane 1250 can work with an identity service to check the appropriate credentials of the user of vPLC.R1, after which resource manager 1252 can identify all resources dedicated to vPLC.R1, such as an IP address range. A VM instance (e.g., 1218a) is then launched on hypervisor 1216 within vPLC.R1 segment 1210. A response is then sent by control plane 1250 back to the requesting user through vPLC.R1 endpoint 1270. If the request is for a bare metal server (e.g., 1214a), resources for vPLC.R1 can be provisioned with all necessary configurations and allocated to vPLC.R1 by resource manager 1252. Similar processes may be performed for requests from users associated with customers of reseller R2 1282, which are to be satisfied by vPLC.R2 segment 1220, and for requests from users associated with customers of reseller Rn 1284, which are to be satisfied by vPLC.Rn segment 1230. Each process for a vPLC may be performed independently of and concurrently with another vPLC.

[0250] The vPLC segmentation model may have several advantages, such as physical separation and sharing of a common control plane 1250 operated by the CSP. Additionally, a resource manager 1252 manages resources across the vPLCs 1210-1230. However, resources, including networking and storage, are duplicated to be designated for each vPLC, and therefore scaling challenges may be present. Small footprints, such as a single rack or server, are also difficult to achieve under this model. However, the vPLC segmentation model may still share power, cooling, etc. As mentioned above, the segmentation model is only one end of the spectrum, and there are various ways to partition based on resources and efficiency.

[0251] FIG. 13 is a block diagram illustrating a vPLC nested model for resource partitioning, according to an embodiment. The nested model of FIG. 13 illustrates the other end of the spectrum of resource partitioning at the VM level. In other words, a VM may be allocated exclusively to a particular vPLC and may not be used by other vPLCs or by a CSP to provide CSP-supplied services to its direct customers. However, a hypervisor, server, or rack may be shared between a CSP's customers and one or more vPLC customers. In some embodiments, the nested model may have a base layer, such as a hypervisor, as a shared foundation that is not visible to the customer; abstraction layers, such as VMs, may be built on top of the base layer, allowing each reseller to clone and customize the abstraction layers for their customers.

[0252] In the embodiment of FIG. 13 , all vPLCs (e.g., vPLC.R1, vPLC.R2, and vPLC.Rn) are hosted on a single segment with a shared infrastructure, shown as a “server CSP” 1320 (e.g., a bare-metal server) operated by a CSP, which may have control plane functionality. Server CSP 1320 may be configured to host a hypervisor 1330 that manages virtual machines (VMs) 1332a-n running on the hypervisor. In some embodiments, a segment may comprise multiple racks. Here, two VM instances for vPLC.R1 1332a-b, one VM instance for vPLC.R2 1332c, and one VM instance for vPLC.Rn 1332n reside on the shared hypervisor 1330. All vPLCs share the same data plane 1312 within segment 1310. In some embodiments, a segment may comprise multiple racks. Because the hypervisor, servers, and racks may be shared by multiple vPLCs, the servers for these vPLCs may be from different racks.

[0253] 13 further illustrates a variation of the split scenario, where a customer of reseller R2 may request an additional bare metal server, server vPLC.R2 1322, provisioned as a dedicated server. In the vPLC nesting model, a shared server pool can be located wherever space is available. This therefore provides a lower cost point for the reseller.

[0254] In FIG. 13 , a gateway 1340 (e.g., IGW or DRG) shared by the vPLCs has connectivity to the Internet or the corporate network, as needed. Incoming packets from the Internet / corporate network 1360 may be tagged (i.e., encapsulated in each packet header) with vPLC-related information (e.g., vPLC ID and customer tenancy ID) at the gateway 1340 for tracking purposes to the appropriate vPLC. Packet encapsulation helps separate traffic within a single segment 1310 shared by multiple vPLCs. As an example, a packet destined for VM 1332a belonging to a tenancy associated with customer C1 of reseller R1 associated with vPLC.R1 may be encapsulated in the packet header with vPLC_ID.vPLC.R1 and customer tenancy ID (T_ID.R1.C1). The packet can then be routed to VMs 1332a-b separated for vPLC.R1 based on the vPLC ID (vPLC_ID.vPLC.R1). A customer tenancy ID (T_ID.R1.C1) associated with reseller R1's customer C1 in the packet header can further assist in routing the packet to tenant C1 running on VM 1332a. The packet is decapsulated before being delivered to VM 1332a.

[0255] Similarly, a packet destined for VM 1332c belonging to a tenancy associated with customer C1 of reseller R2, which is associated with vPLC.R2, may be encapsulated in the packet header with vPLC_ID.vPLC.R2 and a customer tenancy ID (T_ID.R2.C1). The packet can then be routed to resources carved out for vPLC.R2 based on the vPLC ID (vPLC_ID.vPLC.R2). The customer tenancy ID (T_ID.R2.C1) associated with customer C1 of reseller R2 in the packet header can further assist in routing the packet to tenant C1 running on VM 1332c in vPLC.R2. The packet is decapsulated before being delivered to VM 1332c.

[0256] Outgoing packets can be performed in the reverse process. For example, an outgoing packet originating from VM 1332a in vPLC.R1 and destined for the Internet 1360 may be encapsulated in a packet header with vPLC_ID.vPLC.R1 and a customer tenancy ID (T_ID.R1.C1) associated with reseller R1's customer C1. When the packet reaches gateway 1340, the packet may be decapsulated before heading to the Internet 1360. Similarly, an outgoing packet originating from VM 1332c in vPLC.R2 and destined for the Internet 1360 may be encapsulated in a packet header with vPLC_ID.vPLC.R2 and a customer tenancy ID (T_ID.R2.C1) associated with reseller R2's customer C1. When the packet reaches gateway 1340, the packet may be decapsulated before heading to the Internet 1360. As a result, both incoming and outgoing traffic is segregated within a single segment 1310 shared by multiple vPLCs.

[0257] In some embodiments, in a nested model, one or more VMs may belong to a direct customer of the CSP and may share the same hypervisor with other vPLCs.

[0258] Reseller users accessing the vPLC infrastructure are authenticated in both segmented and nested models. A user associated with a particular reseller's customer can access an endpoint and be authenticated as a user of the vPLC associated with the particular reseller by using the user's credentials, such as a username and customer tenancy ID. For example, in some embodiments, in FIG. 13 , a user associated with reseller R1's customer C1 can send a request to launch a VM instance through one of the vPLC.R1 endpoints 1302. The particular vPLC.R1 endpoint 1302 can augment the request with the vPLC ID for vPLC.R1. The control plane 1350 can cooperate with the identity service to check the appropriate credentials for the user of vPLC.R1 (e.g., the user's username and customer C1's tenancy ID). Resource manager 1352 can then identify all resources allocated to vPLC.R1, such as server 1320 and hypervisor 1330 based on the vPLC ID, and VM 1332a based on the customer tenancy ID of customer C1. VM instance 1332a is then launched on hypervisor 1330 accordingly.

[0259] 14 is a flow diagram illustrating a sign-up process for a customer of a reseller associated with a vPLC, according to one embodiment. When a direct customer of a CSP subscribes or registers (i.e., signs up) for a cloud service offered by the CSP (i.e., a CSP-provided cloud service), a tenancy or account is created for that customer. Similarly, when a customer of an individual reseller subscribes or registers for a cloud service offered by that reseller (i.e., a reseller-provided cloud service), a tenancy or account is created for that customer.

[0260] FIG. 14 illustrates the sign-up process for a particular customer C1 of reseller R1. However, the sign-up process is applicable to customers of individual resellers. In FIG. 14, in step S1, reseller R1's customer C1 1401 can sign up to create a new account by entering user identification credentials, such as a username or account name (e.g., John), using a sign-up portal on the console of vPLC.R1 1402. In step S2, vPLC1.R1 console 1402 can augment the sign-up request by adding a vPLC ID (e.g., vPLC_ID.vPLC.R1) that identifies the vPLC being created for reseller R1 to the user's credentials and forward the sign-up request to CSP (control plane (CP) or management plane (MP)) 1404. In some embodiments, the creation of reseller R1's customer C1's account can be handled by reseller R1 with assistance from the CSP's CP or MP.

[0261] In step S3, the CSP then creates a customer tenancy (or account, T.R1.C1) for reseller R1's customer C1 1401. A tenancy identifier (T_ID.T.R1.C1) is assigned to the newly created tenancy (T.R1.C1) of customer C1. A vPLC ID (e.g., vPLC_ID.vPLC.R1) identifying vPLC.R1 for reseller R1 may be associated with customer C1's newly created customer tenancy ID (e.g., T_ID.T.R1.C1), shown at 1430, and may be stored in database 1420 along with other authentication information (e.g., account name and password). Other customers of reseller R1 and reseller R2 may also be stored in database 1420. For example, the customer tenancy ID of reseller R1's customer C2 (i.e., R1.C2) is associated with the vPLC ID of vPLC.R1, as shown in 1432 as T_ID.T.R1.C2 and vPLC_ID.vPLC.R1. The customer tenancy ID of reseller R2's customer C2 (i.e., R2.C1) is associated with the vPLC ID of vPLC.R2, as shown in 1434 as T_ID.T.R2.C1 and vPLC_ID.vPLC.R2.

[0262] The association between the vPLC ID and the tenancy IDs of individual resellers' customers can provide isolation between customer tenancies within a vPLC. For example, customer tenancy R1.C1 can be securely isolated from customer tenancy R1.C2 by using different customer tenancy IDs, i.e., T_ID.T.R1.C1 for customer C1 and T_ID.T.R1.C2 for customer C2, even though both customer tenancies are within the same vPLC.R1. The tenancies associated with reseller R1 and reseller R2 can be securely isolated using their respective vPLC IDs, i.e., vPLC_ID.vPLC.R1 for reseller R1 and vPLC_ID.vPLC.R2 for reseller R2.

[0263] In step S4, CSP 1404 can return a handle including the customer tenancy ID (e.g., T_ID.T.R1.C1), a temporary default password, and an account name to vPLC.R1 console 1402. In step S5, vPLC.R1 console can receive the temporary authentication information and forward the customer tenancy ID and password to reseller R1's customer C1 1401. As shown in FIG. 14 , reseller R1's customer C1 1401 can interact directly with reseller R1 through vPLC.R1 console 1402 to sign up and subscribe to one or more reseller-supplied cloud services. Similarly, the CSP's customers can interact separately with the CSP to sign up and subscribe to CSP-supplied cloud services at the CSP's console (not shown).

[0264] As described above in connection with tenancy creation, the CSP's database (e.g., 1420 of FIG. 14) can store vPLC IDs, customer tenancy IDs, and their corresponding authentication information. In an embodiment, multiple customer tenancy IDs may be associated with the same vPLC ID, as shown at 1430 (T_ID.T.R1.C1 is associated with vPLC_ID.vPLC.R1) and 1432 (T_ID.T.R1.C2 is associated with vPLC_ID.vPLC.R1). Tenancies may be defined at the realm level, and thus, a tenancy may have a realm identifier associated with it.

[0265] 15 is a flow diagram illustrating a login process for a user associated with a customer of a reseller associated with a vPLC, according to one embodiment. In FIG. 15, steps S1 and S2 comprise the login phase after an individual reseller customer completes the sign-up process described in FIG. 14.

[0266] In step S1, a user 1501 associated with a customer of reseller R1 can log in using a customer tenancy ID (or CT ID), username, and password at the vPLC.R1 endpoint 1510. At S2, the user 1501 and their customer tenancy can be verified at the vPLC.R1 endpoint 1510 by going through a verification process, which may include, but is not limited to, user authentication (e.g., verifying valid credentials such as username and password), session management (e.g., session tokens or timeouts), and access control enforcement (based on access rights, permissions, and policies).

[0267] In step S3, the vPLC.R1 endpoint 1510 can return a session handle. The session handle may refer to a token or unique identifier returned to the customer after successfully logging into their account. The session handle can be used for subsequent requests to access cloud services and resources (e.g., launch an instance) without having to re-authenticate for each individual request during the active login session (or API session).

[0268] Cloud resource IDs (CIDs), which include a customer tenancy ID, may be generated for resources when those resources are created. Virtual machines, accounts, tenancies, authentication policies, virtual network subnets, ACLs, and load balancers may be examples of cloud resources and therefore may be referenced by their respective CIDs.

[0269] 16 is a flow diagram illustrating a resource allocation and provisioning process for a reseller's customer associated with a vPLC, according to one embodiment. A resource manager (RM) manages the resources of the CSP-provided cloud infrastructure within a region and the resource partitions assigned to each vPLC.

[0270] As described above, a vPLC-specific endpoint may be used by a user associated with a customer of a reseller to access resources associated with the vPLC and to perform one or more functions or operations involving resources allocated to the vPLC. In Figure 16, in step S1, a user 1601 associated with customer C1 of reseller R1 may issue a request to launch a compute instance, such as a VM instance with 1 GB RAM, through a vPLC-specific endpoint, i.e., vPLC.R1 endpoint 1602.

[0271] The vPLC.R1 endpoint 1602, in step S2, can augment (or annotate) the request to indicate that the request is coming from a customer of reseller R1 by tagging the request with the vPLC.R1 endpoint's corresponding vPLC ID (e.g., vPLC_ID.vPLC.R1) and forwards the request to resource manager (RM) 1604. In other words, the annotated information (e.g., vPLC ID) enables the authorization, once the request is authenticated, to be processed using all policies associated with vPLC.R1. In some embodiments, the policies associated with the vPLC may include, for example, the type of resource, the amount of resource allowed for allocation (i.e., capacity limits), permissions, etc.

[0272] In step S3, RM 1604 queries its database (i.e., resource database) 1606 to determine whether the requested resources are available to vPLC.R1 based on the policy. Based on the query, in step S4, RM can perform resource allocation after confirming that the requested resources are available based on the policy. In other words, RM allocates a portion of the resources already allocated to vPLC.R1 of reseller R1 to a customer tenancy (T.R1.C1) associated with reseller R1's customer C1 (based on the customer tenancy ID when the user logs in). This resource allocation may be performed by allocating or assigning a portion of the available cloud resources (already allocated to vPLC.R1) to a user associated with customer C1 based on policies agreed upon between customer C1 and reseller R1, such as capacity limits or permissions. For example, reseller R1 and its customer C1 may have a contract policy to allow a maximum of 100 GB of available memory to be used. Currently, 80 GB is allocated (or used). If a user associated with customer C1 requests 10 GB, the RM can identify the CID to have the 10 GB provisioned by the CP. If a user associated with customer C1 requests 30 GB, which would exceed the total 100 GB limit, the request may be denied.

[0273] In step S5, the RM 1604 obtains from the resource database the cloud resource ID assigned to the resource (vPLC.R1.C1) allocated to the tenancy (T.R1.C1) of the customer C1 of the reseller R1. In step S6, the RM passes the resource ID to the vPLC.R1 endpoint 1602. In step S7, the vPLC.R1 endpoint 1602 makes a resource request to the control plane (CP) 1608 for a resource having a particular resource ID assigned to the resource allocated to the tenancy (T.R1.C1) of the customer C1 of the reseller R1. In other words, the resource manager 1604 can augment the user's request (e.g., step S6) with information related to the vPLC (e.g., the resource ID assigned to vPLC.R1.C1), which can be used to provision the resource by the control plane 1608 (e.g., step S8).

[0274] In step S8, the control plane can provision the requested resources (e.g., launch a specific VM instance and storage). The resources may include compute, networking, storage, etc. In step S9, the control plane 1608 can respond to the vPLC.R1 endpoint 1602 with a status and a resource handle. The resource handle may refer to a unique resource identifier assigned to the provisioned resource, such as a cloud resource ID assigned to the provisioned compute instance (e.g., VM), storage volume, virtual cloud network (VCN or subnet), database, etc. The resource handle can enable the customer to access and manage the provisioned resources through APIs or command line tools.

[0275] In step S10, the status and resource handle are forwarded to user 1601 associated with customer C1 of reseller R1. In step S11, the control plane may notify the RM regarding the resource provision status, e.g., success or failure. In step S12, the RM may update a record in resource DB 1606 that tracks resource usage for customer C1 associated with the user. Continuing with the above example, the record may be updated to indicate that the total amount of memory used by customer C1 is 90 GB after successfully provisioning 10 GB in addition to the previously provisioned 80 GB, or that it remains at 80 GB after the user's request was denied. Records maintained by the resource DB may aid in the enforcement of policies associated with the vPLC.

[0276] Figure 17 is a flow diagram illustrating a resource allocation and provisioning process for a reseller's customer associated with a vPLC, according to an embodiment. Figure 17 is an alternative embodiment to Figure 16. Instead of using a resource manager (RM) as a central role for communicating with all subsystems and receiving requests to coordinate the resource allocation and provisioning process, a control plane (CP) may also be used to play such a central role.

[0277] For example, in FIG. 17 , in step S1, a user 1701 associated with a customer C1 of reseller R1 can issue a request to a vPLC-specific endpoint, i.e., vPLC.R1 endpoint 1702. The vPLC.R1 endpoint 1702 can augment the request to indicate that the request is coming from a customer of reseller R1 by tagging it with a corresponding vPLC ID in step S2 and forwards the request to a control plane (CP) 1704. In step S3, the CP 1704 forwards the request to a resource manager (RM) 1706. In step S4, the RM 1706 queries its database (i.e., resource database) 1708 to determine whether the requested resources are available for vPLC.R1. Based on the query, in step S5, the RM performs resource allocation if the requested resources are available.

[0278] In step S6, the RM obtains from the resource database the resource ID assigned to the resource (vPLC.R1.C1) allocated to the tenancy (T.R1.C1) of reseller R1's customer C1. In step S7, the RM passes the resource ID to CP 1704. In step S8, the control plane can provision the requested resource. In step S9, the control plane 1704 can respond to vPLC.R1 endpoint 1702 with the status and resource handle. In step S10, the status and resource handle can be forwarded to user 1701. In step S11, the control plane can notify the RM regarding the resource provision status. In step S12, the RM can update a record in the resource DB that tracks resource usage of the customer associated with the user.

[0279] 17, the CP 1704 can play a central role by forwarding the resource request to the RM 1706 (e.g., step S3), receiving a resource ID from the RM (e.g., step S6), and provisioning the requested resources (e.g., step S8). Compared to FIG. 16, this embodiment avoids the need for the vPLC.R1 endpoint 1702 to perform extra steps, such as making a resource request to the CP, as shown in steps S7 and S8 of FIG. 16.

[0280] FIG. 18 is a simplified diagram illustrating a use case in which a vPLC is hosted by infrastructure in a first realm but associated with a second, different realm, according to one embodiment. In FIG. 18, there are two separate realms provided by a CSP: realm A 1802 and realm B 1812. Each realm is a logical collection of multiple regions. The realms are isolated from each other and do not share data. Thus, realm A 1802 is isolated from realm B 1812, and no data or communications are shared between the two realms. A tenancy created for a CSP's customer exists within a single realm and cannot be used to access another realm or region within another realm. Realms allow a CSP to provide a defined level of service across regions within that realm that meets specific organizational needs. For example, one realm may be created by a CSP for an enhanced level of security and isolation, such as a “government” realm created for a government agency that is a customer of the CSP. A second realm with a lower level of security and isolation may be created by the CSP, such as a "Commercial" realm for non-government commercial customers of the CSP. Examples of realms that may be created by a CSP include (a) a Commercial realm, (b) a U.S. Government FedRAMP-certified realm, (c) a U.S. Government IL5-certified realm, (d) a Country A Government realm, (e) a Country B Government realm, etc. For example, in FIG. 18, Realm A 1802 may have a higher security posture than Realm B 1812.

[0281] A realm can have one or more regions. Each realm has its own identity and trust profile (e.g., passwords, authentication information, etc.). This identity and trust profile information is shared between regions within the same realm, so regions within a realm can communicate with each other. The identity and trust profile of a realm is configured so that cross-realm communication is not enabled.

[0282] In the example depicted in Figure 18, realm A 1802 includes region A 1804. The infrastructure provided by the CSP within region A 1804 is organized as datacenter 1806. In Figure 18, datacenter 1806 is shown as a dedicated regional cloud datacenter (DRCC) hosted on the customer premises. A DRCC may refer to a fully managed cloud region built with CSP-specified high-performance infrastructure to help customers bring cloud primitives and services closer to their existing data and applications. A DRCC can be dedicated to a single customer and operates inside the customer's datacenter, but is fully managed by the CSP.

[0283] In FIG. 18 , realm B 1812 includes region B 1814. The infrastructure provided by the CSP in region B 1814 is organized as datacenter 1816. As shown in FIG. 18 , datacenter 1816 hosts vPLC 1818. Thus, the portion of the infrastructure provided by the CSP in datacenter 1816 in region B 1812 is allocated to vPLC 1818. Thus, vPLC 1818 is physically located within region B 1814 in realm B 1812.

[0284] As previously mentioned, when a vPLC is created, as part of configuring the vPLC, the vPLC is associated with a particular realm. Typically, a vPLC is associated with the same realm associated with the region in which the CSP-provided regional infrastructure hosting the vPLC is located. In some embodiments, when a vPLC is configured, it may be associated with a realm different from the realm associated with the regional infrastructure hosting the vPLC. For example, in the example depicted in FIG. 18 , vPLC 1818 may be associated with Realm A 1802 instead of Realm B 1812. The dashed box 1820 and associated arrow 1830 are intended to indicate that vPLC 1818 is physically hosted by datacenter 1816 in Realm B 1812, but is logically or virtually associated with and belongs to Realm A 1802. Realm B 1812 is referred to as the hosting realm or host realm of vPLC 1818. Realm A 1802 is referred to as the logical or virtual realm of the vPLC 1818.

[0285] This places vPLC 1818 and datacenter 1806 in the same realm, i.e., realm A 1802. Because infrastructure in the same realm shares the same identity and trust profile, vPLC 1818 is configured with the identity and trust profile of realm A 1802. Configuring identity and trust profile information for realm A 1802 can be achieved by creating a mapping in vPLC 1818. For example, an identity service API in the virtual realm (i.e., realm A 1802) may call a vPLC in the host realm (i.e., realm B 1812), which can accept the call and establish trust (i.e., share identity and trust profile information) between both realms. When an instance is created in a customer tenancy in a vPLC in the host realm, the resource ID and tenancy ID associated with the newly created instance in the host realm can be mapped to a new resource ID and tenancy ID in the virtual realm. Thus, a table of mappings (or a “shadow tenancy”) may be maintained within the host realm (i.e., Realm B 1812). The same process and mappings are also implemented within the virtual realm (i.e., Realm A 1802). Thus, both vPLC 1818 and Realm A 1802 (i.e., the virtual realm) share the same identity and trust profile information. As a result, because vPLC 1818 and datacenter 1806 are within Realm A 1802, they can communicate with each other even though vPLC 1818 is physically hosted by infrastructure in different Realm B 1812. vPLC 1818 may not be able to communicate with other infrastructure in Region B 1814 associated with Realm B 1812.

[0286] Additionally, vPLC 1818 is also logically associated with region A 1804, even if the vPLC is physically hosted by region B 1814. Region B 1814 is referred to as the hosting region or host region of vPLC 1818. Region A 1804 within realm A 1802 is referred to as the logical region or virtual region of vPLC 1818. The virtual realms and virtual regions of a vPLC may be configured when the vPLC is created.

[0287] As described with respect to the resource ID of a resource, it includes a portion that identifies the realm associated with the resource and a portion that identifies the region associated with the resource. A vPLC is considered a resource and is therefore assigned a resource ID. In one embodiment, the vPLC ID, which uniquely identifies a vPLC, is a type of resource ID. The vPLC ID includes a portion that identifies the virtual realm associated with the vPLC identified by the vPLC ID and a portion that identifies the virtual region associated with the vPLC. Thus, for purposes of managing identities in Realm A 1802, a vPLC is considered to be part of the realm and region identified by the vPLC ID corresponding to the vPLC.

[0288] In one embodiment, configuring a vPLC in the manner described above, where a vPLC is hosted by one region and one realm but associated with different virtual regions and different virtual realms, may be implemented using virtual bootstrap environment (ViBE) technology. A ViBE may refer to a virtual cloud network (VCN) provisioned within an overlay of an existing region (e.g., a “host region”). Once provisioned, the ViBE connects to the new region using a communication channel (e.g., an IPSec tunnel VPN). Certain required core services (or “seed” services), such as a deployment orchestrator, public key infrastructure (PKI) services, etc., may be provisioned within the ViBE. These services can provide the capabilities necessary to bring hardware online, establish a chain of trust within the new region, and deploy the remaining services within the new region. Using a virtual bootstrap environment can prevent circular dependencies between bootstrap resources by utilizing resources from the host region. Services can be staged and tested in the ViBE before the physical region (e.g., the target region) is available.

[0289] For example, in some embodiments, Realm B 1812 can create a ViBE within its realm, which is shared with identity and trust profile information of Realm A 1802. The ViBE within Realm B 1812 can then be used to bootstrap new regions within Realm A 1802. Further details related to ViBEs are described in the following patent applications, which are incorporated herein by reference for all purposes: (1) U.S. Nonprovisional Patent Application No. 18 / 105,779, filed February 3, 2023, entitled "VIRTUAL BOOTSTRAP ENVIRONMENT FOR BUILDING REGIONAL DATA CENTERS"; (2) U.S. Nonprovisional Patent Application No. 18 / 105,768, filed February 3, 2023, entitled "TECHNIQUES FOR A VIRTUAL BOOTSTRAP ENVIRONMENT IN A DISTRIBUTED VIRTUAL PRIVATE NETWORK," and (3) U.S. Nonprovisional Patent Application No. 18 / 105,779, filed February 3, 2023, entitled "TECHNIQUES FOR MIGRATING SERVICES FROM A VIRTUAL BOOTSTRAP ENVIRONMENT."

[0290] The architecture depicted in FIG. 18 , in which vPLCs are hosted by one region and one realm but associated with different virtual regions and different virtual realms, can be used for a variety of purposes. For example, in the example depicted in FIG. 18 , vPLC 1818 can serve as a backup for datacenter 1806. This makes it possible to have a backup for disaster recovery (DR) in a different geographic area. For example, if datacenter 1806 is a DRCC located on a customer's premises, the customer can ask a CSP to provide backup services for DRCC 1806. In response, the CSP can select vPLC 1818 to back up DRCC 1806, where vPLC 1818 is physically hosted in region B 1812, which is far away from region A 1802 that hosts DRCC 1806 and in a different geographic area. This can provide a more cost-effective solution for the CSP instead of having to build a separate datacenter for the customer in realm A 1802 solely for backup purposes. In some embodiments, the customer may not even know where the backup is performed, i.e., the existence and location of the vPLC 1818 may be invisible to the customer. The vPLC used for backup may be physically hosted in a CSP-provided cloud in a different realm and region.

[0291] Because the goal of the DR site is to introduce geographic separation between the primary and DR sites and minimize the occurrence of correlated failures by avoiding simultaneous changes to both the primary and secondary locations, the DRCC 1806 can enjoy DR guarantees given that the CSP can guarantee that the DRCC and its paired vPLC 1818 (and by extension, its hosting region in Realm B 1812) will not be changed at the same time.

[0292] 19 is a flow diagram 1900 illustrating an exemplary method for creating a vPLC, where the vPLC is hosted by a CSP-provided infrastructure in a particular region in a particular realm, but is virtually associated with a different region in a different realm, according to an embodiment. The process depicted in FIG. 19 may be performed as part of the process performed to create the vPLC. In an embodiment, the process depicted in FIG. 19 may be performed primarily by the CSP-provided infrastructure in the hosting region and realm.

[0293] Processing begins at 1902 when information is received identifying a host region within a host realm in which a vPLC to be created will be physically hosted. For example, for the embodiment depicted in Figure 18, information may be received that a vPLC will be created in region B 1814 within realm A 1812. The region and realm identified at 1902 represent the host region and host realm for the vPLC to be created.

[0294] At 1904, information may be received identifying a region within a particular realm, the particular realm being different from the host realm identified at 1902, and the region and particular realm identified at 1904 to be associated as a virtual region and virtual realm for the vPLC to be created. For example, for the embodiment depicted in FIG. 18, information may be received that the vPLC will be virtually associated with Region A 1804 within Realm A 1802.

[0295] At 1906, identity and trust profile information is obtained for the particular realm identified at 1904. The information obtained at 1906 may include identity authentication information, including passwords, certificates, etc., associated with the realm that will become the virtual realm for the vPLC.

[0296] At 1908, a vPLC ID is generated for the vPLC to be created, the vPLC ID identifying the region and particular realm identified in the information received at 1904.

[0297] At 1910, a vPLC is created in the host realm and resources are allocated to the vPLC, the allocated resources being selected from resources in the infrastructure provided by the CSP that are within the region identified in 1902 and within the realm identified in 1902. These resources may include one or more physical resources (e.g., racks, servers, routers, memory, or storage resources) and / or logical resources (e.g., hypervisors, virtual machines, virtual routers). For example, for the embodiment depicted in FIG. 18, resources are allocated to the vPLC from data center 1816.

[0298] At 1912, a vPLC is configured using the identity and trust profile information obtained at 1906. For example, for the embodiment depicted in Figure 18, the vPLC may be configured using the identity and trust profile information of Realm A 1802.

[0299] At 1914, the vPLC initiates communication with the CSP-provided infrastructure within the particular realm identified in 1904, which is the vPLC's virtual realm. For example, for the embodiment depicted in Figure 18, the vPLC 1818 communicates with the data center 1806. The communication may be initiated by the vPLC 1818 or the data center 1806.

[0300] Virtual Private Label Cloud (vPLC) - Identity Management In one embodiment, the Identity Cloud Service (IDCS) can have two models: a shared IDCS stack model and an independent IDCS stack model. An IDCS stack (also called an identity service stack) can refer to a set of resources (e.g., compute nodes, memory, databases, etc.) capable of performing identity management functions. In the shared IDCS stack model, two or more vPLCs can use the same identity stack to perform identity management functions for their respective vPLCs. In the independent IDCS stack model, each vPLC has a dedicated IDCS stack to perform identity management functions for that vPLC.

[0301] When a reseller subscribes to a vPLC service provided by a CSP to create a vPLC for providing a reseller-supplied cloud service, the identity cloud service as part of the reseller-supplied cloud service can be either a shared IDCS stack model or an independent IDCS stack model, depending on the agreement between the reseller and the CSP. The CSP control plane (CP) may also request an endpoint management service (EMS) to create a default identity service endpoint associated with the identity cloud service for the vPLC. A namespace is also created and reserved for the vPLC. The identity cloud service can then store identity information (including identity configuration information), which may be implemented as one or more identity records, in the namespace associated with the vPLC ID.

[0302] In the shared IDCS stack model, a single shared IDCS stack can simultaneously manage identities for multiple different namespaces associated with multiple different vPLCs by tracking vPLC IDs and never allowing identity information from two different vPLC namespaces to be used together. On the other hand, in the independent IDCS stack model, each independent IDCS stack is associated with namespace 0 (e.g., reserved for CSPs) but can be an exact clone of an IDCS stack residing in a different namespace. A clone IDCS stack can have its own set of resources (e.g., compute nodes, storage, databases, etc.) and operate as an independent identity service dedicated to a single vPLC and its corresponding namespace.

[0303] FIG. 20 illustrates a shared IDCS stack model, according to one embodiment. The distributed environment 2000 illustrated in FIG. 20 is merely exemplary and is not intended to unduly limit the scope of the claimed embodiments. Many variations, alternatives, and modifications are possible. For example, in some other embodiments, the distributed environment 2000 may have more or fewer systems or components than those shown in FIG. 20, may combine two or more systems, or may have a different configuration or arrangement of systems. The systems, subsystems, and other components illustrated in FIG. 20 may be implemented in software (e.g., code, instructions, programs) executed by one or more processing units (e.g., processors, cores) of the respective systems, using hardware, or a combination thereof. The software may be stored on a non-transitory storage medium (e.g., on a memory device).

[0304] In FIG. 20 , a single IDCS stack may be responsible for performing identity management functions for a CSP in namespace 0 2020, a reseller R1 associated with vPLC.R1 in namespace 1 2030, and a reseller R2 associated with vPLC.R2 in namespace 2 2040. In some embodiments, the IDCS stack 2010 may include a set of resources (e.g., compute nodes, storage, databases, etc.) of a CSP-provided regional cloud infrastructure.

[0305] In Figure 20, the CSP has an identity endpoint and console (sometimes referred to as identity endpoint only) 2002 for access by the CSP's non-reseller customers. vPLC.R1 has an identity endpoint 2004 for access by reseller R1's 2082 customers (e.g., C1 and C2) and their customers' users. vPLC.R2 has an identity endpoint 2006 for access by reseller R2's 2084 customers (e.g., C1 and C2) and their customers' users.

[0306] In FIG. 20 , namespace 0 for CSP 2020 may contain identity records (i.e., identity information including identity configuration information) 2022 of the CSP's non-reseller customers and their associated users. Namespace 1 for vPLC.R1 2030 may contain identity information 2032 for reseller R1 and identity information 2034 for reseller R1's customers and their associated users. In one embodiment, reseller R1 can manage the identity information stored (e.g., create, modify, or delete user information) for both reseller R1 and its users 2032 and for R1's customers and their associated users 2034. Reseller R1 can grant its customers permission to manage the identity information stored for R1's customers and their associated users 2034.

[0307] Namespace 2 for vPLC.R2 2040 may contain identity information 2042 for reseller R2 and identity information 2044 for reseller R2's customers and their associated users. In one embodiment, reseller R2 can manage the identity information stored in both reseller R2 and its users 2042 and R2's customers and their associated users 2044 (e.g., create, modify, or delete user information). Reseller R2 can grant its customers permission to manage the identity information stored in R2's customers and their associated users 2044. In one embodiment, namespaces and identity information may be stored in a repository, such as non-volatile memory, a database, a disk, or the like, of the CSP-provided region cloud infrastructure. Each namespace (e.g., namespace 0 2020, namespace 1 2030, or namespace 2 2040) may be part of a repository. In other embodiments, each namespace may have a dedicated repository.

[0308] When reseller R1 subscribes to a vPLC service offered by a CSP to provide a reseller-supplied and reseller-branded cloud service using a vPLC.R1 created for reseller R1, the CSP can request authentication information (e.g., username and password) from reseller R1 and its users and store the authentication information in namespace1 2030 as identity information 2032 for identity checks (e.g., authentication and authorization) when the CSP later creates namespace1 2030 for vPLC.R1. When a customer (e.g., C1) of reseller R1 subscribes to a reseller-supplied cloud service through the vPLC.R1 endpoint and console 2004, reseller R1 can request authentication information of the customer and its associated users from the customer when creating the customer account and store the authentication information in namespace1 2030 as identity information 2034 for later identity checks. Identity information of the reseller, customer, and their respective users may include, but is not limited to, usernames, passwords, roles, administrative status, and access rights. A customer (e.g., C1) of reseller R1 can create or modify identity configuration information for the customer's users according to the user's role and function. A similar setup process can be performed for reseller R2 and its associated users, R2's customers, and their associated users.

[0309] When a user of reseller R1 or a user of a customer of R1 2082 requests some operation for cloud services (e.g., accessing resources in vPLC.R1 or launching a compute instance), vPLC.R1 identity endpoint 2004 may be invoked in the background. vPLC.R1 identity endpoint 2004 may augment the request with vPLC-related information of vPLC.R1 (e.g., vPLC ID and customer tenancy ID) and send the information to IDCS stack 2010. Based on the vPLC-related information, the IDCS stack may perform an identity check on the user by accessing the user's identity information in namespace 1 for vPLC.R1 2030 to determine whether the request is permitted or denied. When a user of reseller R2 or a user of a customer of R2 2084 requests some operation for cloud services, the request may pass through vPLC.R2 identity endpoint 2006, which augments the request with vPLC-related information of vPLC.R2, and also reach the shared IDCS stack 2010. Thus, the shared IDCS stack can use the user's identity information in namespace 2 for vPLC.R2 2040 to determine whether the request is allowed or denied. Even if users of reseller R1 and reseller R2 use the same identity cloud service provided by the same IDCS stack 2010, the IDCS stack can use the vPLC-related information to track and identify the appropriate identity information in their corresponding namespaces for each vPLC.

[0310] FIG. 21 illustrates another embodiment of a shared IDCS stack model, according to one embodiment. Both FIGs. 21 are similar to FIG. 21 in that both diagrams illustrate a shared IDCS stack for implementing identity management functions for a CSP, a reseller R1 associated with vPLC.R1, and a reseller R2 associated with vPLC.R2. However, in FIG. 21, identity information for resellers R1 and R2 and their associated users is stored in namespace 0 2120 for the CSP, rather than namespace 1 2130 for vPLC.R1 and namespace 2 2140 for vPLC.R2. Identity information for customers of reseller R1 and reseller R2 and their associated users is still stored in namespace 1 2130 for vPLC.R1 and namespace 2 2140 for vPLC.R2, respectively.

[0311] In this embodiment of the shared IDCS stack model, the identity information of resellers (e.g., R1 and R2) and their associated users is stored in namespace 0, so the CSP needs to provide resellers with access to this identity information for management (e.g., creating, modifying, or deleting user information), for example, by creating shadow copies of this identity information.

[0312] FIG. 22 illustrates an independent IDCS stack model, according to one embodiment. The distributed environment 2200 illustrated in FIG. 22 is merely exemplary and is not intended to unduly limit the scope of the claimed embodiments. Many variations, alternatives, and modifications are possible. For example, in some other embodiments, the distributed environment 2200 may have more or fewer systems or components than those shown in FIG. 22, may combine two or more systems, or may have a different configuration or arrangement of systems. The systems, subsystems, and other components illustrated in FIG. 22 may be implemented in software (e.g., code, instructions, programs) executed by one or more processing units (e.g., processors, cores) of the respective systems, using hardware, or a combination thereof. The software may be stored on a non-transitory storage medium (e.g., on a memory device).

[0313] In FIG. 22 , there are three IDCS stacks: IDCS stack 2210 for CSP, IDCS stack 2212 for vPLC.R1, and IDCS stack 2214 for vPLC.R2. Each IDCS can operate independently and have its own set of resources (e.g., compute nodes, memory, databases, etc.) capable of performing identity management functions. In other words, for example, IDCS stack 2212 for vPLC.R1 processes requests from identity endpoint 2204 and performs identity management functions only for vPLC.R1 by accessing identity information in namespace 1 2230, i.e., IDCS stack 2212 for vPLC.R1 does not communicate with other IDCS stacks or access identity information associated with other vPLCs. Similarly, IDCS stack 2214 for vPLC.R2 also performs identity management functions only for vPLC.R2.

[0314] In one embodiment, the namespace and identity information is stored in a repository, such as non-volatile memory, a database, a disk, etc., of the CSP-provided region cloud infrastructure. Each namespace (e.g., namespace 0 2220, namespace 1 2230, or namespace 2 2240) may be part of a repository. In other embodiments, each namespace may have a dedicated repository.

[0315] In an embodiment, IDCS stack 2212 for vPLC.R1 associated with namespace 1 may be a clone of IDCS stack 2210 for the CSP associated with namespace 0. Similarly, IDCS stack 2214 for vPLC.R2 associated with namespace 1 may be a clone of IDCS stack 2210 for the CSP associated with namespace 0. In some embodiments, IDCS stack 2210 for the CSP, IDCS stack 2212 for vPLC.R1, and IDCS stack 2214 for vPLC.R2 may be different sets of resources in a CSP-provided regional cloud infrastructure.

[0316] For example, consider the case where reseller R1 subscribes to a vPLC service provided by a CSP to create vPLC.R1, and the CSP-reseller agreement indicates that vPLC.R1 has a dedicated IDCS stack. As a result, the CSP can create an independent IDCS stack 2212 for vPLC.R1 for reseller R1 by using IDCS stack 2210 for the CSP as a model to create a clone using similar architecture and resources. An identity endpoint 2204 for vPLC.R1 is created and associated with an identity cloud service running on IDCS stack 2212 for vPLC.R1. Namespace 1 2230 is also reserved for vPLC.R1, and namespace 1 stores the authentication information of reseller R1 and its associated users as identity information 2232 and the authentication information of reseller R1's customers and their associated users as identity information 2234.

[0317] Similarly, an identity endpoint 2206 for vPLC.R2 is created and associated with the identity cloud service running on the IDCS stack 2214 for vPLC.R2. Namespace2 2240 is also reserved for vPLC.R2, and namespace2 stores the credentials of reseller R2 and its associated users as identity information her2242 and the credentials of reseller R2's customers and their associated users as identity information 2244.

[0318] Identity management configurations may be implemented at the reseller level and the reseller customer level. At the reseller level, an identity cloud service is configured to perform identity management functions for the reseller and its associated users. At the reseller customer level, an identity cloud service is configured to perform identity management functions for the reseller's customers and their associated users.

[0319] FIG. 23 is a flow diagram illustrating an identity management configuration process at the reseller level, according to an embodiment. The process illustrated in FIG. 23 may be implemented in software (e.g., code, instructions, programs) executed by one or more processing units (e.g., processors, cores) of the respective systems, using hardware, or a combination thereof. The software may be stored on a non-transitory storage medium (e.g., on a memory device). The method presented in FIG. 23 and described below is intended to be exemplary and non-limiting. While FIG. 23 depicts various processing steps occurring in a particular sequence or order, this is not intended to be limiting. In an alternative embodiment, the process may be performed in some different order, or some steps may also be performed in parallel. It should be appreciated that in alternative embodiments, the process illustrated in FIG. 23 may include more or fewer steps than depicted in FIG. 23.

[0320] In one embodiment, for example, the process depicted in Figure 23 may be performed by a CSP. Some steps may be performed by components within the CSP-provided region infrastructure (e.g., control plane (CP), resource manager (RM), and management plane (MP), etc.) or one or more CSP-provided cloud services, e.g., endpoint management service (EMS), identity management (and identity cloud services), etc. For example, step 2310 may be performed by a CSP CP receiving a request to configure identity management for a vPLC.

[0321] In step 2310, in one embodiment, a request to configure identity management for a vPLC associated with reseller R may be received when the reseller subscribes to a vPLC service provided by a CSP to create a vPLC. In step 2312, the CSP may check the type of IDCS stack model to be used for the vPLC, i.e., a shared IDCS stack or an independent IDCS stack. In step 2314, the CSP determines the IDCS stack model to be used. If the reseller selects to use the shared IDCS stack model, the process proceeds to step 2320. If the reseller selects to use the independent IDCS stack model, the process proceeds to step 2350.

[0322] For a shared IDCS stack model, in step 2320 (dashed box in FIG. 23 ), which includes steps 2324-2328, a CSP can configure an existing IDCS stack (e.g., 2010 in FIG. 20 ) to manage identities for the vPLC. In step 2324, the existing IDCS stack (or referred to as a generic IDCS stack) can create a namespace for the vPLC. For example, in FIG. 20 , an already-existing IDCS stack 2010, which provides identity cloud services (i.e., part of the identity management functionality) for non-reseller customers of CSP 2080, can be configured to perform identity services for reseller R1, R1's customers, and their associated users 2082. As a result, namespace 1 2030 is created for vPLC R1. In step 2326, the existing IDCS stack can create the reseller's identity information within the namespace. For example, in FIG. 20, an existing IDCS stack 2010 can obtain authentication information from resellers and their users, such as usernames, passwords, and other information, for identity management purposes, and store this information in repository 2032 within namespace 1 2030.

[0323] In step 2328, the existing IDCS stack can assign one or more identity management roles to one or more users of the reseller. Once the IDCS stack creates the identity information, it can also perform other tasks, such as assigning a role (e.g., the authority to create accounts for the reseller's customers) to the requesting reseller R based on a CSP-reseller agreement. A user of the reseller R can act as a registration agent / administrator for the vPLC and create accounts for the reseller's customers. In some embodiments, the reseller R and its associated users can create spaces (i.e., accounts with identity information), such as 2034 in FIG. 20, for the reseller's customers within a namespace (e.g., 2030 in FIG. 20) associated with the vPLC. In other embodiments, the reseller may include different levels of users with different permissions for different roles (e.g., business managers and financial personnel). Thus, different users may require different identity configuration information to log in.

[0324] Step 2320, including steps 2324-2328, may be repeated for each vPLC. In step 2340, the existing IDCS stack may manage / initiate identity management functions (described below) for the vPLC and other vPLCs, if any.

[0325] For the independent IDCS stack model, the steps are similar to those of the shared IDCS stack model because both models require creating a namespace for the vPLC and adding identity information to that namespace. This similarity is marked as dashed boxes in steps 2320 and 2360 in both flows. However, the independent IDCS stack model has an additional step (e.g., 2350) of cloning from an existing IDCS stack. In step 2350, a CSP can clone an existing / general-purpose IDCS stack to become a vPLC IDCS stack, i.e., an independent IDCS stack dedicated to the vPLC. For example, in FIG. 22, an existing (or general-purpose) IDCS stack 2210 already exists and can provide identity cloud services to non-reseller customers of CSP 2280. The CSP can create a new vPLC IDCS stack 2212 for vPLC.R1 using another set of resources (e.g., compute nodes, storage, databases, etc.) that can perform identity management functions like the existing IDCS stack 2210.

[0326] When a new independent vPLC IDCS stack is created for a vPLC, the new vPLC IDCS stack may perform steps similar to those performed in the shared IDCS stack model. In step 2360 (dashed box in FIG. 23 ), which includes steps 2364-2368, the newly created vPLC IDCS stack may configure the newly created vPLC IDCS stack to manage identities for the vPLC. In step 2364, the vPLC IDCS stack may create a namespace for the vPLC (e.g., namespace 1 2230 for vPLC.R1 in FIG. 22 ). In step 2366, the vPLC IDCS stack may create identity information within the namespace for the reseller. For example, in FIG. 22 , the vPLC IDCS stack 2212 may obtain authentication information from the reseller and its associated users, such as usernames, passwords, and other information, for identity management purposes, and store this information in repository 2232 within namespace 1 2230.

[0327] In step 2368, the vPLC IDCS stack may assign one or more identity management roles to one or more users of reseller R, such as the role of vPLC registration officer / administrator who creates accounts for the reseller's customers. Step 2360, including steps 2364-2368, may be repeated for each vPLC. Finally, because a vPLC ID is dedicated to a particular vPLC, in step 2380 the vPLC IDCS stack may manage / initiate identity management functions (described below) for that vPLC.

[0328] FIG. 24 is a flow diagram illustrating another embodiment of an identity management configuration process at the reseller level, according to an embodiment. The process illustrated in FIG. 24 may be implemented in software (e.g., code, instructions, programs) executed by one or more processing units (e.g., processors, cores) of the respective systems, using hardware, or a combination thereof. The software may be stored on a non-transitory storage medium (e.g., on a memory device). The method presented in FIG. 24 and described below is intended to be exemplary and non-limiting. While FIG. 24 depicts various processing steps occurring in a particular sequence or order, this is not intended to be limiting. In an alternative embodiment, the process may be performed in some different order, or some steps may also be performed in parallel. It should be appreciated that in alternative embodiments, the process illustrated in FIG. 24 may include more or fewer steps than depicted in FIG. 24.

[0329] FIG. 24 is similar to the identity management configuration process depicted in FIG. 23 . However, in FIG. 24 , a CSP (e.g., control plane) may perform namespace creation and identity information creation for the vPLC before branching into two subflows for the shared IDCS stack model or the independent IDCS stack model. In some embodiments, an existing IDCS may perform namespace creation and identity information creation for the vPLC with assistance from a CSP CP. In contrast, in FIG. 23 , the identity management configuration process branches into two subflows; first, in the subflow for the independent IDCS stack model, the vPLC IDCS stack takes responsibility for performing namespace creation and identity information creation for the vPLC. In other words, the vPLC IDCS stack handles most of the identity management configuration for that vPLC in FIG. 23 , while the CSP handles most of the identity management configuration for the vPLC in FIG. 24 .

[0330] 24 , in one embodiment, a CSP CP may receive a request to configure identity management for a vPLC associated with a reseller R when the reseller subscribes to a vPLC service provided by the CSP to create a vPLC. In step 2412, the CSP may create a namespace for the vPLC associated with the reseller, such as, for example, either namespace 1 2030 for vPLC.R1 in FIG. 20 for a shared IDCS stack model or namespace 1 2230 for vPLC.R1 in FIG. 22 for an independent IDCS stack model.

[0331] In step 2414, the CSP may create the reseller's identity information within a namespace for the vPLC, such as, for example, either identity information 2032 within namespace 1 2030 for vPLC.R1 in FIG. 20 for a shared IDCS stack model, or identity information 2232 within namespace 1 2230 for vPLC.R1 in FIG. 22 for an independent IDCS stack model.

[0332] In step 2418, the CSP may check the type of IDCS stack model to be used for the vPLC, i.e., whether it is a shared IDCS stack or an independent IDCS stack. In step 2418, the CSP determines the IDCS stack model to be used. If the reseller selects to use the shared IDCS stack model, the process proceeds to step 2420. If the reseller selects to use the independent IDCS stack model, the process proceeds to step 2450.

[0333] For the shared IDCS stack model, in step 2420 (dashed box in FIG. 24), which includes steps 2424-2428, a CSP can configure an existing IDCS stack (e.g., 2010 in FIG. 20) to manage identities for the vPLC. In step 2424, since the CSP has created a namespace and reseller identity information for the vPLC, the CSP can provide the existing / general-purpose IDCS stack with access to the namespace and reseller identity information for the vPLC. Similar to 2328 in FIG. 23, in step 2428, the existing IDCS stack can assign one or more identity management roles to one or more users of the reseller.

[0334] Step 2420, including steps 2424-2428, may be repeated for each vPLC. In step 2440, the existing IDCS stack may manage / initiate identity management functions (described below) for the vPLC and other vPLCs, if any.

[0335] For the independent IDCS stack model, the steps are similar to those of the shared IDCS stack model with respect to configuring an IDCS stack to manage identities for a vPLC. This similarity is marked as dashed boxes in steps 2420 and 2460 in both flows. However, the independent IDCS stack model has an additional step (e.g., 2450) of cloning from an existing IDCS stack. Similar to step 2350 in FIG. 23, in step 2350, a CSP can clone an existing / general-purpose IDCS stack to become a vPLC IDCS stack, i.e., an independent IDCS stack dedicated to the vPLC.

[0336] In step 2360 (dashed box in FIG. 23 ), which includes steps 2364-2368, the CSP may configure the newly created vPLC IDCS stack to manage identities for the vPLC. In step 2464, now that the CSP has created a namespace and reseller identity information for the vPLC, the CSP may provide the vPLC IDCS stack with access to the namespace and reseller identity information for the vPLC. In step 2468, the CSP may assign one or more users of reseller R one or more identity management roles, such as a vPLC registration officer / administrator role that creates accounts for the reseller's customers.

[0337] Step 2460, including steps 2464-2468, may be repeated for each vPLC. Finally, because the vPLC ID is dedicated to a particular vPLC, in step 2480 the vPLC IDCS stack may manage / initiate identity management functions (described below) for that vPLC.

[0338] After the identity management configuration process for a reseller associated with a vPLC is complete, the reseller can set up the identity information of the reseller's customers within the namespace for the vPLC as part of the functions performed by the reseller in the role of registration officer / administrator for the vPLC.

[0339] FIG. 25 is a flow diagram illustrating an identity management configuration process at a reseller customer level, according to an embodiment. The process illustrated in FIG. 25 may be implemented in software (e.g., code, instructions, programs) executed by one or more processing units (e.g., processors, cores) of the respective systems, using hardware, or a combination thereof. The software may be stored on a non-transitory storage medium (e.g., on a memory device). The method presented in FIG. 25 and described below is intended to be exemplary and non-limiting. While FIG. 25 depicts various processing steps occurring in a particular sequence or order, this is not intended to be limiting. In an alternative embodiment, the process may be performed in some different order, or some steps may also be performed in parallel. It should be appreciated that in alternative embodiments, the process illustrated in FIG. 25 may include more or fewer steps than depicted in FIG. 25.

[0340] In some embodiments, the configuration process for setting up identity information for a reseller's customers may be performed by the CSP's identity service if the reseller delegates the task to the CSP, or by an identity service running on an existing IDCS stack. In other embodiments, this setup process may be performed in combination with an existing IDCS stack and a vPLC IDCS stack, such as by handing over from the existing IDCS stack to the vPLC IDCS stack after the vPLC IDCS stack is cloned.

[0341] At step 2510, a request may be received to set up identity information for a customer of a reseller associated with a vPLC. At step 2512, it may need to be determined whether the vPLC associated with the reseller uses a shared IDCS stack model or an independent IDCS stack model. At step 2514, if the vPLC uses a shared IDCS stack model, the setup process proceeds to step 2520. If the vPLC uses an independent IDCS stack model, the setup process proceeds to step 2540.

[0342] For a shared IDCS stack model, in step 2520, the existing IDCS stack can create customer identity information and customer user identity information for the vPLC. For example, in FIG. 20 , an identity service running on the existing IDCS stack (e.g., 2010 in FIG. 20 ) can obtain authentication information from reseller R1 2082's customers and each customer's user for storage in namespace1 2030 of vPLC.R1 as identity information 2034. In some embodiments, the access permissions of a reseller's two customers can be controlled based on two different policies (i.e., reseller-customer agreements) negotiated between these two customers and the reseller. For example, one customer can access all services, while the other customer can access only a subset of services (e.g., only compute services). In some embodiments, the permissions and roles of a customer's users, such as how the partitions and resources contained in the customer's tenancy within the vPLC are accessed by that customer's users, can be managed by the reseller's customers. In step 2530, the existing IDCS stack can manage / initiate identity management functions (described below) for the customer and its users.

[0343] For the independent IDCS stack model, identity management for the reseller may need to be identified in step 2540, which the vPLC IDCS stack handles. For example, in FIG. 22, the vPLC's identity services may initially run on the existing IDCS stack 2210. Once the IDCS stack 2212 for vPLC.R1 is set up and identified, the existing IDCS stack can hand over control to the identified vPLC IDCS stack 2212.

[0344] In step 2542, identity information for the reseller's customers and identity information for the customers' users may be created. For example, in FIG. 22, an identity service for running on an existing vPLC IDCS stack (e.g., 2212 in FIG. 22) can obtain authentication information from reseller R1 2282's customers and each customer's user for storage in namespace 1 2230 of vPLC.R1 as identity information 2234. In step 2550, the vPLC IDCS stack can manage / initiate identity management functions (described below) for the customers and their users.

[0345] As mentioned above, identity information of the reseller and its associated users, as well as identity information of the reseller's customers and their associated users, may be stored in a namespace for the vPLC, such as, for example, namespace 1 2030 for vPLC.R1 in FIG. 20 or namespace 1 2230 for vPLC.R1 in FIG. 22.

[0346] 26 and 27 illustrate a mechanism for storing identity information within a customer tenancy of a vPLC, according to an embodiment. As discussed above in connection with FIGS. 9A and 9B regarding the mechanism for storing information in a customer tenancy of a vPLC, FIGS. 26 and 27 illustrate identity information that is part of the information stored in a customer tenancy of a vPLC. Identity information may be organized and stored in two ways: (1) a distributed manner (or record space per vPLC) as shown in FIG. 26; and (2) a centralized manner (or using the vPLC ID as the partition key) as shown in FIG. 27. In the distributed manner, identity information is stored separately for each vPLC. Stored identity information may be for a reseller and its associated users, the reseller's customers and their associated users. In the centralized manner, all identity information is stored in a central repository (e.g., a table), and the vPLC ID (or the reseller's tenancy ID) is one partition key (e.g., a column in a table).

[0347] In FIG. 26 , in one embodiment, there are two vPLC-specific tables: 2610 for vPLC.R1 and 2620 for vPLC.R2. In each vPLC-specific table, each row may include a tenancy ID (in the first column) and identity information associated with that tenancy ID (in the second column). For example, in vPLC-specific table 2610 for vPLC.R1, the first row (or record) has the tenancy ID of reseller R1 (i.e., R1's T_ID) and identity information for R1, which may be information about a representative (e.g., organization) of reseller R1. The second row has identity information for users associated with R1, which may be information about user IDs of users of reseller R1 and personnel assigned different roles within reseller R1's organization. The third row has the tenancy ID of reseller R1's customer C1 and identity information for customer C1, which may be information about a representative (e.g., organization) of customer C1. The fourth row has the user IDs of users associated with customer C1 and their identity information, which may be information about personnel assigned different roles within customer C1's organization. The fifth and sixth rows are for reseller R1's customer C2 and their associated users. A similar scheme may be applied to vPLC-specific table 2620 for vPLC.R2.

[0348] In some embodiments, the identity information in vPLC-specific table 2610 for vPLC.R1 is stored in namespace 1 2030 for vPLC.R1 in FIG. 20 for a shared IDCS stack model. The first two rows / records in vPLC-specific table 2610 for vPLC.R1 in FIG. 26 are identity information for reseller R1 and its associated users, represented as logical partition 2032 in FIG. 20. The last four rows / records in vPLC-specific table 2610 for vPLC.R1 in FIG. 26 are identity information for reseller R1's customers and their associated users, represented as logical partition 2034 in FIG. 20.

[0349] Similarly, the identity information in vPLC-specific table 2610 for vPLC.R1 may be stored in namespace 1 2030 for vPLC.R1 in Figure 22 for an independent IDCS stack model. Logical partition 2232 in Figure 22 may represent the first two records in vPLC-specific table 2610 of Figure 26 that contain identity information for reseller R1 and its associated users. Logical partition 2234 in Figure 22 may represent the last four records in vPLC-specific table 2610 of Figure 26 that contain identity information for reseller R1's customers and their associated users.

[0350] In Figure 27, in one embodiment, table 2730 may have four columns, with the first three columns each containing a tenancy ID as the partition key for the table. For example, the first column contains the tenancy ID (or vPLC ID) of the reseller. The second column contains the tenancy ID of the reseller's customer. The third column contains the user ID of the user associated with the reseller's customer. The fourth column contains identity information.

[0351] In FIG. 27, rows 1 to 6 may include identity information of reseller R1, customers of reseller R1, and their associated users. For example, the first row (or record) has the tenancy ID of reseller R1 (i.e., R1's T_ID or vPLC ID) and identity information of R1. The second row has the tenancy ID of reseller R1, user IDs of users of reseller R1, and identity information of users associated with R1. The third row has the tenancy ID of reseller R1, the tenancy ID of customer C1 of reseller R1, and identity information of customer C1. The fourth row has the tenancy ID of reseller R1, the tenancy ID of customer C1 of reseller R1, user IDs of users associated with customer C1, and identity information of these users. A similar scheme may be applied to identity information for reseller R2, reseller R2's customers, and their associated users, as shown in lines 7-12.

[0352] Similarly, the identity information in table 2730 of FIG. 27 may be logically partitioned to represent identity information in namespace 1 2030 for vPLC.R1 and namespace 2 2040 for vPLC.R2 of FIG. 20 for the shared IDCS stack model. For example, rows 1 and 2 in table 2730 may represent identity information 2032 of reseller R1 and its associated users in FIG. 20. Rows 3-6 in table 2730 may represent identity information 2034 of customers of reseller R1 and their associated users in FIG. 20. Similarly, rows 7 and 8 in table 2730 may represent identity information 2042 of reseller R2 and its associated users in FIG. 20. Rows 9-12 in table 2730 may represent identity information 2044 of customers of reseller R2 and their associated users in FIG. 20. The same logical partitioning may apply for the independent IDCS stack model of FIG. 22.

[0353] As described above, the IDCS stack for a vPLC (either a shared or independent IDCS stack) can perform identity management functions using stored identity information of a user when the IDCS receives a user request. Identity management functions may include authentication (e.g., identity checks), authorization, access control, etc. The verification process (including authentication and authorization) may be performed separately for the reseller's users and the reseller's customer's users associated with the vPLC.

[0354] FIG. 28 is a flow diagram illustrating a process for implementing identity management functions for a reseller's users using an IDCS, according to an embodiment. The process illustrated in FIG. 28 can be implemented in software (e.g., code, instructions, programs) executed by one or more processing units (e.g., processors, cores) of a respective system, using hardware, or a combination thereof. The software can be stored on a non-transitory storage medium (e.g., on a memory device). The method presented in FIG. 28 and described below is intended to be exemplary and non-limiting. While FIG. 28 depicts various processing steps occurring in a particular sequence or order, this is not intended to be limiting. In an alternative embodiment, the process may be performed in some different order, or some steps may also be performed in parallel. It should be appreciated that in alternative embodiments, the process illustrated in FIG. 28 may include more or fewer steps than depicted in FIG. 28.

[0355] In step 2810, an IDCS stack (either a shared IDCS stack or an independent IDCS stack) may receive a request to perform an identity management function for an operation to be performed by a user. In an embodiment, the operation may be part of a process to access one or more reseller-supplied cloud services or manage a vPLC on behalf of a reseller organization. The request may come directly from a user or through other components involved in the process of providing one or more reseller-supplied cloud services.

[0356] When receiving the request, the IDCS stack may also receive the user ID and vPLC-related information (e.g., the vPLC ID and customer tenancy ID augmented by the accessed identity service endpoint). In step 2820, the IDCS stack can determine the reseller / vPLC associated with the requesting user based on the user ID and the vPLC-related information. In step 2822, the IDCS stack can obtain the reseller's identity information and the reseller's user's identity information. For example, in FIG. 26, using R1's vPLC ID (or tenancy ID) and R1's user's user ID (e.g., both augmented by the vPLC.R1 endpoint 2004 in FIG. 20), the IDCS stack can access the first two rows / records of table 2610 per vPLC for vPLC.R1, which contains the reseller R1 and its associated user's identity information (see also 2032 in FIG. 20 for a shared IDCS stack model or 2232 for an independent IDCS stack model). Alternatively, in FIG. 27, the IDCS stack can access the first two rows / records of a centralized table 2730, which contains identity information for reseller R1 and its associated users (see also 2032 in FIG. 20 for a shared IDCS stack model or 2232 for an independent IDCS stack model).

[0357] In step 2840, the IDCS stack can use the obtained identity information to determine whether the user is authorized to perform the requested action on the requested vPLC. This step can involve using the obtained identity information to perform verification, including authentication and authorization. For example, the identity service can check whether the credentials provided by the user match the stored credentials (i.e., the user actually belonging to the requested vPLC), the user's role, and their access permissions. In step 2842, the identity service running on the IDCS stack can send a response signal indicating the result of the verification performed, i.e., whether the request is allowed or denied.

[0358] FIG. 29 is a flow diagram illustrating a process for implementing identity management functions for users of a reseller's customers using an IDCS, according to an embodiment. The process illustrated in FIG. 29 can be implemented in software (e.g., code, instructions, programs) executed by one or more processing units (e.g., processors, cores) of the respective system, using hardware, or a combination thereof. The software can be stored on a non-transitory storage medium (e.g., on a memory device). The method presented in FIG. 29 and described below is intended to be exemplary and non-limiting. While FIG. 29 depicts various processing steps occurring in a particular sequence or order, this is not intended to be limiting. In an alternative embodiment, the process may be performed in some different order, or some steps may also be performed in parallel. It should be appreciated that in alternative embodiments, the process illustrated in FIG. 29 may include more or fewer steps than depicted in FIG. 29.

[0359] In step 2910, the IDCS stack may receive a request to perform an identity management function for an operation to be performed by a user. In an embodiment, the operation may be part of a process to access one or more of the reseller-supplied cloud services. The request may come directly from the user or through other components involved in the process of providing one or more reseller-supplied cloud services.

[0360] When receiving the request, the IDCS stack may also receive the user ID and vPLC-related information (e.g., the vPLC ID and customer tenancy ID augmented by the accessed identity service endpoint). In step 2920, the IDCS stack may determine a customer associated with the requesting user based on at least the customer tenancy ID. In step 2922, the IDCS stack may further determine a reseller / vPLC associated with the customer based on at least the vPLC ID.

[0361] In step 2924, the IDCS stack can obtain the identity information of the reseller, the identity information of the reseller's customers, and the identity information of the reseller's customers' users. For example, in FIG. 26, using R1's vPLC ID (or tenancy ID), R1.C1's tenancy ID, and R1.C1's user's user ID, the IDCS stack can access rows 1, 3, and 4 of vPLC.R1's per-vPLC table 2610, which contains the identity information of reseller R1, R1.C1, and R1.C1's users (see also 2034 in FIG. 20 for a shared IDCS stack model or 2234 for an independent IDCS stack model). Alternatively, in FIG. 27, the IDCS stack can access rows 1, 3, and 4 of a centralized table 2730 containing identity information for resellers R1, R1.C1, and R1.C1's users (see also 2034 in FIG. 20 for a shared IDCS stack model or 2234 for an independent IDCS stack model).

[0362] In step 2940, the IDCS stack can use the obtained identity information to determine whether the user is authorized to perform the requested action on the requested vPLC. This step can involve using the obtained identity information to perform verification, including authentication and authorization. For example, the identity service can check whether the credentials provided by the user match the stored credentials (i.e., the user actually belonging to the requested vPLC and customer), the user's role (e.g., system administrator or regular user), and their access privileges. In step 2942, the identity service running on the IDCS stack can send a response signal indicating the result of the performed verification, i.e., whether the request is allowed or denied.

[0363] Exemplary Cloud Architecture As mentioned above, infrastructure as a service (IaaS) is one specific type of cloud computing. IaaS can be configured to provide virtualized computing resources over a public network (e.g., the Internet). In the IaaS model, a cloud computing provider can host infrastructure components (e.g., servers, storage devices, network nodes (e.g., hardware), deployment software, platform virtualization (e.g., hypervisor layer), etc.). In some cases, an IaaS provider can also supply various services to accompany those infrastructure components (example services include billing software, monitoring software, logging software, load balancing software, clustering software, etc.). Accordingly, these services can be policy-driven, allowing IaaS users to implement policies to drive load balancing to maintain application availability and performance.

[0364] In some cases, IaaS customers can access resources and services over a wide area network (WAN) such as the Internet and use the cloud provider's services to install the remaining elements of their application stack. For example, a user can log into an IaaS platform, create virtual machines (VMs), install an operating system (OS) on each VM, deploy middleware such as databases, create storage buckets for workloads and backups, and even install enterprise software on the VMs. The customer can then use the provider's services to perform a variety of functions, including balancing network traffic, troubleshooting application issues, monitoring performance, managing disaster recovery, etc.

[0365] In most cases, the cloud computing model requires the involvement of a cloud provider, which may, but does not necessarily have to, be a third-party service that specializes in providing (e.g., supplying, renting, selling) IaaS. An entity may also choose to deploy a private cloud and become its own provider of infrastructure services.

[0366] In some examples, IaaS deployment is the process of putting a new application or a new version of an application onto a provisioned application server, etc. It may also include the process of provisioning the server (e.g., installing libraries, daemons, etc.), which is often managed by the cloud provider below the hypervisor layer (e.g., server, storage, network hardware, and virtualization). Thus, the customer may be responsible for handling (OS), middleware, and / or application deployment (e.g., on self-service virtual machines (e.g., that can be spun up on demand)).

[0367] In some examples, IaaS provisioning may refer to obtaining computers or virtual hosts for use and installing the necessary libraries or services on them. In most cases, deployment does not include provisioning, which may need to be performed first.

[0368] In some cases, IaaS provisioning presents two distinct challenges. First, there is the initial challenge of provisioning an initial set of infrastructure before anything is operational. Second, there is the challenge of evolving the existing infrastructure (e.g., adding new services, modifying services, removing services, etc.) after everything is provisioned. In some cases, these two challenges can be addressed by allowing the configuration of the infrastructure to be defined declaratively. In other words, the infrastructure (e.g., which components are needed and how they interact) can be defined by one or more configuration files. Thus, the overall topology of the infrastructure (e.g., which resources depend on which and how they work together with each other) can be described declaratively. In some cases, once the topology is defined, workflows can be generated to create and / or manage the different components described in the configuration files.

[0369] In some examples, the infrastructure may have many interconnected elements. For example, there may be one or more virtual private clouds (VPCs) (e.g., configurable and / or shared, possibly on-demand pools of computing resources), also known as core networks. In some examples, there may also be one or more inbound / outbound traffic group rules and one or more virtual machines (VMs) provisioned to define how the network's inbound and / or outbound traffic is set up. Other infrastructure elements, such as load balancers, databases, etc., may also be provisioned. The infrastructure may evolve incrementally as more and more infrastructure elements are desired and / or added.

[0370] In some cases, continuous deployment techniques may be utilized to enable deployment of infrastructure code across various virtual computing environments. Additionally, the described techniques may enable infrastructure management within these environments. In some examples, a service team may write code that is desired to be deployed to one or more, but often many, different production environments (e.g., across a variety of different geographic locations, sometimes spanning the entire world). In some examples, however, the infrastructure onto which the code will be deployed is set up first. In some cases, that provisioning may be done manually, and provisioning tools may be utilized to provision resources and / or deployment tools may be utilized to deploy the code once the infrastructure has been provisioned.

[0371] 30 is a block diagram 3000 illustrating an example pattern of an IaaS architecture according to at least one embodiment. A service operator 3002 can be communicatively coupled to a secure host tenancy 3004, which can include a virtual cloud network (VCN) 3006 and a secure host subnet 3008. In some examples, the service operator 3002 can employ one or more client computing devices, which can be portable handheld devices (e.g., iPhones, mobile phones, iPads, computing tablets, personal digital assistants (PDAs)) or wearable devices (e.g., Google Glass head-mounted displays) running software such as Microsoft Windows Mobile and / or various mobile operating systems such as iOS, Windows Phone, Android, BlackBerry 8, Palm OS, etc., and which are enabled for Internet, email, short message service (SMS), Blackberry, or other communications protocols. Alternatively, the client computing devices may be general-purpose personal computers, including, by way of example, personal and / or laptop computers running various versions of Microsoft Windows, Apple Macintosh, and / or Linux operating systems. The client computing devices may be workstation computers running any of the various commercially available UNIX or UNIX-like operating systems, including, but not limited to, various GNU / Linux operating systems such as Google Chrome OS.Alternatively, or in addition, the client computing device may be any other electronic device capable of communicating over a network, such as a thin client computer, an Internet-enabled gaming system (e.g., a Microsoft Xbox gaming console with or without a Kinect® gesture input device), and / or a personal messaging device, that has access to VCN 3006 and / or the Internet.

[0372] VCN 3006 may include a local peering gateway (LPG) 3010, which may be communicatively coupled to a secure shell (SSH) VCN 3012 via an LPG 3010 included in SSH VCN 3012. SSH VCN 3012 may include an SSH subnet 3014, which may be communicatively coupled to a control plane VCN 3016 via an LPG 3010 included in control plane VCN 3016. SSH VCN 3012 may also be communicatively coupled to a data plane VCN 3018 via LPG 3010. Control plane VCN 3016 and data plane VCN 3018 may be included in a service tenancy 3019, which may be owned and / or operated by the IaaS provider.

[0373] The control plane VCN 3016 may include a control plane demilitarized zone (DMZ) tier 3020 that acts as a perimeter network (e.g., a portion of an enterprise network between the enterprise intranet and an external network). DMZ-based servers may have limited roles and may help keep breaches contained. Additionally, the DMZ tier 3020 may include one or more load balancer (LB) subnets 3022, a control plane app tier 3024 that may include an app subnet 3026, and a control plane data tier 3028 that may include a database (DB) subnet 3030 (e.g., a front-end DB subnet and / or a back-end DB subnet). LB subnet 3022 included in control plane DMZ tier 3020 can be communicatively coupled to app subnet 3026 included in control plane app tier 3024 and to internet gateway 3034, which may be included in control plane VCN 3016, and app subnet 3026 can be communicatively coupled to DB subnet 3030 included in control plane data tier 3028, as well as to service gateway 3036 and network address translation (NAT) gateway 3038. Control plane VCN 3016 can include service gateway 3036 and NAT gateway 3038.

[0374] Control plane VCN 3016 can include a data plane mirror app tier 3040, which can include an app subnet 3026. The app subnet 3026 included in data plane mirror app tier 3040 can include a virtual network interface controller (VNIC) 3042 on which a compute instance 3044 can run. The compute instance 3044 can communicatively couple the app subnet 3026 of the data plane mirror app tier 3040 to the app subnet 3026, which can be included in the data plane app tier 3046.

[0375] Data plane VCN 3018 can include a data plane app tier 3046, a data plane DMZ tier 3048, and a data plane data tier 3050. Data plane DMZ tier 3048 can include an LB subnet 3022 that can be communicatively coupled to an app subnet 3026 of the data plane app tier 3046 and an internet gateway 3034 of the data plane VCN 3018. App subnet 3026 can be communicatively coupled to a service gateway 3036 of the data plane VCN 3018 and a NAT gateway 3038 of the data plane VCN 3018. Data plane data tier 3050 can also include a DB subnet 3030 that can be communicatively coupled to the app subnet 3026 of the data plane app tier 3046.

[0376] The internet gateways 3034 of the control plane VCN 3016 and the data plane VCN 3018 may be communicatively coupled to a metadata management service 3052, which may be communicatively coupled to the public internet 3054. The public internet 3054 may be communicatively coupled to NAT gateways 3038 of the control plane VCN 3016 and the data plane VCN 3018. The service gateways 3036 of the control plane VCN 3016 and the data plane VCN 3018 may be communicatively coupled to cloud services 3056.

[0377] In some examples, service gateway 3036 of control plane VCN 3016 or data plane VCN 3018 can make application programming interface (API) calls to cloud services 3056 without traversing the public internet 3054. API calls from service gateway 3036 to cloud services 3056 can be one-way, i.e., service gateway 3036 can make API calls to cloud services 3056 and cloud services 3056 can send requested data to service gateway 3036. However, cloud services 3056 cannot initiate API calls to service gateway 3036.

[0378] In some examples, secure host tenancy 3004 can connect directly to service tenancy 3019, which may be otherwise separate. Secure host subnet 3008 can communicate with SSH subnet 3014 through LPG 3010, which may enable bidirectional communication through otherwise separate systems. Connecting secure host subnet 3008 to SSH subnet 3014 allows secure host subnet 3008 to access other entities in service tenancy 3019.

[0379] The control plane VCN 3016 can enable users of the service tenancy 3019 to set up or otherwise provision desired resources. The desired resources provisioned in the control plane VCN 3016 can be deployed or otherwise used in the data plane VCN 3018. In some examples, the control plane VCN 3016 can be separate from the data plane VCN 3018, and the data plane mirror app tier 3040 of the control plane VCN 3016 can communicate with the data plane app tier 3046 of the data plane VCN 3018 via a VNIC 3042, which can be contained within the data plane mirror app tier 3040 and the data plane app tier 3046.

[0380] In some examples, a user or customer of the system may make a request, such as a create, read, update, or delete (CRUD) operation, over the public Internet 3054, which may communicate the request to the metadata management service 3052. The metadata management service 3052 may communicate the request to the control plane VCN 3016 through the Internet gateway 3034. The request may be received by the LB subnet 3022 contained within the control plane DMZ tier 3020. The LB subnet 3022 may determine that the request is valid, and in response to this determination, the LB subnet 3022 may send the request to the app subnet 3026 contained within the control plane app tier 3024. If the request is validated and requires a call to the public Internet 3054, the call to the public Internet 3054 may be sent to the NAT gateway 3038, which may make the call to the public Internet 3054. Metadata that may be desired to be stored by the request may be stored in the DB subnet 3030.

[0381] In some examples, data plane mirror app tier 3040 can facilitate direct communication between control plane VCN 3016 and data plane VCN 3018. For example, it may be desirable for a change, update, or other appropriate modification to a configuration to be applied to resources contained within data plane VCN 3018. Via VNIC 3042, control plane VCN 3016 can communicate directly with resources contained within data plane VCN 3018, thereby performing the configuration change, update, or other appropriate modification on the resources contained within data plane VCN 3018.

[0382] In some embodiments, the control plane VCN 3016 and the data plane VCN 3018 can be included in the service tenancy 3019. In this case, a user or customer of the system may not own or operate either the control plane VCN 3016 or the data plane VCN 3018. Instead, an IaaS provider may own or operate the control plane VCN 3016 and the data plane VCN 3018, both of which may be included in the service tenancy 3019. This embodiment may enable network isolation that can prevent users or customers from interacting with the resources of other users or customers. This embodiment may also enable users or customers of the system to store databases privately without having to rely on the public internet 3054, which may not have the desired level of threat protection for storage.

[0383] In another embodiment, LB subnet 3022 contained within control plane VCN 3016 can be configured to receive signals from service gateway 3036. In this embodiment, control plane VCN 3016 and data plane VCN 3018 can be configured to be called by customers of the IaaS provider without calling the public internet 3054. Customers of the IaaS provider may desire this embodiment because databases used by the customers can be stored in service tenancy 3019, which can be controlled by the IaaS provider and isolated from the public internet 3054.

[0384] Figure 31 is a block diagram 3100 illustrating another example pattern of an IaaS architecture, according to at least one embodiment. A service operator 3102 (e.g., service operator 3002 of Figure 30) can be communicatively coupled to a secure host tenancy 3104 (e.g., secure host tenancy 3004 of Figure 30), which can include a virtual cloud network (VCN) 3106 (e.g., VCN 3006 of Figure 30) and a secure host subnet 3108 (e.g., secure host subnet 3008 of Figure 30). VCN 3106 can include a local peering gateway (LPG) 3110 (e.g., LPG 3010 of Figure 30), which can be communicatively coupled to a secure shell (SSH) VCN 3112 (e.g., SSH VCN 3012 of Figure 30) via LPG 3010 included in SSH VCN 3112. SSH VCN 3112 can include SSH subnet 3114 (e.g., SSH subnet 3014 in FIG. 30), and SSH VCN 3112 can be communicatively coupled to control plane VCN 3116 (e.g., control plane VCN 3016 in FIG. 30) via LPG 3110 included in control plane VCN 3116. Control plane VCN 3116 can be included in service tenancy 3119 (e.g., service tenancy 3019 in FIG. 30), and data plane VCN 3118 (e.g., data plane VCN 3018 in FIG. 30) can be included in customer tenancy 3121, which can be owned or operated by a user or customer of the system.

[0385] The control plane VCN 3116 may include a control plane DMZ tier 3120 (e.g., control plane DMZ tier 3020 of FIG. 30) that may include a LB subnet 3122 (e.g., LB subnet 3022 of FIG. 30), a control plane app tier 3124 (e.g., control plane app tier 3024 of FIG. 30) that may include an app subnet 3126 (e.g., app subnet 3026 of FIG. 30), and a control plane data tier 3128 (e.g., control plane data tier 3028 of FIG. 30) that may include a database (DB) subnet 3130 (e.g., similar to DB subnet 3030 of FIG. 30). LB subnet 3122 included in control plane DMZ tier 3120 can be communicatively coupled to app subnet 3126 included in control plane app tier 3124 and to internet gateway 3134 (e.g., internet gateway 3034 in FIG. 30 ) which may be included in control plane VCN 3116, and app subnet 3126 can be communicatively coupled to DB subnet 3130 and to service gateway 3136 (e.g., service gateway 3036 in FIG. 30 ) and network address translation (NAT) gateway 3138 (e.g., NAT gateway 3038 in FIG. 30 ) included in control plane data tier 3128. Control plane VCN 3116 can include service gateway 3136 and NAT gateway 3138.

[0386] Control plane VCN 3116 can include a data plane mirror app tier 3140 (e.g., data plane mirror app tier 3040 of FIG. 30 ), which can include an app subnet 3126. App subnet 3126 included in data plane mirror app tier 3140 can include a virtual network interface controller (VNIC) 3142 (e.g., VNIC 3042) on which computer instance 3144 (e.g., similar to computer instance 3044 of FIG. 30 ) can run. Computer instance 3144 can facilitate communication between app subnet 3126 of data plane mirror app tier 3140 and app subnet 3126, which can be included in data plane app tier 3146 (e.g., data plane app tier 3046 of FIG. 30 ), via VNIC 3142 included in data plane mirror app tier 3140 and VNIC 3142 included in data plane app tier 3146.

[0387] The internet gateway 3134 contained within the control plane VCN 3116 can be communicatively coupled to a metadata management service 3152 (e.g., metadata management service 3052 of FIG. 30), which can be communicatively coupled to the public internet 3154 (e.g., public internet 3054 of FIG. 30). The public internet 3154 can be communicatively coupled to a NAT gateway 3138 contained within the control plane VCN 3116. The service gateway 3136 contained within the control plane VCN 3116 can be communicatively coupled to cloud services 3156 (e.g., cloud services 3056 of FIG. 30).

[0388] In some examples, data plane VCN 3118 can be included in customer tenancy 3121. In this case, an IaaS provider can provide a control plane VCN 3116 for each customer, and the IaaS provider can set up a unique compute instance 3144 for each customer that is contained within service tenancy 3119. Each compute instance 3144 can enable communication between the control plane VCN 3116 contained within service tenancy 3119 and the data plane VCN 3118 contained within customer tenancy 3121. The compute instance 3144 can enable resources provisioned in the control plane VCN 3116 contained within service tenancy 3119 to be deployed or otherwise used in the data plane VCN 3118 contained within customer tenancy 3121.

[0389] In another example, a customer of the IaaS provider may have a database that resides in customer tenancy 3121. In this example, control plane VCN 3116 may include data plane mirror app tier 3140, which may include app subnet 3126. Data plane mirror app tier 3140 may reside in data plane VCN 3118, but data plane mirror app tier 3140 may not reside in data plane VCN 3118. That is, data plane mirror app tier 3140 may have access to customer tenancy 3121, but data plane mirror app tier 3140 may not reside in data plane VCN 3118 or be owned or operated by the IaaS provider's customer. Data plane mirror app tier 3140 may be configured to make calls to data plane VCN 3118, but may not be configured to make calls to any entities contained within control plane VCN 3116. A customer may wish to deploy or otherwise use resources in data plane VCN 3118 that are provisioned in control plane VCN 3116, and data plane mirror app tier 3140 can facilitate the deployment or other use of the customer's desired resources.

[0390] In some examples, an IaaS provider's customer can apply filters to the data plane VCN 3118. In this embodiment, the customer can determine what the data plane VCN 3118 can access, and the customer can restrict access from the data plane VCN 3118 to the public internet 3154. The IaaS provider may not be able to filter or otherwise control the data plane VCN 3118's access to any external networks or databases. The customer's application of filters and control to the data plane VCN 3118 contained within the customer tenancy 3121 can help isolate the data plane VCN 3118 from other customers and the public internet 3154.

[0391] In some embodiments, cloud services 3156 can make calls through service gateway 3136 to access services that may not reside on the public internet 3154, control plane VCN 3116, or data plane VCN 3118. The connection between cloud services 3156 and control plane VCN 3116 or data plane VCN 3118 may not be live or continuous. Cloud services 3156 may reside on different networks owned or operated by the IaaS provider. Cloud services 3156 can be configured to receive calls from service gateway 3136 and not receive calls from the public internet 3154. Some cloud services 3156 may be isolated from other cloud services 3156, and control plane VCN 3116 may be isolated from cloud services 3156 that may not be in the same region as control plane VCN 3116. For example, control plane VCN 3116 may be located in "Region 1," and cloud service "deployment 30" may be located in Region 1 and Region 2. If a call to deployment 30 is made by a service gateway 3136 contained within control plane VCN 3116 located in Region 1, the call may be sent to deployment 30 in Region 1. In this example, control plane VCN 3116 or deployment 30 in Region 1 may not be communicatively coupled to or otherwise communicate with deployment 30 in Region 2.

[0392] Figure 32 is a block diagram 3200 illustrating another example pattern of an IaaS architecture, according to at least one embodiment. A service operator 3202 (e.g., service operator 3002 of Figure 30) can be communicatively coupled to a secure host tenancy 3204 (e.g., secure host tenancy 3004 of Figure 30), which can include a virtual cloud network (VCN) 3206 (e.g., VCN 3006 of Figure 30) and a secure host subnet 3208 (e.g., secure host subnet 3008 of Figure 30). VCN 3206 can include an LPG 3210 (e.g., LPG 3010 of Figure 30), which can be communicatively coupled to an SSH VCN 3212 (e.g., SSH VCN 3012 of Figure 30) via LPG 3210 included in SSH VCN 3212. SSH VCN 3212 can include SSH subnet 3214 (e.g., SSH subnet 3014 in FIG. 30), which can be communicatively coupled to control plane VCN 3216 (e.g., control plane VCN 3016 in FIG. 30) via LPG 3210 included in control plane VCN 3216, and can be communicatively coupled to data plane VCN 3218 (e.g., data plane VCN 3018 in FIG. 30) via LPG 3210 included in data plane VCN 3218. Control plane VCN 3216 and data plane VCN 3218 can be included in service tenancy 3219 (e.g., service tenancy 3019 in FIG. 30).

[0393] The control plane VCN 3216 may include a control plane DMZ tier 3220 (e.g., control plane DMZ tier 3020 of FIG. 30 ) that may include a load balancer (LB) subnet 3222 (e.g., LB subnet 3022 of FIG. 30 ), a control plane app tier 3224 (e.g., control plane app tier 3024 of FIG. 30 ) that may include an app subnet 3226 (e.g., similar to app subnet 3026 of FIG. 30 ), and a control plane data tier 3228 (e.g., control plane data tier 3028 of FIG. 30 ) that may include a DB subnet 3230. LB subnet 3222 included in control plane DMZ tier 3220 can be communicatively coupled to app subnet 3226 included in control plane app tier 3224 and to internet gateway 3234 (e.g., internet gateway 3034 in FIG. 30 ) which may be included in control plane VCN 3216, and app subnet 3226 can be communicatively coupled to DB subnet 3230 and to service gateway 3236 (e.g., service gateway in FIG. 30 ) and network address translation (NAT) gateway 3238 (e.g., NAT gateway 3038 in FIG. 30 ) included in control plane data tier 3228. Control plane VCN 3216 can include service gateway 3236 and NAT gateway 3238.

[0394] Data plane VCN 3218 may include a data plane app tier 3246 (e.g., data plane app tier 3046 in FIG. 30 ), a data plane DMZ tier 3248 (e.g., data plane DMZ tier 3048 in FIG. 30 ), and a data plane data tier 3250 (e.g., data plane data tier 3050 in FIG. 30 ). Data plane DMZ tier 3248 may include a LB subnet 3222 that may be communicatively coupled to a trusted app subnet 3260 and an untrusted app subnet 3262 of data plane app tier 3246 and to an Internet gateway 3234 contained within data plane VCN 3218. Trusted app subnet 3260 may be communicatively coupled to a service gateway 3236 contained within data plane VCN 3218, a NAT gateway 3238 contained within data plane VCN 3218, and a DB subnet 3230 contained within data plane data tier 3250. Untrusted app subnet 3262 can be communicatively coupled to a service gateway 3236 contained within data plane VCN 3218 and to a DB subnet 3230 contained within data plane data tier 3250. Data plane data tier 3250 can include DB subnet 3230 that can be communicatively coupled to a service gateway 3236 contained within data plane VCN 3218.

[0395] The untrusted app subnet 3262 can include one or more primary VNICs 3264(1)-(N) that can be communicatively coupled to tenant virtual machines (VMs) 3266(1)-(N). Each tenant VM 3266(1)-(N) can be communicatively coupled to a respective app subnet 3267(1)-(N), which can be contained within a respective container egress VCN 3268(1)-(N), which can be contained within a respective customer tenancy 3270(1)-(N). Each secondary VNIC 3272(1)-(N) can facilitate communication between the untrusted app subnet 3262 contained within the data plane VCN 3218 and the app subnet contained within the container egress VCN 3268(1)-(N). Each container egress VCN 3268(1)-(N) may include a NAT gateway 3238 that may be communicatively coupled to the public Internet 3254 (e.g., public Internet 3054 in FIG. 30).

[0396] The internet gateway 3234 contained within the control plane VCN 3216 and the internet gateway 3234 contained within the data plane VCN 3218 can be communicatively coupled to a metadata management service 3252 (e.g., metadata management service 3052 of FIG. 30 ), which can be communicatively coupled to the public internet 3254. The public internet 3254 can be communicatively coupled to a NAT gateway 3238 contained within the control plane VCN 3216 and the NAT gateway 3238 contained within the data plane VCN 3218. The service gateway 3236 contained within the control plane VCN 3216 and the service gateway 3236 contained within the data plane VCN 3218 can be communicatively coupled to cloud services 3256.

[0397] In some embodiments, data plane VCN 3218 can be integrated with customer tenancy 3270. This integration can be useful or desirable for an IaaS provider's customer in some cases, such as when they may want support when executing code. A customer may provide code for execution that may be disruptive, communicate with other customer resources, or otherwise cause undesirable effects. In response, the IaaS provider can determine whether to execute the code provided to the IaaS provider by the customer.

[0398] In some examples, a customer of an IaaS provider can grant temporary network access to the IaaS provider and request a function to be attached to data plane app tier 3246. The code to execute the function can run within VMs 3266(1)-(N), and the code may not be configured to run anywhere else on data plane VCN 3218. Each VM 3266(1)-(N) can be connected to one customer tenancy 3270. Each container 3271(1)-(N) contained within VM 3266(1)-(N) can be configured to execute code. In this case, there may be double isolation (e.g., containers 3271(1)-(N) executing code, where containers 3271(1)-(N) may be contained within VMs 3266(1)-(N) that are contained within at least untrusted app subnet 3262), which may help prevent incorrect or otherwise unwanted code from damaging the IaaS provider's network or the network of a different customer. Containers 3271(1)-(N) may be communicatively coupled to customer tenancy 3270 and configured to send or receive data from customer tenancy 3270. Containers 3271(1)-(N) may not be configured to send or receive data from any other entity in data plane VCN 3218. Once code execution is complete, the IaaS provider may stop or otherwise destroy containers 3271(1)-(N).

[0399] In some embodiments, trusted app subnet 3260 can execute code that may be owned or operated by the IaaS provider. In this embodiment, trusted app subnet 3260 can be communicatively coupled to DB subnet 3230 and configured to perform CRUD operations on DB subnet 3230. Untrusted app subnet 3262 can be communicatively coupled to DB subnet 3230, but in this embodiment, the untrusted app subnet can be configured to perform read operations on DB subnet 3230. Containers 3271(1)-(N), which may be contained within each customer's VMs 3266(1)-(N) and may execute code from the customer, may not be communicatively coupled to DB subnet 3230. ...

Claims

1. 1. A method comprising: providing one or more cloud service provider (CSP)-supplied cloud services to one or more customers of the CSP using a first portion of the CSP-supplied infrastructure in a first region; creating a first virtual private label cloud (vPLC) for a first reseller based on the CSP-provided infrastructure, the first vPLC including allocating a second portion of the CSP-provided infrastructure to the first vPLC; using the first vPLC to provide cloud services provided by one or more first resellers to one or more customers of the first resellers; configuring identity management for the CSP based on a CSP-provided infrastructure; configuring identity management for the first reseller based on the CSP-provided infrastructure within a region; creating identity information associated with a customer of the CSP within a first namespace; creating identity information associated with the first reseller within a second namespace; performing identity management functions for the customer of the CSP using the identity information associated with the customer of the CSP; performing identity management functions for users of the first reseller using the identity information associated with the first reseller; and A method comprising:

2. The method of claim 1 , wherein the identity management functionality for the customer of the CSP is performed by a first identity services stack that includes a first set of resources of the CSP-provided infrastructure.

3. The method of claim 2 , wherein the identity management functionality for the users of the first reseller is performed by the first identity services stack.

4. 4. The method of claim 2 or claim 3, wherein the identity management functionality for the users of the first reseller is performed by a second identity services stack that includes a second set of resources of the CSP-provided infrastructure.

5. The method of claim 4 , wherein the second identity service stack is a clone of the first identity service stack.

6. 10. The method of any preceding claim, wherein creating the identity information associated with the first reseller is performed by the first identity service stack.

7. 10. The method of any preceding claim, wherein creating the identity information associated with the first reseller is performed by the second identity service stack.

8. configuring the identity management for a second reseller of the CSP based on the CSP-provided infrastructure; creating identity information associated with the second reseller within a third namespace; 10. The method of any preceding claim, further comprising:

9. 10. The method of any preceding claim, wherein the identity information associated with the first reseller includes identity information for users of the first reseller and users of customers of the first reseller.

10. A non-transitory computer-readable medium storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform operations, the operations including: providing one or more cloud service provider (CSP)-supplied cloud services to one or more customers of the CSP using a first portion of the CSP-supplied infrastructure in a first region; creating a first virtual private label cloud (vPLC) for a first reseller based on the CSP-provided infrastructure, the first vPLC including allocating a second portion of the CSP-provided infrastructure to the first vPLC; using the first vPLC to provide cloud services provided by one or more first resellers to one or more customers of the first resellers; configuring identity management for the CSP based on a CSP-provided infrastructure; configuring identity management for the first reseller based on the CSP-provided infrastructure within a region; creating identity information associated with a customer of the CSP within a first namespace; creating identity information associated with the first reseller within a second namespace; performing identity management functions for the customer of the CSP using the identity information associated with the customer of the CSP; performing identity management functions for users of the first reseller using the identity information associated with the first reseller; and 1. A non-transitory computer-readable medium comprising:

11. 11. The non-transitory computer-readable medium of claim 10, wherein the identity management functionality for the customer of the CSP is performed by a first identity services stack that includes a first set of resources of the CSP-provided infrastructure.

12. 12. The non-transitory computer-readable medium of claim 11, wherein the identity management functionality for the users of the first reseller is performed by the first identity services stack.

13. 13. The non-transitory computer-readable medium of claim 11 or claim 12, wherein the identity management functionality for the users of the first reseller is performed by a second identity services stack that includes a second set of resources of the CSP-provided infrastructure.

14. 14. The non-transitory computer-readable medium of claim 13, wherein creating the identity information associated with the first reseller is performed by the first identity service stack.

15. The non-transitory computer-readable medium of any of claims 10 to 14, wherein creating the identity information associated with the first reseller is performed by the second identity service stack.

16. 16. The non-transitory computer-readable medium of claim 10, wherein the identity information associated with the first reseller includes identity information for users of the first reseller and users of customers of the first reseller.

17. 1. A system comprising: one or more processors; one or more memories storing computer-executable instructions; Equipped with The computer-executable instructions, when executed by the one or more processors, cause the system to: providing one or more cloud service provider (CSP)-supplied cloud services to one or more customers of the CSP using a first portion of the CSP-supplied infrastructure in a first region; creating a first virtual private label cloud (vPLC) for a first reseller based on the CSP-provided infrastructure, the first vPLC including allocating a second portion of the CSP-provided infrastructure to the first vPLC; using the first vPLC to provide cloud services provided by one or more first resellers to one or more customers of the first resellers; configuring identity management for the CSP based on a CSP-provided infrastructure; configuring identity management for the first reseller based on the CSP-provided infrastructure within a region; creating identity information associated with a customer of the CSP within a first namespace; creating identity information associated with the first reseller within a second namespace; performing identity management functions for the customer of the CSP using the identity information associated with the customer of the CSP; performing identity management functions for users of the first reseller using the identity information associated with the first reseller; and A system that allows the following to be performed.

18. 20. The system of claim 17, wherein the identity management functionality for the customer of the CSP is performed by a first identity services stack that includes a first set of resources of the CSP-provided infrastructure.

19. 20. The system of claim 18, wherein the identity management functions for the users of the first reseller are performed by the first identity services stack.

20. 20. The system of claim 18 or claim 19, wherein the identity management functionality for the users of the first reseller is performed by a second identity services stack that includes a second set of resources of the CSP-provided infrastructure.

Images