Region Seed Settings

Abstract

A framework for configuring a new region and / or a new realm is disclosed. For example, a technique for configuring a new region and / or a new realm includes receiving a request to configure a new data center in a region of a cloud service infrastructure, the request including a configuration for the new data center, and the technique further includes determining a set of one or more services provided by the new data center based at least in part on the configuration, determining whether reference seed instructions corresponding to the set of one or more services are stored based at least in part on the set of one or more services, generating a set of seed instructions for provisioning the set of one or more services in the new data center based on whether reference seed instructions corresponding to the set of one or more services are stored, and providing a set of seed instructions for provisioning the new data center to provide the set of one or more services.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of U.S. Provisional Patent Application No. 63 / 315,001, entitled "Region Seed Establishment," filed February 28, 2022, U.S. Provisional Patent Application No. 63 / 308,003, entitled "Techniques for Bootstrapping Region Build," filed February 8, 2022, U.S. Provisional Patent Application No. 63 / 312,814, entitled "Techniques for Implementing Virtual Data Centers," filed February 22, 2022, and U.S. Nonprovisional Patent Application No. 18 / 163,671, entitled "Region Seed Establishment," filed February 2, 2023, the contents of which are incorporated by reference in their entireties herein for all purposes.

[0002] Field The present disclosure relates to region seed settings. [Background technology]

[0003] background Cloud service providers (CSPs) use different systems and infrastructure services to provide a variety of services on demand to users or customers. CSPs provide infrastructure services that customers can use to build their own networks and deploy customer resources. Each customer can have one or more defined realms and can be served by one or more regions. Summary of the Invention [Problem to be solved by the invention]

[0004] In the traditional approach, setting up a new realm and possibly a new region for a customer requires a member of the CSP to program the new realm for the customer. Due to the size of the realm and the complexity of the coding, the procedure for setting up a new realm is very time intensive. To speed up this process, it is common practice to copy the programming of the old realm to the new realm, but this often results in unnecessary resources being set up in the new realm.

[0005] overview The present disclosure generally relates to a framework for setting up new regions and / or realms within a CSP infrastructure. Various embodiments are described herein, including methods, systems, non-transitory computer-readable storage media storing programs, codes or instructions executable by one or more processors, and the like. These exemplary embodiments are mentioned not to limit or define the disclosure, but to provide examples to aid in the understanding of the disclosure. Additional embodiments are described in the detailed description section, where further description is provided. [Means for solving the problem]

[0006] One aspect of the present disclosure is directed to a method for configuring a new region or new realm, which may include identifying a configuration for the new region or new realm, generating seed instructions for provisioning the new region or new realm based on the configuration, and provisioning resources for the new region or new realm.

[0007] Another aspect of the disclosure is directed to one or more non-transitory computer-readable media having instructions stored thereon that, when executed by one or more processors, cause a system to perform a process including receiving a request to configure a new datacenter in a region of a cloud service infrastructure, the request including a configuration for the new datacenter. The process further includes determining a set of one or more services to be provided by the new datacenter based at least in part on the configuration, and determining seed data for a set of seed instructions for provisioning the set of one or more services in the new datacenter based at least in part on the set of one or more services. The process further includes generating a set of seed instructions for provisioning the set of one or more services in the new datacenter based at least in part on the seed data, and providing a set of seed instructions for provisioning the new datacenter with the seed infrastructure to provide the set of one or more services.

[0008] Another aspect of the disclosure is directed to a method that includes receiving a request to configure a new data center in a region of a cloud services infrastructure, the request including a configuration for the new data center. The method further includes determining a set of one or more services to be provided by the new data center based at least in part on the configuration and determining, at least in part on the set of one or more services, whether reference seed instructions corresponding to the set of one or more services are stored. The method further includes generating a set of seed instructions for provisioning the set of one or more services in the new data center based on whether reference seed instructions corresponding to the set of one or more services are stored and providing the set of seed instructions for provisioning the new data center to provide the set of one or more services.

[0009] Another aspect of the disclosure is directed to a system including a memory storing a configuration for a data center and a processor receiving a request to configure a new data center within a region of a cloud service infrastructure, the request including the configuration for the new data center. The processor further stores the configuration for the new data center in the memory, determines a set of one or more services provided by the new data center based at least in part on the configuration, and determines seed data for a set of seed instructions for provisioning the set of one or more services in the new data center based at least in part on the set of one or more services. The processor further generates a set of seed instructions based at least in part on the seed data to provide the set of seed instructions for provisioning the set of one or more services.

[0010] The above, together with other features and embodiments, will become more apparent with reference to the following description, claims and accompanying drawings.

[0011] The features, embodiments and advantages of the present disclosure will become better understood from the following Detailed Description when read in conjunction with the accompanying drawings. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a block diagram illustrating an environment in which a Cloud Infrastructure Orchestration Service (CIOS) can operate to dynamically provide bootstrap services in a region, according to at least one embodiment. [Diagram 2] FIG. 1 is a block diagram illustrating an environment and method for building a virtual bootstrap environment (ViBE) according to at least one embodiment. [Diagram 3] 1 is a block diagram illustrating an environment and method for bootstrapping a service into a target region using ViBE, according to at least one embodiment. [Figure 4] FIG. 2 illustrates a block diagram of an example seed generator mechanism according to at least one embodiment. [Diagram 5] FIG. 1 illustrates a block diagram of another example seed generator mechanism according to at least one embodiment. [Figure 6] FIG. 1 illustrates an example procedure for provisioning a seed infrastructure, according to at least one embodiment. [Figure 7] FIG. 1 illustrates an example procedure for provisioning seed infrastructure based on whether a reference seed infrastructure creation instruction exists for a region, according to at least one embodiment. [Figure 8] FIG. 1 is a block diagram illustrating one pattern for implementing a cloud infrastructure as a service system, according to at least one embodiment. [Figure 9] FIG. 1 is a block diagram illustrating another pattern for implementing a cloud infrastructure as a service system, in accordance with at least one embodiment. [Figure 10] FIG. 1 is a block diagram illustrating another pattern for implementing a cloud infrastructure as a service system, in accordance with at least one embodiment. [Figure 11] FIG. 1 is a block diagram illustrating another pattern for implementing a cloud infrastructure as a service system, in accordance with at least one embodiment. [Figure 12] FIG. 1 is a block diagram illustrating an example of a computer system in accordance with at least one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Detailed Description In the following description, for purposes of explanation, specific details are set forth in order to provide a thorough understanding of certain embodiments. It will be understood, however, that various embodiments can be practiced without these specific details. The figures and descriptions are not intended to be limiting. The term "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any embodiment or design described herein as "exemplary" should not necessarily be construed as preferred or advantageous over other embodiments or designs.

[0014] Example Data Center Construction (Region Construction) Infrastructure Recently, there has been an exponential increase in the adoption of cloud services. Currently, various types of cloud services are offered by various different cloud service providers (CSPs). The term cloud service is commonly used to refer to services or functionality provided on-demand (e.g., via a subscription model) by a CSP to a user or customer using systems and infrastructure (cloud infrastructure) provided by the CSP. Typically, the servers and systems that make up the CSP's infrastructure and that are used to provide cloud services to customers are separate from the customer's own on-premise services and systems. Thus, customers can utilize cloud services provided by the CSP without having to purchase separate hardware and software resources for the service. Cloud services are designed to give subscribing customers easy, scalable, on-demand access to applications and computing resources without the customer having to invest in procuring the infrastructure used to provide the service or functionality. Various different types or models of cloud services may be offered, such as Software-as-a-Service (SaaS), Platform-as-a-Service (PaaS), Infrastructure-as-a-Service (IaaS), etc. A customer may subscribe to one or more cloud services offered by a CSP. A customer may be any entity, such as an individual, an organization, or a business.

[0015] As mentioned above, a CSP is responsible for providing the infrastructure and resources used to provide cloud services to subscribing customers. The resources provided by a CSP may include both hardware and software resources. These resources may include, for example, computational resources (e.g., virtual machines, containers, applications, processors), memory resources (e.g., databases, data stores), networking resources (e.g., routers, host machines, load balancers), identity and other resources. In an embodiment, the resources provided by a CSP to provide a set of cloud services CSP are organized as a data center. A data center can be configured to provide a particular set of cloud services. The CSP is responsible for equipping the data center with the infrastructure and resources used to provide that particular set of cloud services. A CSP may build one or more data centers.

[0016] Data centers offered by a CSP can be hosted in different regions. A region is a localized geographic area and can be identified by a region name. Regions are generally independent of each other and can be separated by great distances, such as across multiple countries or even continents. Regions are grouped into realms. Examples of CSP regions may include US West, US East, Australia East, Australia Southeast, etc.

[0017] A region may include one or more data centers, and the data centers may be located within a geographic area corresponding to the region. As an example, the data centers in a region may be located in a city within the region. For example, for a particular CSP, a data center in the US West region may be located in San Jose, California, a data center in the US East region may be located in Ashburn, Virginia, a data center in the Australia East region may be located in Sydney, Australia, a data center in the Australia Southeast region may be located in Melbourne, Australia, etc.

[0018] Data centers within a region can be organized into one or more availability domains, which are used for high availability and disaster recovery purposes. An availability domain can contain one or more data centers within a region. Availability domains within a region are isolated from each other, fault tolerant, and designed in a manner that makes it highly unlikely that data centers in multiple availability domains will fail simultaneously. For example, availability domains within a region can be built in a manner that makes it highly unlikely that a failure in one availability domain within a region will affect the availability of data centers in other availability domains in the same region.

[0019] When a customer or subscriber subscribes or signs up for one or more services offered by a CSP, the CSP creates a tenancy for the customer. A tenancy is like an account created for a customer. In one embodiment, the customer's tenancy exists in a single realm and has access to all regions that belong to that realm. The customer's users can then access the services to which the customer subscribes under this tenancy.

[0020] As described above, a CSP builds or deploys a data center to provide cloud services to the CSP's customers. As the CSP's customer base grows, the CSP typically builds a new data center in a new region or expands the capacity of an existing data center to accommodate the growing demand of the customers and to better serve the customers. Preferably, the data center is built in close geographic proximity to the location of the customers served by the data center. The geographical proximity of a data center to the customers served by the data center facilitates more efficient use of resources and faster and more reliable service to the customers. Thus, a CSP typically builds a new data center in a new region in a geographic area that is geographically close to the customers served by the data center. For example, for a growing customer base in Germany, the CSP may build one or more data centers in a new region in Germany.

[0021] Building a data center (or multiple data centers) in a region may also be referred to as building a region. The term "region construction" is used to refer to building one or more data centers in a region. Building a data center in a region involves provisioning or creating a new set of resources that are needed or used to provide the set of services that the data center is configured to provide. The end result of the region construction process is the creation of a data center in the region that is capable of providing the set of services targeted to that data center and includes the set of resources used to provide the set of services.

[0022] Building a new data center in a region is a highly complex task that requires extensive coordination between various bootstrapping activities. Broadly speaking, it requires the execution and coordination of various tasks, such as identifying the set of services to be provided by the data center, identifying the various resources required to provide the set of services, creating, provisioning and deploying the identified resources, and properly connecting the resources so that they can be used in the intended manner. Each of these tasks has further subtasks that require coordination, which further increases the complexity. Due to this complexity, currently, building a data center in a region involves several manual initiation or manual control tasks that require careful manual coordination. As a result, the task of building a new region (i.e., building one or more data centers in a region) is very time-consuming. It can take time, for example, several months, to build a data center. Moreover, this process is highly error-prone and may require several re-runs before the desired configuration of the data center is achieved, thereby further increasing the time required to build the data center. These limitations and problems significantly limit the ability of CSPs to scale computing resources in a timely manner to meet growing customer needs.

[0023] This disclosure describes techniques for reducing construction time, reducing wasteful use of computing resources, and reducing the risks associated with constructing one or more data centers in a region. Using the techniques described herein, new data centers can be constructed in a region in a relatively much shorter time than the weeks and months required to construct a data center in a traditional region, while reducing the risk of error over traditional approaches.

[0024] Disclosed herein is a cloud infrastructure orchestration service (CIOS) configured to bootstrap (e.g., provision and deploy) services to a new datacenter based on a predefined configuration file that identifies resources (e.g., infrastructure components and software to be deployed) to implement a given change to the datacenter. The CIOS can parse and analyze the configuration file (e.g., flock config) to identify dependencies between resources, execution targets, phases, and flocks. The CIOS can generate specific data structures from the analysis and can use these data structures to drive operations and to manage the order in which services are bootstrapped into regions. The CIOS can use these data structures to identify when the CIOS can bootstrap services, when bootstrapping is blocked, and / or when bootstrapping operations associated with previously blocked services can be resumed. Advantageously, the CIOS can identify circular dependencies in the data structures and perform operations to remove / resolve those circular dependencies prior to task execution. Using these techniques, the CIOS substantially reduces the risk of executing tasks before the resources they depend on are available.

[0025] Using the techniques disclosed herein, CIOS can optimize the parallelism of executing changes to datacenters while ensuring that tasks are not initiated until the functionality they depend on is available in the region. In this way, CIOS allows region construction to be performed more efficiently, thereby significantly reducing the time it takes to construct a datacenter and the wasted computing resources seen in traditional approaches.

[0026] Certain definitions A "region" is a logical abstraction that corresponds to a geographic location. A region may include any suitable number of one or more execution targets. In an embodiment, an execution target may correspond to a data center.

[0027] An "Execution Target" is the smallest unit of change for executing a release. A "Release" is an expression of intent to orchestrate a particular change to a service (e.g., deploy version 8, "add internal DNS record", etc.). For most services, an execution target represents an "instance" of the service. A single service can be bootstrapped onto one or more execution targets each. An execution target can be associated with a set of devices (e.g., a data center).

[0028] "Bootstrapping" is intended to refer to the collective tasks associated with the provisioning and deployment of any suitable number of resources (e.g., infrastructure components, artifacts, etc.) that correspond to a single service.

[0029] "Service" refers to functionality provided by a set of resources. The set of resources for a service includes any suitable combination of infrastructure, platform, or software (e.g., applications) hosted by a cloud provider that can be configured to provide the functionality of the service. Services can be provided to users over the Internet.

[0030] "Artifact" refers to an infrastructure component or code that is deployed to a Kubernetes engine cluster, which may include software (e.g., applications), configuration information for the infrastructure component (e.g., configuration files), etc.

[0031] "Flock config" refers to a configuration file (or set of configuration files) that describes the set of all resources (e.g., infrastructure components and artifacts) associated with a single service. A Flock config may contain declarative statements that specify one or more aspects that correspond to a desired state of the service's resources.

[0032] "Service State" refers to a point-in-time snapshot of all resources (e.g., infrastructure resources, artifacts, etc.) associated with a service. The service state indicates the situation corresponding to the provisioning and / or deployment tasks associated with the service resources.

[0033] IaaS provisioning (or "provisioning") refers to obtaining a computer or virtual host for use and installing the necessary libraries or services on it. The phrase "provisioning a device" refers to making a device available for use by an end user for a specific use. A device that has undergone the provisioning process may be called a "provisioned device." Preparing a provisioned device (installing libraries and daemons) may be part of provisioning. This preparation is distinct from deploying a new application or a new version of an application to a prepared device. In most cases, deployment does not include provisioning, which may need to be done first. After preparation, the device may be called an "infrastructure component."

[0034] IaaS deployment (or "deployment") refers to the process of providing and / or installing an application or a new version of an application onto provisioned infrastructure components. After the infrastructure components are provisioned (e.g., acquired, allocated, prepared, etc.), additional software can be deployed (e.g., provisioned and installed onto the infrastructure components). After provisioning and deployment are complete, the infrastructure components can be referred to as "resources." Examples of resources can include, but are not limited to, virtual machines, databases, object storage, block storage, load balancers, etc.

[0035] A "capability" identifies a unit of functionality associated with a service. A unit may be some or all of the functionality provided by a service. For example, a capability may be exposed that indicates that a resource is available for authorization / authentication processing (e.g., a subset of the functionality provided by the resource). As another example, a capability may be exposed that indicates that the full functionality of the service is available. Capabilities may be used to identify functionality on which a resource or service depends and / or functionality of the resource or service that is available for use.

[0036] A "virtual bootstrap environment" (ViBE) refers to a virtual cloud network that is provisioned in an overlay (e.g., a "host region") of an existing region. Once provisioned, the ViBE is connected to the new region using a communication channel (e.g., an IPSec tunnel VPN). Certain important core services (or "seed" services) can be provisioned in the ViBE, such as a deployment orchestrator, public key infrastructure (PKI) services, etc. These services can provide the capabilities required to bring the hardware online, establish a chain of trust to the new region, and deploy the remaining services in the new region. The use of a virtual bootstrap environment can prevent circular dependencies between bootstrap resources by using resources in the host region. Services can be staged and tested in the ViBE before the physical region (e.g., the target region) is available.

[0037] "Cloud Infrastructure Orchestration Services" (CIOS) may refer to a system configured to manage the provisioning and deployment operations of any suitable number of services as part of a region construction.

[0038] A Multi-Flock Orchestrator (MFO) can be a computing component (e.g., services) that coordinates events among components of the CIOS to provision and deploy services to a target region (e.g., a new region). The MFO tracks relevant events for each service in the region build and takes actions in response to those events.

[0039] "Host Region" refers to a region that hosts a Virtual Bootstrap Environment (ViBE). A host region can be used to bootstrap a ViBE.

[0040] "Target region" refers to the region being constructed. "Exposing a capability" refers to providing an indication that a particular capability is available (or unavailable), through "publishing" as used in "publisher-subscriber" computing design, or otherwise. Capabilities are "exposed" (e.g., collected by a capability service, provided to a capability service, pushed, pulled, etc.) to provide an indication that functionality of a resource / service is available. In one embodiment, capabilities can be exposed / transmitted via events, notifications, data submissions, function calls, API calls, etc. An event (or other notification / data submission, etc.) indicating availability of a particular capability can be broadcast / addressed (e.g., published) to the capability service.

[0041] A "capability service" can be a configured flock that models dependencies between different flocks. Capability services can be provided within a cloud infrastructure orchestration service and can define what capabilities, services, and features are made available in a region.

[0042] A "Real-time Regional Data Distributor" (RRDD) can be a service or system configured to manage regional data that can be populated into a flock config to dynamically create execution targets for new regions.

[0043] Described herein, in certain examples, are techniques for implementing a cloud infrastructure orchestration service (CIOS). Such techniques are configurable to manage the bootstrapping (e.g., provisioning and deployment of software to infrastructure components) in a cloud environment (e.g., a region), as briefly described above. In some cases, the CIOS may include computing components (e.g., CIOS central and CIOS regional, both of which are described in more detail below) that are configurable to manage the bootstrapping tasks (provisioning and deployment) for a given service and a multi-flock orchestrator (also described in more detail below) that is configured to initiate / manage a region build (e.g., a bootstrapping operation corresponding to multiple services).

[0044] CIOS enables region construction and worldwide infrastructure provisioning and code deployment with minimal manual run-time effort from service teams (e.g., beyond initial approval and / or physical transportation of hardware, in some cases). High-level responsibilities of CIOS include, but are not limited to, coordinating region construction, providing users with a view of the current state of resources managed by CIOS (e.g., in a region, across multiple regions, worldwide, etc.), and managing bootstrapping operations for bootstrapping resources within a region.

[0045] CIOS can provide view reconciliation, where a view of a desired state of a resource (e.g., a desired configuration) can be matched with the current / actual state of the resource (e.g., a current configuration). In some cases, view reconciliation can include obtaining state data to determine what resources are actually running and the current configuration and / or state of the resource. Reconciliation can occur at various granularities, such as at the service level.

[0046] The CIOS may perform plan generation, where differences between desired and current states of resources are identified. Part of plan generation may include identifying actions that would need to be performed to bring resources from their current state to the desired state. In some examples, the CIOS may present the generated plan to a user for approval. In these examples, the CIOS may mark the plan as approved or rejected based on user input from the user. Thus, because the plan is generated by a machine, the user may spend less time making judgments about the plan, and the plan may be more accurate. Although the plan is too detailed for human use, the CIOS may provide this data via an advanced user interface (UI).

[0047] In one example, the CIOS can handle change control execution by executing approved plans. Once an execution plan has been created and approved, engineers may not need to be involved in change control unless the CIOS initiates a rollback. The CIOS can handle rollbacks to a previous service version by generating a plan to revert the service to a previous (e.g., pre-release) state (e.g., if the CIOS detects a deterioration in the service health during execution).

[0048] CIOS can measure service health by monitoring alarms and running integration tests. CIOS can help teams quickly define rollback actions in case of service degradation, which CIOS can execute. CIOS can generate and display plans, and can track approvals. CIOS can combine provisioning and deployment functions into a single system that coordinates these tasks across region construction. CIOS also supports discovery of flocks (e.g., service resources such as flock configs corresponding to any suitable number of services), artifacts, resources, and dependencies. CIOS can discover dependencies between execution tasks at any level (e.g., resource level, execution target level, phase level, service level, etc.) by static analysis (e.g., including content parsing and processing) of one or more configuration files. Using these dependencies, CIOS can generate various data structures that can be used to drive task execution (e.g., tasks related to infrastructure resource provisioning and artifact deployment across regions) from these dependencies.

[0049] FIG. 1 is a block diagram of an environment 100 in which a cloud infrastructure orchestration (CIOS) 102 can operate to dynamically provide bootstrap services in a region, according to at least one embodiment. CIOS 102 can include, but is not limited to, the following components: a real-time regional data distributor (RRDD) 104, a multi-flock orchestrator (MFO) 106, a CIOS central 108, a CIOS regional 110, and a capability service 112. The specific functions of CIOS central 108 and CIOS regional 110 are described in more detail in U.S. patent application Ser. No. 17 / 016,754, entitled "Techniques for Deploying Infrastructure Resources with a Declarative Provisioning Tool," which is incorporated in its entirety for all purposes. In an embodiment, any suitable combination of the components of CIOS 102 can be provided as a service. In an embodiment, a portion of CIOS 102 can be deployed to a region (e.g., a data center represented by host region 103). In one embodiment, CIOS 102 may include any suitable number of cloud services (not shown in FIG. 1), as described in more detail in U.S. patent application Ser. No. 17 / 016,754 and with respect to FIGS. 2 and 3 below.

[0050] The real-time regional data distributor (RRDD) 104 can be configured to maintain and provide regional data that identifies realms, regions, execution targets, and availability domains. In some cases, the regional data can be in any suitable form (e.g., JSON format, data object / container, XML, etc.). The regional data maintained by the RRDD 104 can include any suitable number of subsets of data that can be individually referenced by a corresponding identifier. For example, an identifier "all_regions" can be associated with a data structure (e.g., a list, structure, object, etc.) that includes metadata for all defined regions. As another example, an identifier such as "realms" can be associated with a data structure that identifies metadata for a number of realms and a set of regions that correspond to each realm. In general, the regional data can maintain any suitable attributes, such as identifiers, DNS suffixes, state (e.g., region state), etc., for one or more realms, regions, availability domains (ADs), execution targets (ETs), etc. The RRDD 104 can be configured to manage regional state as part of the regional data. The regional state can include any suitable information indicative of the state of bootstrap in the region. For example, some example region states may include "initial," "under construction," "production," "suspended," or "to be decommissioned." The "initial" state may indicate a region that has not yet been bootstrapped. The "under construction" state may indicate that bootstrap of one or more flocks in the region has begun. The "production" state may indicate that bootstrap is complete and the region is ready for validation. The "suspended" state may indicate that CIOS Central 108 or CIOS Regional 110 has suspended internal interaction with the regional stack, possibly due to operational issues. The "decommissioned" state may indicate that the region has been decommissioned and is likely unavailable and / or will not be contacted again in the future.

[0051] CIOS central 108 may be configured to provide any suitable number of user interfaces through which a user (e.g., user 109) may interact with CIOS 102. For example, a user may make changes to region data through a user interface provided by CIOS central 108. CIOS central 108 may further provide various interfaces that allow a user to view changes made to flock configurations and / or artifacts, generate and view plans, approve / reject plans, and view the status of plan execution (e.g., corresponding to tasks involving infrastructure provisioning, deployment, region construction, and / or the desired state of any suitable number of resources managed by CIOS 102). CIOS central 108 may implement a control plane that may be configured to manage any suitable number of CIOS regional 110 instances. CIOS central 108 may provide one or more interfaces for presenting region data, thereby allowing user 109 to view and / or modify the region data. CIOS central 108 may be configured to invoke functions of RRDD 104 through any suitable number of interfaces. In general, CIOS central 108 can be configured to manage region data, either directly or indirectly (e.g., via RRDD 104). CIOS central 108 can be configured to edit the flock config to populate the region data as variables in the flock config.

[0052] Each instance of CIOS regional 110 may correspond to a module configured to perform bootstrapping tasks associated with a single service of the region. CIOS regional 110 may receive desired state data from CIOS central 108. In one embodiment, the desired state data may include a flock config that declares (e.g., by declarative statements) a desired state of resources associated with the service. CIOS central 108 may maintain current state data that indicates any suitable aspect of the current state of resources associated with the service. In one embodiment, CIOS regional 110 may identify that changes need to be made to one or more resources by comparing the desired state data to the current state data. For example, CIOS regional 110 may determine that one or more infrastructure components need to be provisioned, one or more artifacts need to be deployed, or any suitable changes need to be made to the resources of the service to match the desired state. When CIOS regional 110 performs a bootstrap operation, it may expose data that indicates various capabilities of the resources as the capabilities become available. A "capability" identifies a unit of functionality associated with a service. A unit may be some or all of the functionality provided by a service. For example, a capability may be exposed to indicate that a resource is available for authorization / authentication processing (e.g., a subset of the functionality provided by the resource). As another example, a capability may be exposed to indicate that the full functionality of a service is available. Capabilities may be used to identify functionality on which a resource or service depends and / or functionality of the resource or service that is available for use.

[0053] The capability service 112 is configured to maintain capability data that indicates: 1) what capabilities of various services are currently available; 2) whether any resources / services are waiting for a particular capability; 3) what particular resources and / or services are waiting for a given capability; or any suitable combination of the above. The capability service 112 may provide an interface through which the capability data may be requested. The capability service 112 may provide one or more interfaces (e.g., application programming interfaces) that enable the capability service 112 to send capability data to the MFO 106 and / or the CIOS regional 110 (e.g., each instance of the CIOS regional 110). In an embodiment, any suitable component or module of the MFO 106 and / or the CIOS regional 110 may be configured to request the capability data from the capability service 112.

[0054] In one embodiment, a multi-flock orchestrator (MFO) 106 can be configured to drive region construction activities. In one embodiment, the MFO 106 can manage information describing what flocks / flock config versions and / or artifact versions to use to bootstrap a given service in a region (or to make a unit of change to a target region). In one embodiment, the MFO 106 can be configured to monitor (or otherwise be notified of) changes to the region data managed by the real-time regional data distributor 104. In one embodiment, region construction may be triggered by the MFO 106 upon receiving an indication that the region data has changed. In one embodiment, the MFO 106 may collect various flock configs and artifacts used in region construction. Some or all of the flock configs can be configured to be region agnostic. That is, the flock configs may not explicitly specify which flocks are to be bootstrapped into which regions. In one embodiment, the MFO 106 can trigger a data population process whereby the collected flock configs are recompiled (e.g., by the CIOS Central 108). During recompilation, operations may be performed (e.g., by the CIOS Central 108) to cause the region data maintained by the real-time regional data distributor 104 to be populated into the configuration files. The flock configs can reference the region data by variables / parameters, without requiring a hard-coded identity of the region data. The flock configs can use this data population to be dynamically modified at run time, without having the region data hard-coded and therefore more difficult to change.

[0055] The multi-flock orchestrator 106 may perform static flock analysis, in which the flock config is parsed to identify dependencies between resources, execution targets, phases, and flocks, specifically to identify circular dependencies that need to be removed. In an embodiment, the MFO 106 may generate any suitable number of data structures based on the identified dependencies. These data structures (e.g., directed acyclic graphs, linked lists, etc.) may be used by the cloud infrastructure orchestration service 102 to drive operations to perform region construction. For example, these data structures may collectively define the order in which services are bootstrapped within a region. Examples of such data structures are discussed in more detail below in connection with the construction dependency graph 338 of FIG. 3. If circular dependencies (e.g., service A requires service B and vice versa) exist and are identified by the static flock analysis and / or graph, the MFO may be configured to notify any appropriate service teams that changes need to be made to the corresponding flock config to correct these circular dependencies. The MFO 106 may be configured to traverse one or more data structures to manage the order in which services are bootstrapped into regions. The MFO 106 can identify capabilities available within a given region at any given time (e.g., using data obtained from the capabilities service 112). The MFO 106 can use this data to identify when the MFO 106 can bootstrap a service, when bootstrap is blocked, and / or when a bootstrap operation associated with a previously blocked service can be resumed. Based on this scanning, the MFO 106 can perform various releases in which instructions are sent by the MFO 106 to the CIOS central 108 to perform bootstrap operations corresponding to any suitable number of block configurations.In one example, the MFO 106 can be configured to identify that one or more flock configs may require multiple releases due to circular dependencies found in the graph, and as a result, the MFO 106 can send multiple sets of instructions to the CIOS central 108 for a given flock config to break the circular dependencies identified in the graph.

[0056] In one embodiment, a user may request that a new region (e.g., target region 114) be built. This may involve bootstrapping resources corresponding to various services. In one embodiment, the target region 114 may not be available for communication (and / or may not be secure) at the time the region build request is issued. Rather than deferring bootstrapping until the target region 114 is available and configured to perform the bootstrapping operation, CIOS 102 may initiate region build using a virtual bootstrap environment 116. The virtual bootstrap environment (ViBE) 116 may be an overlay network hosted by the host region 103 (an existing region that has previously been configured with a core set of services, is available for communication, and is secure). The MFO 106 may utilize the resources of the host region 103 to bootstrap resources to the ViBE 116 (commonly referred to as "building the ViBE"). For example, MFO 106 can provide instructions via CIOS central 108 to cause an instance of CIOS regional 110 in a host region (e.g., host region 103) to bootstrap another instance of CIOS regional in ViBE 116. After the CIOS regional in ViBE is available for processing, bootstrapping of services for the target region 114 can continue in ViBE 116. Services previously bootstrapped in ViBE 116 can be migrated to the target region 114 when the target region 114 is available to perform the bootstrap operation. Using these techniques, CIOS 102 can greatly increase the speed at which regions are built by greatly reducing the need for manual input and / or configuration to be provided.

[0057] 2 is a block diagram illustrating an environment 200 and method for building a virtual bootstrap environment (ViBE) 202 (an example of ViBE 116 of FIG. 1) according to at least one embodiment. ViBE 202 represents a virtual cloud network that is provisioned in an overlay of an existing region (e.g., hosted region 204, which is an example of hosted region 103 of FIG. 1 and in one embodiment is a hosted region service enclave). ViBE 202 represents an environment in which services can be staged for a target region (e.g., a region under construction, such as target region 114 of FIG. 1) before the target region is available.

[0058] To bootstrap a new region (e.g., target region 114 in FIG. 1), a core set of services can be bootstrapped. These core set of services exist in the host region 204 but do not yet exist in ViBE (nor in the target region). These basic core services provide the functionality required for provisioning devices, establishing a chain of trust to the new region, and deploying the remaining services (e.g., flocks) to the region. ViBE 202 may be a tenancy that is deployed in the host region 204. This can be thought of as a virtual region.

[0059] When the target region is available to provide the bootstrap operations, ViBE 202 can be connected to the target region such that services in the ViBE can interact with services and / or infrastructure components of the target region. This allows for the deployment of production-level services instead of a self-contained seed service as in previous systems, which would require connectivity over the Internet to the target region. Traditionally, a seed service would be deployed as part of a container collection and used to bootstrap the dependencies required to build the region. Using the existing region's infrastructure / tooling, resources can be bootstrapped (e.g., provisioned and deployed) to ViBE 202 until the target region is self-sufficient and can communicate directly, and it can connect to the service enclaves of the region (e.g., host region 204) to provision hardware and deploy services. The use of ViBE 202 allows for the maintenance of dependencies and services required to be able to provision / prepare infrastructure and deploy software while utilizing the host region's resources to break circular dependencies for core services.

[0060] A multi-flock orchestrator (MFO) 206 can be configured to perform operations to build (e.g., configure) ViBE 202. The MFO 206 can obtain appropriate flock configs corresponding to the various resources to be bootstrapped into a new region (in this case, the ViBE region, ViBE 202). For example, the MFO 206 can obtain a flock config (e.g., a "ViBE flock config") that specifies aspects of the bootstrap of capability services 208 and workers 210. As another example, the MFO 206 can obtain another flock config corresponding to the bootstrap of domain name services (DNS) 212 into ViBE 202.

[0061] In step 1, MFO 206 can instruct CIOS central 214 (e.g., an example of CIOS central 108 and CIOS central 214 of FIGS. 1 and 2, respectively). For example, MFO 206 can send a request (e.g., including a ViBE flock config) to request bootstrap of capability services 208 and workers 210 that do not yet exist in ViBE 202 at this point. In one embodiment, CIOS 214 has access to all flock configs. Thus, in one example, MFO 206 can send an identifier for a ViBE flock config rather than the file itself, and CIOS central 214 can retrieve it independently from storage (e.g., from DB 308 or Flock DB 312 of FIG. 3).

[0062] In step 2, CIOS central 214 can provide the ViBE Flock Config via a corresponding request to CIOS regional 216. CIOS regional 216 can parse the ViBE Flock Config to identify and perform specific infrastructure provisioning and deployment operations in step 3.

[0063] In one embodiment, CIOS regional 216 can use additional corresponding services for provisioning and deployment. For example, in step 4, CIOS regional 216 can instruct deployment orchestrator 218 (e.g., an example of a core service or other creation, build and deployment application software in host region 204) to execute instructions to bootstrap capability service 208 and worker 210 within ViBE 202.

[0064] In step 5, a capability may be sent to capability service 208 (from CIOS regional 216, deployment orchestrator 218 via worker 210, or otherwise) indicating that resources corresponding to the ViBE block are available. Capability service 208 may maintain this data. In one embodiment, capability service 208 adds this information to a list of available capabilities that capability service 208 maintains with ViBE. For example, the capability provided to capability service 208 in step 5 may indicate that capability service 208 and worker 210 are available for processing.

[0065] In step 6, the MFO 206 can determine that the capability service 208 and the worker 210 are available based on receiving or obtaining data (identifiers corresponding to capabilities) from the capability service 208.

[0066] In step 7, as a result of receiving / obtaining the data in step 6, MFO 206 can instruct CIOS Central 214 to bootstrap a DNS service (e.g., DNS 212) into ViBE 202. These instructions may specify or include a particular block config that corresponds to the DNS service.

[0067] In step 8, CIOS Central 214 can instruct CIOS Regional 216 to deploy DNS 212 to ViBe 202. In one embodiment, the DNS block config for DNS 212 is provided by CIOS Central 214.

[0068] In step 9, now that worker 210 is deployed to ViBE 202, worker 210 can be assigned by CIOS regional 216 to the task of deploying DNS 212. The worker can run a declarative infrastructure provisioner in the manner described above in connection with FIG. 3 to identify the set of operations that need to be performed to deploy DNS 212 (e.g., by comparing the flock config (desired state) with the current state of the resources associated with the flock (which currently do not exist)).

[0069] At step 10, deployment orchestrator 218 instructs worker 210 to deploy DNS 212 according to the actions identified at step 9. As shown, worker 210 proceeds to perform the actions to deploy DNS 212 to ViBE 202 at step 11. At step 12, worker 210 notifies capability service 208 that DNS 212 is available on ViBE 202. MFO 206 then determines that resources associated with the ViBE flock config and the DNS flock config are available, and any suitable number of additional resources can subsequently be bootstrapped into ViBE.

[0070] After steps 1 through 12 are completed, the process for building ViBE202 may be considered complete, and ViBE202 may be considered built.

[0071] FIG. 3 is a block diagram illustrating an environment 300 and method for bootstrapping a service into a target region using ViBE, according to at least one embodiment.

[0072] In step 1, a user 302 can use any suitable user interface provided by CIOS central 304 (e.g., an example of CIOS central 108 and CIOS central 214 of FIGS. 1 and 2, respectively) to modify region data. For example, a user 302 can create a new region into which several services are bootstrapped.

[0073] In step 2, CIOS central 304 may perform an operation to send the changes to RRDD 306 (e.g., an example of RRDD 104 in FIG. 1). In step 3, RRDD 306 may store the received region data in database 308, which is a data store configured to store region data including any suitable identifiers, attributes, states, etc., such as region, AD, realm, ET, etc. In one embodiment, updater 307 may be used to store the region data in database 308 or any suitable data store from which the updates may be accessible (by a service team). In one embodiment, updater 307 may be configured to notify (e.g., by any suitable electronic notification) of updates made to database 308.

[0074] In step 4, the MFO 310 (e.g., an example of the MFO 106 and MFO 206 of FIGS. 1 and 2, respectively) may detect changes to the region data. In one embodiment, the MFO 310 may be configured to poll the RRDD 306 for changes to the region data. In one embodiment, the RRDD 306 may be configured to publish or otherwise notify the MFO 310 of the region changes.

[0075] In step 5, detection of a change in region data can trigger MFO 310 to retrieve a version set (e.g., a version set associated with a particular identifier, such as a "golden version set" identifier) ​​that identifies the specific version of each flock (e.g., service) to be bootstrapped into the new region and the specific version of each artifact corresponding to that flock. The version set can be retrieved from DB 312. As a flock evolves and changes, the corresponding config and artifact versions used for region construction can change. These changes can be maintained in flock DB 312 so that MFO 310 can identify which versions of flock config and artifacts to use for construction of the region (e.g., ViBE region, target region / non-ViBE region, etc.). Flock configs (e.g., all versions of flock configs) and / or artifacts (e.g., all versions of artifacts) can be stored in DB 308, DB 312, or any suitable data store accessible to CIOS Central 304 and / or MFO 310.

[0076] In step 6, MFO 310 can request CIOS Central 304 to recompile each of the flock configs associated with the version set with the current region data. In one embodiment, the request can indicate the version of each flock config and / or artifacts that correspond to those flock configs.

[0077] In step 7, CIOS Central 304 can obtain the current regional data from DB 308 (e.g., directly or via Real-Time Regional Data Distributor 306) and retrieve any appropriate flock configurations and artifacts according to the version requested by MFO 310.

[0078] In step 8, CIOS central 304 can re-edit the flock config with the region data obtained in step 7 to populate the flock config with the current region data. CIOS central 304 can return the edited flock config to MFO 310. In one embodiment, CIOS central 304 can simply indicate that the edit is complete, and MFO 310 can access the re-edited flock config via RRDD 306.

[0079] In step 9, the MFO 310 may perform a static analysis of the recompiled flock config. As part of the static analysis, the MFO 310 may parse the flock config (e.g., using libraries associated with a declarative infrastructure provisioner (e.g., Terraform, etc.)) to identify dependencies between flocks. From this analysis and the identified dependencies, the MFO 310 may generate a build dependency graph 338. The build dependency graph 338 may be an acyclic directed graph that specifies the order in which flocks are bootstrapped (and / or changes indicated in the flock config are applied) into new regions. Each node in the graph may correspond to the bootstrap of any appropriate portion of a particular flock. A particular bootstrap order may be specified based at least in part on the dependencies. In an embodiment, the dependencies may be represented as attributes of the nodes and / or may be indicated by the edges of the graph connecting the nodes. The MFO 310 may traverse the graph (e.g., starting from the origin node) to drive the operation of region construction.

[0080] In an embodiment, MFO 310 can use a cycle detection algorithm to detect the presence of a cycle (e.g., service A depends on service B and vice versa). MFO 310 can identify orphan capability dependencies. For example, MFO 310 can identify an orphan node in construction dependency graph 338 that does not connect to any other node. MFO 310 can identify a capability that was published in error (e.g., when a capability is published prematurely and the corresponding functionality is not actually available yet). MFO 310 can detect from the graph that there are one or more instances of publication of the same capability. In an embodiment, any suitable number of such errors can be detected, and MFO 310 (or another suitable component, such as CIOS Central 304) can be configured to notify or otherwise present this information to a user (e.g., via an electronic notification, a user interface, etc.). In one embodiment, MFO 310 may be configured to force delete / recreate resources to break circular dependencies and may redirect CIOS Central 304 to perform bootstrap operations for those resources and / or the corresponding block configs.

[0081] The starting node may correspond to bootstrapping the ViBE flock, and the second node may correspond to bootstrapping the DNS. Steps 10-15 correspond to deployment (by deployment orchestrator 317, which is an example of deployment orchestrator 218 in FIG. 2) of the ViBE flock to ViBE 316 (e.g., an example of ViBE 116 and ViBE 202 in FIG. 1 and FIG. 2, respectively). That is, steps 10-15 of FIG. 3 generally correspond to steps 1-6 of FIG. 2. After being notified that capabilities exist corresponding to the ViBE flock to be deployed (e.g., indicating that capability service 318 and worker 320, which correspond to capability service 208 and worker 210 in FIG. 2, are available), MFO 310 resumes traversing build dependency graph 338 to identify the next operation to perform.

[0082] For example, MFO 310 may continue traversing construction dependency graph 338 to identify that a DNS block should be deployed. Steps 16-21 may be performed to deploy DNS 322 (an example of DNS 212 in FIG. 2). These operations may generally correspond to steps 7-12 in FIG. 2.

[0083] At step 21, a capability indicating that DNS 322 is available may be stored. Upon detecting this capability, MFO 310 may resume traversing build dependency graph 338. During this traversal, MFO 310 may identify that any appropriate portions of an instance of CIOS Regional (e.g., an instance of CIOS Regional 314) should be deployed to ViBE 316. In one embodiment, steps 16-21 may be substantially repeated for the deployment of CIOS Regional (ViBE) 326 (an instance of CIOS Regional 314, CIOS Regional 110 in FIG. 1) and Worker 328 to ViBE 316. The capability that CIOS Regional (ViBE) 326 is available may be sent to capability service 318.

[0084] Upon detecting that CIOS Regional (ViBE) 326 is available, MFO 310 may resume traversing build dependency graph 338. During this traversal, MFO 310 may identify that a deployment orchestrator (e.g., deployment orchestrator 330, which is an example of deployment orchestrator 317) is deployed to ViBE 316. In one embodiment, steps 16-21 may be substantially repeated for the deployment of deployment orchestrator 330. Information may be sent to capability service 318 identifying capabilities indicating that deployment orchestrator 330 is available.

[0085] After the deployment orchestrator 330 is deployed, ViBE 316 can be considered available to process subsequent requests. Upon detecting that the deployment orchestrator 330 is available, MFO 310 can instruct that subsequent bootstrap requests are routed to the ViBE component without using the host region component (the component of the host region 332). Thus, MFO 310 can continue to traverse the build dependency graph 338 and at each node, instruct flock deployment to ViBE 316 via CIOS Central 304. CIOS Central 304 can request CIOS Regional (ViBE) 326 to deploy resources according to the flock configuration.

[0086] At some point during this process, the target region 334 may become available. An indication that the target region is available may be identifiable from region data for the target region 334 provided by the user 302 (e.g., as an update to the region data). The availability of the target region 334 may depend on the establishment of a network connection between the target region 334 and an external network (e.g., the Internet). The network connection may be supported over a public network (e.g., the Internet), but software security measures (e.g., IPSec) may be used to provide one or more encrypted tunnels (e.g., IPSec tunnels such as tunnel 336) from ViBE 316 to the target region 334. As used herein, "IPSec" refers to a protocol suite for authenticating and encrypting network traffic over networks that use the Internet Protocol (IP), and may include one or more available embodiments of the protocol suite (e.g., Openswan, Libreswan, strongSwan, etc.). The network may connect ViBE 316 to a service enclave of the target region 334.

[0087] Prior to establishing the IPSec tunnel, the initial network connectivity to the target region 334 may be on a sufficient connection (e.g., an out-of-band VPN tunnel) to allow bootstrapping of networking services until an IPSec gateway can be deployed on an asset (e.g., a bare metal asset) in the target region 334. To bootstrap the network resources of the target region 334, the deployment orchestrator 330 can deploy an IPSec gateway on the asset in the target region 334. The deployment orchestrator 330 can then deploy a VPN host in the target region 334 configured to terminate the IPSec tunnel from ViBE 316. After a service in ViBE 316 (e.g., deployment orchestrator 330, service A, etc.) can establish an IPsec connection with a VPN host in the target region 334, the bootstrapping operation from ViBE 316 to the target region 334 can begin.

[0088] In an embodiment, the bootstrap operation may begin with a service in ViBE 316 provisioning resources in the target region 334 to support hosting instances of core services when deployed from ViBE 316. For example, a host provisioning service may provision a hypervisor on infrastructure (e.g., bare metal hosts) in the target region 334 to allocate computing resources to VMs. Once the host provisioning service completes the allocation of physical resources in the target region 334, the host provisioning service may publish information indicating a capability indicating that physical resources have been allocated in the target region 334. The capability may be published to the capability service 318 (e.g., by worker 328) via CIOS regional (ViBE) 326.

[0089] With the hardware allocation for the target region 334 established and notified to the capability service 318, CIOS regional (ViBE) 326 can orchestrate the deployment of instances of core services from ViBE 316 to the target region 334. This deployment may be similar to the process described above for building ViBE 316, but using ViBE components (e.g., CIOS regional (ViBE) 326, worker 328, deployment orchestrator 330) instead of host region 332 service enclave components. The deployment operations may generally correspond to steps 16-21 described above.

[0090] When a service is deployed from ViBE 316 to a target region 334, a DNS record associated with the service may correspond to an instance of the service in ViBE 316. The DNS record associated with the service can be updated later to complete the deployment of the service to the target region 334. In other words, the instance of the service in ViBE 316 can continue to receive traffic (e.g., requests) to the service until the DNS record is updated. The service can be partially deployed to the target region 334 and can publish information (e.g., to the capability service 318) indicating a capability that the service is partially deployed. For example, a service running in ViBE 316 can be deployed to the target region 334 along with corresponding compute instances, load balancers, and associated applications and other software, but may need to wait for database data to migrate to the target region 334 before being fully deployed. The DNS record (e.g., managed by DNS 322) can still be associated with the service in ViBE 316. Once the data migration for the service is complete, the DNS record can be updated to point to the operational service deployed in the target region 334. The deployed service in target region 334 can then receive traffic (eg, requests) for that service, while the instance of the service in ViBE 316 will no longer be able to receive traffic for that service.

[0091] Region Seed Settings The present disclosure describes techniques for setting up a new data center in a region, including a new data center in a new region and / or a new data center in a new region in a new realm. Specifically, the present disclosure includes a system for setting up a seed in a new region according to specifications of a requester from which services and resources can be generated in the data center. The seed can be set early in the process of setting up a new data center in the region and / or realm in which the new data center is set up. For example, the seed can be set prior to the setting up of one or more control planes in the data center, such as an identity control plane, a compartment control plane, a limit control plane, or some combination thereof.

[0092] The techniques described herein can replace traditional techniques of manually configuring a new datacenter (such as in a new region and / or a new realm), which required time-intensive programming of a large amount of code or copying data from another datacenter to generate the new datacenter. Time-intensive programming techniques can take up a large amount of a programmer's time, which can result in errors in the implementation of the datacenter, which is undesirable. Techniques that copy another datacenter to generate a new datacenter can result in the new datacenter being configured with unnecessary services and / or services that are not used by customers, which can result in wasteful use of the datacenter's resources and / or inefficient operation.

[0093] The techniques described herein can receive a configuration for a data center to be set up and provision the data center according to the configuration. For example, the system can determine services for the data center based on the configuration. The system can also determine what is needed in a seed to enable services to be built in the data center. The system can generate seed instructions to provision the seed and use the seed instructions to provision the seed to the data center to facilitate the setting up of the data center. Provisioning the seed to the data center can include provisioning the seed to one or more resources for the construction of one or more data centers corresponding to a region and / or realm.

[0094] FIG. 4 illustrates a block diagram of an example seed generator mechanism 400, according to at least one embodiment. In particular, the seed generator mechanism 400 illustrates elements that can facilitate the setup of a data center by generating seed instructions and by using the seed instructions to provision the seeds to a data center. Elements of the seed generator mechanism 400, or portions thereof, may be implemented in and / or part of a cloud infrastructure 418 that can provide services to customers. The seeds may include code and / or software that can be provisioned to the data center prior to commissioning a control plane of the data center. For example, the system can set up a data center on infrastructure hardware that is not pre-initialized for the data center, and thus a control plane for the data center may not be set up on the infrastructure hardware.

[0095] The seed generator mechanism 400 may include a requester 402. The requester 402 may be a device requesting the setup of a new data center. For example, the requester 402 may be operated by a customer and / or authorized personnel of a cloud service provider. In an embodiment, the customer, authorized personnel, and / or the requester 402 may be verified for authorization to request the setup of a new data center. For example, the customer and / or authorized personnel may be required to provide credentials verifiable by one or more elements of the cloud infrastructure 418 to determine whether the customer and / or personnel are authorized to request the setup of a new data center. In an embodiment, the requester 402 may provide a token and / or identifier of the requester 402 that the cloud service provider may verify to determine whether the requester 402 is authorized to request the setup of a new data center.

[0096] A customer and / or authorized personnel can define the services to be provided by the new data center. A requester 402 can generate a target configuration 404 indicating the services to be provided. The target configuration 404 can include a file or other storage element that can include data indicating the services to be provided and / or resources for the one or more services to be provided. For example, the target configuration 404 can include one or more of the features, or in an embodiment, a block configuration. In an embodiment, the target configuration 404 can further indicate a region type for the new data center. The region type can be indicated by the customer and / or authorized personnel, or can be determined based on a subscription to the cloud service provider corresponding to the customer. For example, a cloud service provider can offer one or more different subscriptions, where the different subscriptions may have different defined resources for the data center, the region in which the data center is located, and / or the realm in which the data center is located. Specifically, one of the subscriptions may be defined to have one type and / or number of resources, while another of the subscriptions may be defined to have a different type and / or number of resources.

[0097] The requester 402 can communicate with the seed maker 406 and the provisioning system 410. The seed maker 406 and the provisioning system 410 can be implemented on the same device, on different devices, on a cloud infrastructure 418, or some combination thereof. For example, the seed maker 406 and / or the provisioning system 410 can include programs executable on resources of the cloud infrastructure 418. The requester 402 can be coupled to the cloud infrastructure 418 and can provide a target configuration 404 to the seed maker 406 and / or the provisioning system 410. For example, the requester 402 can send the target configuration 404 to the seed maker 406 and / or the provisioning system 410. The seed maker 406 and / or the provisioning system 410 can identify the target configuration 404 provided by the requester 402 and can determine that a new data center is requested based at least in part on the target configuration 404. In one embodiment, the seed maker 406 and / or the provisioning system 410 may perform validation that the requester 402 and / or the user of the requester 402 are authorized to request the new data center. If the seed maker 406 and / or the provisioning system 410 determines that the request for the new data center is authorized, the seed maker 406 and / or the provisioning system 410 may proceed with setting up the new data center. If the seed maker 406 and / or the provisioning system 410 determines that the request for the new data center is not authorized, the seed maker 406 and / or the provisioning system 410 may prevent setting up the new data center.

[0098] The seed maker 406 can determine services for the new data center based on the target configuration 404 provided by the requester 402. For example, the seed maker 406 can analyze the target configuration 404 and determine what services will be provided by the data center based on the target configuration 404. Based on the services provided by the new data center, the seed maker 406 can determine what to include in a seed for the new data center. In particular, the seed maker 406 can determine what seed data can be provisioned to the new data center to enable the new data center to be configured.

[0099] The seed maker 406 may include a data store 408. The data store 408 may store seed data for provisioning into a new data center. The seed data stored in the data store 408 may be grouped into different groups, each group may correspond to a service provided by the new data center. For example, a first group of seed data stored in the data store 408 may correspond to a particular database service that may be provided by the data center. A second group of seed data stored in the data store 408 may correspond to a particular computing service that may be provided by the data center. In an embodiment, the groups of seed data may further correspond to a region type, and a first portion of the seed data that corresponds to a particular service and a particular region type may be different from a second portion of the seed data that corresponds to the particular service and a different region type.

[0100] The seed maker 406 can determine which groups of seed data from the data store 408 to include in the seed for the new data center based on the services determined for the new data center and / or the region type for the new data center. For example, the seed maker 406 can use the services and / or region type determined for the new data center to identify corresponding seed data stored in the data store 408. The seed maker 406 can retrieve the identified corresponding seed data from the data store 408 and generate seed instructions for provisioning the seed in the new data center. The seed maker 406 can generate seed instructions for the new data center by combining the corresponding seed data to generate the seed instructions. In an embodiment, the seed maker 406 can further perform the order in which the seed instructions are executed and / or the resources on which the seed is provisioned. Because the seed maker 406 can identify the corresponding data based on the services and / or the region type, the seed maker 406 can filter out excess seed data corresponding to services not provided by the new data center. The elimination of redundant seed data can make the described approach more efficient than traditional approaches of copying configured data centers to generate new data centers, which can result in the new data centers being allocated unused resources.

[0101] The seed generator mechanism 400 may further include a provisioning system 410. The provisioning system 410 may include one or more mechanisms of CIOS regional 110 (FIG. 1), CIOS regional 216 (FIG. 2), CIOS regional 314 (FIG. 3), ViBE 116 (FIG. 1), ViBE 202 (FIG. 2), ViBE 316 (FIG. 3), or some combination thereof. The provisioning system 410 may be used to provision services and / or data to resources of a cloud infrastructure 418 to set up a data center. For example, the provisioning system 410 may identify available cloud infrastructure resources (such as compute resources, memory resources, networking resources, and other resources) that may be used for the data center. The provisioning system 410 may determine resources for the data center and select from the available cloud infrastructure resources to be used for the data center. In an embodiment, the available cloud infrastructure resources to be used for the data center may be defined, such as defined by the target configuration 404, a subscription, a subscriber, or some combination thereof. For example, a data center can be defined for the region and / or realm being configured, while the cloud infrastructure resources selected for the data center are selected from the available cloud infrastructure resources in the defined data center.

[0102] The seed maker 406 can be coupled to the provisioning system 410 and can provide the provisioning system 410 with the seed instructions generated for the new data center. The provisioning system 410 can identify cloud infrastructure resources to be used for the new data center based on the seed instructions provided by the seed maker 406. For example, the provisioning system 410 can determine resources for the new data center based on the seed instructions and can select the determined resources to be used for the new data center from the available cloud infrastructure resources. In an embodiment, the seed instructions can indicate the resources to be used, the realm in which the resources to be used are located, the region in which the resources to be used are located, or some combination thereof. In an embodiment, the provisioning system 410 can be coupled to the requester 402 and the requester 402 can provide an indication (e.g., a target configuration) to the provisioning system 410 indicating the resources to be used for the new data center.

[0103] The provisioning system 410 can provision a data center 420 in a target region 414 in a cloud infrastructure 418 with a seed generated by a seed instruction provided by a seed maker 406. The target region 414 can include one or more of the characteristics of the target region 114 (FIG. 1) and / or the target region 334 (FIG. 3). The target region 414 can be located in a realm 412 of the cloud infrastructure 418. If the request from the requester 402 is for a new realm, the realm 412 can be a new realm, the target region 414 can be a new region within the new realm, and the data center 420 can be a new data center within the new region. If the request from the requester is for a new region without the creation of a new realm, the realm 412 can be a previously established realm and the target region 414 can be a new region within a previously established realm. If the request from the requester is for a new data center without the creation of a new data center and a new realm, the realm 412 and target region 414 may be previously established and the data center 420 may be a new data center within the target region 414.

[0104] Provisioning of data center 420 by provisioning system 410 may include executing seed instructions provided by seed maker 406 to provision one or more resources of cloud infrastructure 418 with the seed to generate seed infrastructure 416. For example, provisioning of the seed may include provisioning one or more resources with the seed to build one or more data centers (such as data center 420) corresponding to target region 414. Seed infrastructure 416 may include resources of cloud infrastructure 418 and the seed. The resources of cloud infrastructure 418 may have been identified by provisioning system 410 based on the seed instructions, and provisioning system 410 may provision the resources with the seed based on the identification.

[0105] The provisioning system 410 can provision a seed to generate the seed infrastructure 416 before one or more control planes for the data center 420 are commissioned. For example, the provisioning system 410 can provision a seed to generate the seed infrastructure 416 before an identity control plane, a compartment control plane, and / or a limit control plane are commissioned for the data center 420. The seed infrastructure 416 (and / or the seed provisioned to generate the seed infrastructure 416) can be associated with a special indication, which can indicate that the seed infrastructure 416 (and / or the seed provisioned to generate the seed infrastructure 416) is part of a configuration process for the data center 420 and / or is provisioned in a different manner than services provisioned after the generation of the seed infrastructure 416. For example, the provisioning of the seed can be performed without making application programming interface (API) calls in an embodiment.

[0106] The seed infrastructure 416 can provide a foundation for building services offered by the data center 420. For example, the seed infrastructure 416 can provide services and / or resources to enable the provisioning system 410 (or other systems) to provision services offered by the data center 420. For example, the seed infrastructure 416 can grant permission for a user and / or system to provision services to the data center 420. For example, in some cases, a database service offered by the data center 420 may require a pre-populated seed database on which the database service can be provisioned. The seed infrastructure 416 can provide a pre-populated seed database on which the database service can be provisioned. In an embodiment, the seed infrastructure 416 can set up an API in the data center 420, where the provisioning system 410 (or other systems) can make API calls to provision services offered by the data center 420.

[0107] After or during the provisioning of services in the data center 420, the seed infrastructure 416 (and / or the seeds provisioned to generate the seed infrastructure 416) may be transitioned from a special association to a normal association. For example, a label associated with the seed infrastructure 416 (and / or the seeds provisioned to generate the seed infrastructure 416) may be transitioned from special to normal. In an embodiment, the transitioning of the seed infrastructure 416 (and / or the seeds provisioned to generate the seed infrastructure 416) may include reprovisioning the seed with a normal label and / or regenerating the seed infrastructure 416 with a normal label. After the seed infrastructure 416 (and / or the seeds provisioned to generate the seed infrastructure 416) is transitioned to normal, the cloud infrastructure 418 may use the seed infrastructure 416 in the same manner as other services of the data center 420. After the data center 420 is provisioned, the requester 402 may interact with the data center 420, and the data center 420 may provide services to the requester 402.

[0108] While in the depicted embodiment of the seed generator mechanism 400, the seed maker 406, the provisioning system 410, and the realm 412 having the target region 414 and the data center 420 are depicted within the cloud infrastructure 418, and the requester 402 is depicted external to the cloud infrastructure 418, it should be understood that in other embodiments, one or more of these elements may have a different relationship to the cloud infrastructure 418. For example, in certain other embodiments, the requester 402 may be within the cloud infrastructure 418. Also, in certain embodiments, the seed maker 406 and / or the provisioning system 410 may be external to the cloud infrastructure 418. Additionally, while in the depicted embodiment of the seed generator mechanism 400, the data store 408 is depicted within the seed maker 406, it should be understood that in other embodiments, the data store 408 may be separate from the seed maker 406, in which case the seed maker 406 may have access to the data store 408, which may be located within or external to the cloud infrastructure 418.

[0109] It should also be understood that elements of the seed generator mechanism 400 may include programs and / or instructions implementable on a system, different systems, or some combination thereof. For example, the requester 402, the seed maker 406, the provisioning system 410, the realm 412, the target region 414, and / or the data center 420 may include programs and / or instructions that cause one or more systems to perform the operations described in connection with each element. In the case of elements implemented within the cloud infrastructure 418 in an embodiment, the elements may include programs and / or instructions that, when implemented by the resources (such as computational resources, memory resources, networking resources, and other resources) of the cloud infrastructure 418, can perform the corresponding operations. In the case of elements implemented outside of the cloud infrastructure 418, the elements may include programs and / or instructions implemented by one or more devices that cause one or more devices to perform the corresponding operations, or may include one or more devices themselves.

[0110] FIG. 5 illustrates a block diagram of another example seed generator mechanism 500, according to at least one embodiment. In particular, the seed generator mechanism 500 illustrates elements that can facilitate the configuration of a data center by generating seed instructions and by using the seed instructions for provisioning the seeds to a data center. Elements of the seed generator mechanism 500, or portions thereof, may be implemented by and / or part of a cloud infrastructure (such as cloud infrastructure 418 (FIG. 4)) that can provide services to customers. The seeds may include code and / or software that can be provisioned to the data center before the control plane of the data center is commissioned. For example, the system can configure a data center on infrastructure hardware that is not pre-initialized for the data center, and thus a control plane for the data center may not be configured on the infrastructure hardware.

[0111] The seed generator mechanism 500 may include a configuration file 502. The configuration file 502 may include one or more of the features of the target configuration 404 (FIG. 4). For example, the configuration file 502 may indicate the services provided by the new data center that is requested to be configured and / or the resources for one or more of the services provided. The configuration file 502 may also indicate a region type for the new data center in an embodiment. The configuration file may have been generated by an entity such as the requester 402 (FIG. 4).

[0112] The seed generator mechanism 500 may further include an orchestrator 504. The orchestrator 504 may include or include one or more features of the multi-floc orchestrator 106 (FIG. 1), the multi-floc orchestrator 206 (FIG. 2), and / or the multi-floc orchestrator 310 (FIG. 3). The orchestrator 504 may coordinate the generation of a new data center. For example, the orchestrator 504 may coordinate the generation of a seed for provisioning to a new data center, where the data center may be in an existing region, a new region, an existing realm, and / or a new realm. The orchestrator 504 may be a system that manages releases for the construction of a data center.

[0113] The orchestrator 504 may be provided with the configuration file 502. The orchestrator 504 may determine, based on data in the configuration file 502, that the configuration file 502 corresponds to a request for a new data center. The orchestrator 504 may determine, based on information provided by the configuration file 502, whether the new data center is authorized. Based on a determination that the new data center is authorized, the orchestrator 504 may determine that a new data center should be built and may initiate procedures for building the new data center. The orchestrator 504 may coordinate one or more elements of the seed generator mechanism 500 to generate a seed for the new data center and to configure the data center. In some embodiments, the orchestrator 504 may identify resources (computational resources, memory resources, networking resources, and other resources) to be used for seeding the new data center. In these embodiments, the orchestrator 504 may indicate the resources to be used to other elements in the seed generator mechanism 500.

[0114] The seed generator mechanism 500 may include an object store 506. The object store 506 may store data used for seed generation. In one embodiment, the object store 506 may be located in the same region as the orchestrator 504. In other embodiments, the object store 506 may be located in a host region (such as host region 103 (FIG. 1), host region 204 (FIG. 2), and / or host region 332 (FIG. 3)), where the host region is an available region for setting up a new data center.

[0115] The seed generator mechanism 500 may include a bare metal cloud (BMC) operator access tenancy (BOAT) pre-sync 508. The BOAT pre-sync 508 may be coupled to an orchestrator 504 and an object store 506. As part of the seed generation process, the orchestrator 504 may trigger the BOAT pre-sync 508 to generate a BOAT pre-sync file for use in generating seed instructions for provisioning the seed. For example, the orchestrator 504 may trigger the generation of a BOAT pre-sync file by the BOAT pre-sync 508 in response to receiving the configuration file 502. The BOAT pre-sync file may include a cloud identifier (e.g., a realm cloud identifier and / or a BOAT tenancy cloud identifier) ​​of the new data center. The BOAT pre-sync 508 may, in an embodiment, retrieve a cloud identifier from the object store 506 for inclusion in the BOAT pre-sync file.

[0116] The seed generator mechanism 500 may include a tenancy creation mechanism 510. The tenancy creation mechanism 510 may be coupled to the orchestrator 504 and the object store 506. As part of the seed generation process, the orchestrator 504 may trigger the tenancy creation mechanism to generate a tenancy creation mechanism file. For example, the orchestrator 504 may trigger the generation of a tenancy creation mechanism file by the tenancy creation mechanism 510 in response to receiving the configuration file 502. The tenancy creation mechanism file may include a tenancy name associated with the new data center and / or administrative policies for the new data center associated with the tenancy. The tenancy creation mechanism 510, in one embodiment, may retrieve the tenancy name and / or administrative policies from the object store 506 for inclusion in the tenancy creation mechanism file.

[0117] The seed generator mechanism 500 may further include a provisioning system 512. The provisioning system 512 may include one or more of the features of the provisioning system 410 (FIG. 4). The provisioning system 512 may be coupled to the orchestrator 504 and the object store 506. As part of the seed generation process, the orchestrator 504 may trigger the provisioning system 512 to generate a state file. For example, the orchestrator 504 may trigger the generation of a state file by the provisioning system 512 in response to receiving the configuration file 502. The state file may define resources for the new data center. The provisioning system 512 may store the state file in the object store 506.

[0118] The seed generator mechanism 500 may further include a seed maker 514. The seed maker 514 may include one or more of the features of the seed maker 406 (FIG. 4). The seed maker 514 may be coupled to the orchestrator 504, the object store 506, the BOAT pre-sync 508, the tenancy creation mechanism 510, and / or the provisioning system 512. The seed maker 514 may receive a BOAT pre-sync file from the BOAT pre-sync 508 and a tenancy creation mechanism file from the tenancy creation mechanism 510, and may retrieve a state file generated by the provisioning system 512 from the object store 506. The seed maker 514 may combine the BOAT pre-sync file, the tenancy creation mechanism file, and the state file to generate seed instructions 522 for provisioning a new data center. The seed instructions 522 may include an identity data plane (IDDP) seed artifact and / or a limit seed file. The SeedMaker 514 may also perform validation that the establishment of a new data center is permitted.

[0119] The seed maker 514 can deliver the seed instructions 522 to the provisioning system 512. The orchestrator 504 can indicate to the provisioning system 512 the resources 516 to be used for seeding the new data center. In an embodiment, the resources 516 can include a data center or a portion of a data center. The provisioning system 512 can provision the seeds to the resources 516. For example, the provisioning system 512 can provision the seeds to one or more resources to build one or more data centers. Provisioning the seeds to the resources 516 can initiate the configuration of a new data center in a region 518 according to the new data center request corresponding to the configuration file 502. The region 518 can include one or more of the characteristics of the target region 414 (FIG. 4). The region 518 can be configured in a realm 520. The realm 520 can include one or more of the characteristics of the realm 412 (FIG. 4). The realm 520 can be a previously configured realm if the request is for the configuration of a new region without a new realm. In other cases where the request is for a new realm configuration, realm 520 may be a new realm with region 518 being a new region. The resultant elements produced by provisioning the seed to resource 516 may include one or more of the characteristics of seed infrastructure 416 (FIG. 4).

[0120] The provisioning system 512 can provision a seed to a resource 516 before one or more control planes for the region 518 and / or realm 520 are commissioned. For example, the provisioning system 512 can execute seed instructions 522 received from the seed maker 514 to provision a seed before an identity control plane, a compartment control plane, and / or a limit control plane are commissioned for a data center configured in the region 518 and / or realm 520. The seed can be associated with a special indication that the seed is part of a configuration process for a data center in the region 518 and / or realm 520 and / or is provisioned in a different manner than services that are provisioned after the control plane is commissioned for the data center. For example, the provisioning of the seed can be performed without making an application programming interface (API) call in an embodiment.

[0121] A seed can provide a foundation for building services offered by a data center. For example, a seed and resources 516 can provide services and / or resources to enable a provisioning system 512 (or other system) to provision services offered by a data center in a region 518 and / or realm 520. For example, a seed can provide authorization for a user and / or system to provision services in a data center. For example, in one instance, a database service offered by a data center may require a pre-populated seed database on which the database service can be provisioned. A seed and resources 516 can provide a pre-populated seed database on which the database service can be provisioned. In one embodiment, a seed can set up an API in a data center in a region 518 and / or realm 520, where a provisioning system 512 (or other system) can make API calls to provision services offered by the data center.

[0122] After or during the provision of the service in the data center, the seed may be transitioned from a special association to a normal association. For example, an indicator associated with the seed may be transitioned from special to normal. In an embodiment, transitioning the seed may include reprovisioning the seed with an indicator. In an embodiment, transitioning the seed from special to normal may include copying the seed from the host realm to the realm 520. After transitioning the seed to normal, the cloud infrastructure (e.g., cloud infrastructure 418) in which the data center of the region 518 and / or realm 520 is configured may use the seed in the same manner as other services configured in the data center of the region 518 and / or realm 520.

[0123] It should be understood that elements of seed generator mechanism 500 may include programs and / or instructions executable on a system, different systems, or some combination thereof. For example, orchestrator 504, BOAT pre-sync 508, tenancy creation mechanism 510, provisioning system 512, seedmaker 514, region 518, and / or realm 520 may include programs and / or instructions that cause one or more systems to perform the operations described in connection with each element. In an embodiment, one or more of orchestrator 504, BOAT pre-sync 508, tenancy creation mechanism 510, provisioning system 512, seedmaker 514, region 518, and / or realm 520 may be implemented in a cloud infrastructure, such as cloud infrastructure 418 (FIG. 4). For elements implemented by a cloud infrastructure in an embodiment, the elements may include programs and / or instructions that, when implemented by the resources (computational resources, memory resources, networking resources, and other resources) of the cloud infrastructure, may perform the corresponding operations. For elements implemented outside the cloud infrastructure, the elements may include programs and / or instructions implemented by one or more devices that cause the one or more devices to perform corresponding operations, or may include the one or more devices themselves.

[0124] 6 illustrates an example procedure 600 for provisioning a seed infrastructure, according to at least one embodiment. For example, procedure 600 may include provisioning a seed infrastructure in a new data center of a region in a realm (such as a data center in target region 114 (FIG. 1), a data center in target region 334 (FIG. 3), data center 420 (FIG. 4) and / or a data center in region 518 (FIG. 5)). Procedure 600 may be performed to set up a new data center, a new region and / or a new realm for a cloud service infrastructure, such as infrastructure as a service (Iaas) in cloud computing. Procedure 600 may be performed by a seed maker (such as seed maker 406 (FIG. 4) and / or seed maker 514 (FIG. 5)). In an embodiment, procedure 600 may be performed by a seedmaker (such as seedmaker 406 and / or seedmaker 514), a multi-flock orchestrator (MFO) (such as orchestrator 504 (FIG. 5)), a tenancy creation mechanism (such as tenancy creation mechanism 510 (FIG. 5)), a BOAT pre-sync (such as BOAT pre-sync 508 (FIG. 5)), a provisioning system (such as provisioning system 410 (FIG. 4) and / or provisioning system 512 (FIG. 5)), or some combination thereof. For simplicity, procedure 600 is described as being performed by a seedmaker, but it should be understood that procedure 600 may be performed by a seedmaker, an MFO, a tenancy creation mechanism, a BOAT pre-sync, a provisioning system, or some combination thereof. Although elements of procedure 600 are described and illustrated in a certain order, it should be understood that in other embodiments, the elements may be in a different order and / or two or more of the elements may be performed simultaneously.

[0125] At 602, a seed maker may receive information identifying a new data center to be created in a region (e.g., target region 114 (FIG. 1), target region 334 (FIG. 3), target region 414 (FIG. 4), and / or region 518 (FIG. 5)) in a realm (e.g., realm 412 (FIG. 4) and / or realm 520 (FIG. 5)) and services to be configured for the new data center. The data center may be the first data center provisioned in the region and / or realm, and procedure 600 is initiated before a control plane is commissioned in the data center, region, and / or realm. For example, procedure 600 may be initiated before an identity control plane, compartment control plane, and / or limit control plane are configured in the data center, region, and / or realm. For example, the seed maker may receive a configuration identifying a set of services and / or resources for the new data center. In an embodiment, the configuration may be a service configuration defined by a requester indicating a set of resources and / or services to be provided by a data center in a region in the realm. The service configuration can be a golden version set that is marked as the service configuration to be used for the new region.

[0126] At 604, the seed maker may determine seed data based on the information received at 602. For example, the seed maker may determine seed data for a set of seed instructions for provisioning a set of services and / or resources in a new data center. The seed data may include code to be provisioned in a seed infrastructure for the new data center. The seed infrastructure may provide a starting point for configuring the remainder of the resources and services identified in the information. For example, the seed maker may determine one or more resources and configurations of resources to be used as a seed infrastructure for setting up other resources and / or services provided by the region. The determined resources and configurations may include resources and configurations to be provisioned in a new data center in a region of a realm before a control plane in the region is commissioned in the data center, region, and / or realm. The determined resources and configurations may be used to set up a control plane in the data center, region, and / or realm and / or configure other resources and / or services determined to be provided by the data center from the information identifying the set of resources and services.

[0127] At 606, the seed maker may generate seed instructions for creating and provisioning a seed infrastructure in the new data center. For example, the seed maker may generate seed instructions based on the seed data determined at 604. Generating the seed instructions may include retrieving and / or generating seed data to be used for the seed infrastructure based on the seed data. For example, the seed maker may determine a bare metal cloud operator access tenancy (BOAT) presync file, a tenancy creation mechanism file, a service output, or some combination thereof to be used for the seed infrastructure in an embodiment. In one of such embodiments, the service output may include a state file of the data center. The seed maker may retrieve seed data from an object store, such as a corporate tenancy object store, a service provider object store, a host region object store, or a combination thereof. In an embodiment, the seed maker may combine the BOAT presync file, the tenancy creation mechanism file, and / or the service output to generate the seed instructions. In an embodiment, the seed instructions include a seed artifact (such as an identity data plane (IDDP) artifact) and / or a limit seed file.

[0128] At 608, the seed maker can use the seed instructions to provision a seed infrastructure. In particular, the seed maker can use the seed instructions generated at 606 to provision a seed infrastructure at a data center in a region of the realm. In one embodiment, provisioning the seed infrastructure includes providing the seed instructions by the seed maker to a provisioning system (e.g., provisioning system 410 (FIG. 4) and / or provisioning system 512 (FIG. 5)), which can use the seed instructions to provision the seed infrastructure at the data center. Provisioning the seed infrastructure at a data center in a region of the realm can, in one embodiment, include provisioning the seed infrastructure to one or more resources (e.g., resources 516 (FIG. 5)) for construction of one or more data centers corresponding to the region. The seed instructions can be provisioned at the data center. In one embodiment, the seed instructions can be distributed to a host region (e.g., host region 103 (FIG. 1), host region 204 (FIG. 2), and / or host region 332 (FIG. 3)) used to set up the new data center, which provisions the seed instructions at the new data center. The seed instructions may include backfilling of the seed infrastructure within the region.

[0129] At 610, the seed maker may mark the seed instruction as a reference seed infrastructure creation instruction for the data center to set up the seed infrastructure for the service. In an embodiment, marking the seed instruction as a reference seed infrastructure creation instruction may include marking the seed instruction as special to indicate that the seed instruction is used to generate a seed in a new data center and that the seed instruction may have provisioning to the data center different than other services and / or resources to be provisioned before the control plane is commissioned in the data center, region, and / or realm. Marking the seed instruction as a reference seed infrastructure creation instruction may enable the seed instruction to be used for provisioning of other data centers in regions in the realm that have the same services as the data center currently being provisioned by procedure 600.

[0130] 7 illustrates an example procedure 700 for provisioning a seed infrastructure based on whether a master seed infrastructure creation command exists for a data center, according to at least one embodiment. For example, the procedure 700 may include provisioning a seed infrastructure in a new data center (such as a data center in target region 114 (FIG. 1), a data center in target region 334 (FIG. 3), a data center in data center 420 (FIG. 4) and / or a data center in region 518 (FIG. 5)). The procedure 700 may include determining whether a master seed infrastructure creation command exists for a realm having the same services as the new data center. The procedure 700 may be performed by a seed maker (such as seed maker 406 (FIG. 4) and / or seed maker 514 (FIG. 5)). In an embodiment, procedure 700 may be performed by a seedmaker (such as seedmaker 406 and / or seedmaker 514), an MFO (such as orchestrator 504 (FIG. 5)), a tenancy creation mechanism (such as tenancy creation mechanism 510 (FIG. 5)), a BOAT pre-sync (such as BOAT pre-sync 508 (FIG. 5)), a provisioning system (such as provisioning system 410 (FIG. 4) and / or provisioning system 512 (FIG. 5)), or some combination thereof. For simplicity, procedure 700 is described as being performed by a seedmaker, but it should be understood that procedure 700 may be performed by a seedmaker, an MFO, a tenancy creation mechanism, or some combination thereof. Although elements of procedure 700 are described and illustrated in a certain order, it should be understood that in other embodiments, the elements may be in a different order and / or two or more of the elements may be performed simultaneously.

[0131] At 702, a seed maker may receive information identifying a new data center to be created in a region (e.g., target region 114 (FIG. 1), target region 334 (FIG. 3), target region 414 (FIG. 4) and / or region 518 (FIG. 5)) to be created in a realm (e.g., realm 412 (FIG. 4) and / or realm 520 (FIG. 5)) and services to be configured for the new data center. For example, the seed maker may receive a configuration identifying a set of resources and / or services. In one embodiment, the configuration may be a service configuration defined by a requester indicating a set of resources / services to be provided by a data center in a region in the realm. The service configuration may be a golden version set marked as the service configuration to be used for the new data center. In one embodiment, the service configuration may be a different service configuration than the golden version set, in which case the services configured for the service configuration may be different from the services configured for the golden version set.

[0132] At 704, the seed maker may determine whether there are reference seed infrastructure creation instructions for a data center in a region in the realm having the same set of services. For example, one or more previously generated seed instructions may be stored as reference seed infrastructure creation instructions for the corresponding set of services. The seed maker may access the previously stored reference seed infrastructure creation instructions and identify any of the stored reference seed infrastructure creation instructions that correspond to the realm. For the stored reference seed infrastructure creation instructions determined to correspond to the realm, the seed maker may determine whether any of the stored reference seed infrastructure creation instructions are for the same set of services to be configured for the new data center. If the seed maker determines that there are no stored reference seed infrastructure creation instructions for the same set of services, procedure 700 may proceed to 706. If the seed maker determines that there are stored reference seed infrastructure creation instructions for the same set of services, procedure 700 proceeds to 712.

[0133] At 706, the seed maker may generate seed instructions for creating and provisioning a seed infrastructure in the new data center. Specifically, the seed maker may generate the seed instructions based on the seed maker determining that no reference seed infrastructure creation instructions exist for a data center in a region in a realm that has the same set of services. The seed maker may generate the seed instructions based on the configuration for the new data center shown in 702. Generating the seed instructions may include retrieving and / or generating seed data to be used for the seed infrastructure based on one or more resources and / or associated configurations. For example, the seed maker, in an embodiment, may determine a BOAT presync file, a tenancy creation mechanism file, a service output, or a combination thereof to be used for the seed infrastructure. In some of such embodiments, the service output may include a state file for the data center. The seed maker may retrieve seed data from an object store, such as an enterprise tenancy object store, a service provider object store, a host region object store, or a combination thereof. In an embodiment, the seed maker may combine a BOAT presync file, a tenancy creation mechanism file, and / or a service output to generate the seed instructions. In one embodiment, the seed infrastructure creation instructions include a seed artifact (such as an identity data plane (IDDP) artifact) and / or a limit seed file.

[0134] At 708, the seed maker can provision a seed infrastructure using the seed instructions. Specifically, the seed maker can provision a seed infrastructure in the new data center using the seed instructions generated at 706. In one embodiment, provisioning the seed infrastructure includes the seed maker providing the seed instructions to a provisioning system (e.g., provisioning system 410 (FIG. 4) and / or provisioning system 512 (FIG. 5)), which can use the seed instructions to provision the seed infrastructure in the data center. Provisioning the seed infrastructure in the new data center in the region of the new realm can include, in one embodiment, provisioning the seed infrastructure in one or more resources (e.g., resources 516 (FIG. 5)) to build one or more data centers corresponding to the region. The seed instructions can be provisioned in the data center. In one embodiment, the seed instructions can be distributed to a host region (e.g., host region 103 (FIG. 1), host region 204 (FIG. 2), and / or host region 332 (FIG. 3)) used to set up the new data center, which provisions the seed instructions in the new data center. The seed instructions may include backfilling of the seed infrastructure within the region.

[0135] At 710, the seed maker can store the seed instructions as reference seed infrastructure creation instructions for the realm for setting up a seed infrastructure for the service. Storing the seed instructions as reference seed infrastructure creation instructions can enable the seed instructions to be used for provisioning of other data centers in the realm that have the same services as the region currently being provisioned by procedure 700.

[0136] At 712, the seed maker can create and provision a seed infrastructure using the stored reference seed infrastructure creation instructions. For example, the seed maker can create and provision a seed infrastructure for a new data center using the reference seed infrastructure creation instructions determined at 704.

[0137] Cloud Service Infrastructure Architecture Examples As mentioned above, Infrastructure as a Service (IaaS) is one particular type of cloud computing. IaaS can be configured to provide virtualized computing resources over a public network (e.g., the Internet). In the IaaS model, a cloud computing provider can host infrastructure components (e.g., servers, storage devices, network nodes (e.g., hardware), deployment software, platform virtualization (e.g., hypervisor layer), etc.). In some cases, an IaaS provider can also provide various services (example services include billing software, monitoring software, logging software, load balancing software, and clustering software, etc.) that accompany those infrastructure components. Thus, these services can be policy-driven, so that an IaaS user can enforce policies to drive load balancing to maintain application availability and performance.

[0138] In some cases, IaaS customers can access resources and services over a wide area network (WAN) such as the Internet, and can use the cloud provider's services to install the remaining elements of their application stack. For example, a user can log into an IaaS platform to create virtual machines (VMs), install an operating system (OS) on each VM, deploy middleware such as databases, create storage buckets for workloads and backups, and even install enterprise software on the VMs. The customer can then use the provider's services to perform a variety of functions, including balancing network traffic, troubleshooting application issues, monitoring performance, managing disaster recovery, and more.

[0139] In most cases, the cloud computing model will require the involvement of a cloud provider, which can be, but is not necessarily, a third-party service that specializes in providing (e.g., offering, renting, selling) IaaS. An entity may choose to deploy a private cloud, thereby becoming its own provider of infrastructure services.

[0140] In one example, an IaaS deployment is the process of placing a new application or a new version of an application onto a prepared application server, etc. It may also include the process of preparing the server (e.g., installing libraries, daemons, etc.). This is often managed by the cloud provider below the hypervisor layer (e.g., server, storage, network hardware, and virtualization). Thus, the customer may be responsible for handling (OS), middleware, and / or application deployment (e.g., self-service virtual machines (which can be spun up on demand, etc.)).

[0141] In one example, IaaS provisioning refers to obtaining a computer or virtual host for use and then installing the required libraries or services on it. In many cases, deployment does not include provisioning, which may need to occur first.

[0142] In some cases, there are two different challenges in IaaS provisioning. First, there is the initial challenge of provisioning an initial set of instructions before anything can be put into production. Second, there is the challenge of evolving the existing infrastructure (e.g. adding new services, modifying services, removing services, etc.) after everything has been provisioned. In some cases, these two challenges can be addressed by allowing the configuration of the infrastructure to be defined declaratively. In other words, one or more configuration files can define the infrastructure (e.g. what components are needed and how they interact). Thus, the overall topology of the infrastructure (e.g. what resources depend on which resources and how each of them work together) can be defined declaratively. In some cases, after the topology is defined, workflows can be generated that create and / or manage the different components described in the configuration files.

[0143] In one example, an infrastructure can have many interconnected elements. For example, there may be one or more virtual private clouds (VPCs) (e.g., a potentially on-demand pool of configurable and / or shared computing resources), also referred to as a core network. In one example, there may also be one or more inbound / outbound traffic group rules that are provisioned to define how the network's inbound / outbound traffic is configured and one or more virtual machines (VMs). Other infrastructure elements, such as load balancers, databases, etc., can also be provisioned. The infrastructure can evolve over time as more and more infrastructure elements are required and / or added.

[0144] In some cases, continuous deployment techniques can be employed to enable deployment of infrastructure code across various virtual computing environments. Additionally, the described techniques can enable infrastructure management within these environments. In some examples, a service team can write code that is required to be deployed to one or more, often many, different production environments (e.g., across various different geographic locations, sometimes spanning the globe). In some examples, however, the infrastructure onto which the code will be deployed must first be set up. In some cases, provisioning can be done manually, provisioning tools can be used to provision the resources, and / or deployment tools can be used to deploy the code once the infrastructure has been provisioned.

[0145] 8 is a block diagram 800 illustrating an example platform for an IaaS architecture in accordance with at least one embodiment. A service operator 802 can be communicatively coupled to a secure host tenancy 804, which can include a virtual cloud network (VCN) 806 and a secure host subnet 808. In one example, the service operator 802 can employ one or more customer computing devices, which can be portable handheld devices (e.g., iPhones, mobile phones, iPads, computing tablets, personal digital assistants (PDAs)) or wearable devices (e.g., Google Glass head-mounted displays) running software such as Microsoft Windows Mobile and / or various mobile operating systems such as iOS, Windows Phone, Android, BlackBerry 8, Palm OS, and the like, and capable of Internet, email, short message service (SMS), Blackberry, or other communications protocols. Alternatively, the customer computing devices may be general purpose personal computers, including, for example, personal and / or laptop computers running various versions of Microsoft Windows, Apple Macintosh, and / or Linux operating systems. The customer computing devices may be workstation computers running any of the various commercially available UNIX or UNIX-like operating systems, including, but not limited to, the various GNU / Linux operating systems, such as Google Chrome OS.Alternatively or additionally, the customer computing device may be any other electronic device, such as a thin-client computer, an Internet-enabled gaming system (e.g., a Microsoft Xbox® game console with or without a Kinect® gesture input device), and / or a personal messaging device capable of communicating over VCN 806 and / or an Internet-accessible network.

[0146] VCN 806 may include a local peering gateway (LPG) 810 that may be communicatively coupled to an SSH VCN 812 via an LPG 810 that is included in a secure shell (SSH) VCN 812. The SSH VCN 812 may include an SSH subnet 814, which may be communicatively coupled to a control plane VCN 816 via an LPG 810 that is included in the control plane VCN 816. The SSH VCN 812 may also be communicatively coupled to a data plane VCN 818 via an LPG 810. The control plane VCN 816 and the data plane VCN 818 may be included in a service tenancy 819 that may be owned and / or operated by the IaaS provider.

[0147] The control plane VCN 816 may include a control plane demilitarized zone (DMZ) tier 820 that serves as a perimeter network (e.g., the portion of an enterprise network between the enterprise intranet and an external network). DMZ-based servers may have limited duties and help thwart intrusions. Additionally, the DMZ tier 820 may include one or more load balancer (LB) subnets 822, a control plane application tier 824 that may include application subnets 826, a control plane data tier 828 that may include database (DB) subnets 830 (e.g., a front-end DB subnet and / or a back-end DB subnet). The LB subnet 822 included in the control plane DMZ layer 820 can be communicatively coupled to an application subnet 826 included in the control plane application layer 824 and an Internet gateway 834 that can be included in the control plane VCN 816, which can be communicatively coupled to a DB subnet 830 included in the control plane data layer 828 and a service gateway 836 and a network address translation (NAT) gateway 838. The control plane VCN 816 can include the service gateway 836 and the NAT gateway 838.

[0148] The control plane VCN 816 may include a data plane mirrored application tier 840, which may include an application subnet 826. The application subnet 826 included in the data plane mirrored application tier 840 may include a virtual network interface controller (VNIC) 842 capable of running a compute instance 844. The compute instance 844 may be communicatively coupled to the application subnet 826 of the data plane mirrored application tier 840 to the application subnet 826, which may be included in the data plane application tier 846.

[0149] The data plane VCN 818 may include a data plane application layer 846, a data plane DMZ layer 848, and a data plane data layer 850. The data plane DMZ layer 848 may include a LB subnet 822 that may be communicatively coupled to an application subnet 826 of the data plane application layer 846 and an Internet gateway 834 of the data plane VCN 818. The application subnet 826 may be communicatively coupled to a service gateway 836 of the data plane VCN 818 and a NAT gateway 838 of the data plane VCN 818. The data plane data layer 850 may also include a DB subnet 830 that may be communicatively coupled to the application subnet 826 of the data plane application layer 846.

[0150] The Internet gateways 834 of the control plane VCNs 816 and the data plane VCNs 818 may be communicatively coupled to a metadata management service 852, which may be communicatively coupled to the public Internet 854. The public Internet 854 may be communicatively coupled to NAT gateways 838 of the control plane VCNs 816 and of the data plane VCNs 818. The service gateways 836 of the control plane VCNs 816 and of the data plane VCNs 818 may be communicatively coupled to cloud services 856.

[0151] In one example, a service gateway 836 in a control plane VCN 816 or in a data plane VCN 818 can make application programming interface (API) calls to a cloud service 856 without traversing the public Internet 854. API calls from a service gateway 836 to a cloud service 856 can be one-way; that is, the service gateway 836 can make API calls to a cloud service 856 and the cloud service 856 can send the requested data to the service gateway 836. However, the cloud service 856 cannot originate API calls to the service gateway 836.

[0152] In one example, a secure host tenancy 804 can be directly connected to an otherwise separable service tenancy 819. A secure host subnet 808 can communicate with an SSH subnet 814 through an LPG 810, which can enable bidirectional communication through otherwise separate systems. Connecting the secure host subnet 808 to the SSH subnet 814 can give the secure host subnet 808 access to other entities in the service tenancy 819.

[0153] The control plane VCN 816 can enable users of a service tenancy 819 to configure or otherwise provision desired resources. The desired resources provisioned in the control plane VCN 816 can be deployed or otherwise used in the data plane VCN 818. In one example, the control plane VCN 816 can be separate from the data plane VCN 818, and a data plane mirror application tier 840 of the control plane VCN 816 can communicate with a data plane application tier 846 of the data plane VCN 818 via a VNIC 842, which can be included in the data plane mirror application tier 840 and the data plane application tier 846.

[0154] In one example, a user or customer of the system may make a request, for example, a create, read, update or delete (CRUD) operation, via the public Internet 854, which may communicate the request to a metadata management service 852. The metadata management service 852 may communicate the request to the control plane VCN 816 via an Internet gateway 834. The request may be received by a LB subnet 822 included in the control plane DMZ layer 820. The LB subnet 822 may determine that the request is valid, and in response to this determination, the LB subnet 822 may send the request to an application subnet 826 included in the control plane application layer 824. If the validity of the request is verified and a call to the public Internet 854 is required, the call to the public Internet 854 may be sent to a NAT gateway 838, which may make a call to the public Internet 854. Metadata that may be desired to be stored by the request may be stored in the DB subnet 830.

[0155] In one example, a data plane mirror application layer 840 can facilitate direct communication between the control plane VCN 816 and the data plane VCN 818. For example, it may be desired that configuration changes, updates, or other suitable modifications be applied to resources included in the data plane VCN 188. VNIC 842 allows the control plane VCN 816 to communicate directly with resources included in the data plane VCN 818, thereby enabling it to perform configuration changes, updates, or other suitable modifications to those resources.

[0156] In an embodiment, the control plane VCN 816 and the data plane VCN 818 may be included in the service tenancy 819. In this case, a user or customer of the system may not own or operate the control plane VCN 816 or the data plane VCN 818. Instead, an IaaS provider may own or operate the control plane VCN 816 and the data plane VCN 818, both of which may be included in the service tenancy 819. This embodiment may allow for network isolation that may prevent a user or customer from interacting with the resources of other users or other customers. This embodiment may also allow a user or customer of the system to store databases privately without having to rely on the public Internet 854, which may not have the desired level of threat protection for storage.

[0157] In another embodiment, the LB subnet 822 included in the control plane VCN 816 can be configured to receive signals from the service gateway 836. In this embodiment, the control plane VCN 816 and the data plane VCN 818 can be configured to be called by the IaaS provider's customers without calling the public Internet 854. Customers of the IaaS provider may desire this embodiment because databases used by the customers can be controlled by the IaaS provider and stored on a service tenancy 819 that is separable from the public Internet 854.

[0158] 9 is a block diagram 900 illustrating another example pattern of an IaaS architecture, according to at least one embodiment. A service operator 902 (e.g., service operator 802 of FIG. 8 ) may be communicatively coupled to a secure host tenancy 904 (e.g., secure host tenancy 804 of FIG. 8 ), which may include a virtual cloud network (VCN) 906 (e.g., VCN 806 of FIG. 8 ) and a secure host subnet 908 (e.g., secure host subnet 808 of FIG. 8 ). The VCN 906 may include a local peering gateway (LPG) 910 (e.g., LPG 810 of FIG. 8 ), which may be communicatively coupled to an SSH VCN 912 (e.g., SSH VCH 812 of FIG. 8 ) via an LPG 810 included in a secure shell (SSH) VCN 912. SSH VCN 912 can include an SSH subnet 914 (e.g., SSH subnet 814 in FIG. 8 ), which can be communicatively coupled to a control plane VCN 916 (e.g., control plane VCN 816 in FIG. 8 ) via an LPG 910 included in the control plane VCN 916. The control plane VCN 916 can be included in a service tenancy 919 (e.g., service tenancy 819 in FIG. 8 ), and the data plane VCN 918 (e.g., data plane VCN 818 in FIG. 8 ) can be included in a customer tenancy 921, which can be owned or operated by a user or customer of the system.

[0159] The control plane VCN 916 may include a control plane DMZ layer 920 (e.g., control plane DMZ layer 820 of FIG. 8 ) that may include a LB subnet 922 (e.g., LB subnet 822 of FIG. 8 ), a control plane layer 924 (e.g., control plane application layer 824 of FIG. 8 ) that may include an application subnet 926 (e.g., application subnet 826 of FIG. 8 ), and a control plane data layer 928 (e.g., control plane data layer 828 of FIG. 8 ) that may include a database (DB) subnet 930 (e.g., similar to DB subnet 830 of FIG. 8 ). LB subnet 922 included in control plane DMZ tier 920 can be communicatively coupled to application subnet 926 included in control plane application tier 924 and to an Internet gateway 934 (e.g., Internet gateway 834 in FIG. 8 ) that may be included in control plane VCN 916, and application subnet 926 can be communicatively coupled to DB subnet 930 included in control plane data tier 928 and to a service gateway 936 (e.g., service gateway 836 in FIG. 8 ) and a network address translation (NAT) gateway 938 (e.g., NAT gateway 838 in FIG. 8 ). Control plane VCN 916 can include service gateway 936 and NAT gateway 938.

[0160] The control plane VCN 916 may include a data plane mirrored application tier 940 (e.g., data plane mirrored application tier 840 of FIG. 8 ), which may include an application subnet 926. The application subnet 926 included in the data plane mirrored application tier 940 may include a virtual network interface controller (VNIC) 942 (e.g., VNIC 842) on which a compute instance 944 (e.g., similar to compute instance 844 of FIG. 8 ) may run. The compute instance 944 may facilitate communication between the application subnet 926 of the data plane mirrored application tier 940 and the application subnet 926 included in the data plane application tier 946 (e.g., data plane application tier 846 of FIG. 8 ) via the VNIC 942 included in the data plane mirrored application tier 940 and the VNIC 942 included in the data plane application tier 946.

[0161] An internet gateway 934 included in the control plane VCN 916 may be communicatively coupled to a metadata management service 952 (e.g., metadata management service 852 of FIG. 8), which may be communicatively coupled to a public internet 954 (e.g., public internet 854 of FIG. 8). The public internet 954 may be communicatively coupled to a NAT gateway 938 included in the control plane VCN 916. A service gateway 936 included in the control plane VCN 916 may be communicatively coupled to cloud services 956 (e.g., cloud services 856 of FIG. 8).

[0162] In one example, the data plane VCN 918 may be included in the customer tenancy 921. In this case, the IaaS provider may provide each customer with a control plane VCN 916, and the IaaS provider may configure a unique compute instance 944 included in the service tenancy 919 for each customer. Each compute instance 944 may enable communication between the control plane VCN 916 included in the service tenancy 919 and the data plane VCN 918 included in the customer tenancy 921. The compute instance 944 may enable resources provisioned in the control plane VCN 916 included in the service tenancy 919 to be deployed or otherwise used in the data plane VCN 918 included in the customer tenancy 921.

[0163] In another example, an IaaS provider customer may have a database that resides within customer tenancy 921. In this example, control plane VCN 916 may include a data plane mirrored application tier 940 that may include application subnet 926. The data plane mirrored application tier 940 may reside in data plane VCN 918, but the data plane mirrored application tier 940 may not reside in data plane VCN 918. That is, the data plane mirrored application tier 940 may have access to customer tenancy 921, but the data plane mirrored application tier 940 may not reside in data plane VCN 918 or be owned or operated by the IaaS provider customer. The data plane mirrored application tier 940 may be configured to make calls to the data plane VCN 918, but may not be configured to make calls to any entities included in the control plane VCN 916. A customer may wish to deploy or otherwise use resources in the data plane VCN 918 that have been provisioned in the control plane VCN 916, and the data plane mirror application tier 940 can facilitate the desired deployment or other use of the customer's resources.

[0164] In one embodiment, an IaaS provider's customer can filter the data plane VCN 918. In this embodiment, the customer can determine what the data plane VCN 918 can access, and the customer can limit access to the public internet 954 from the data plane VCN 918. It may not be possible for the IaaS provider to filter or otherwise control the data plane VCN 918's access to external networks or databases. The application of filters and controls by the customer to the data plane VCN 918 contained in the customer tenancy 921 can facilitate isolating the data plane VCN 918 from other customers and from the public internet 954.

[0165] In an embodiment, cloud services 956 can be called by the service gateway 936 to access services that may not be on the public internet 954, the control plane VCN 916, or the data plane VCN 918. The connection between the cloud services 956 and the control plane VCN 916 or the data plane VCN 918 may not be live or continuous. The cloud services 956 can be on different networks owned or operated by the IaaS provider. The cloud services 956 can be configured to receive calls from the service gateway 936 and not receive calls from the public internet 954. Some cloud services 956 can be isolated from other cloud services 956, and the control plane VCN 916 can be isolated from cloud services 956 that may not be in the same region as the control plane VCN 916. For example, the control plane VCN 916 may be located in "Region 1" and cloud service "Deployment 8" may be located in Region 1 and Region 2. If a call is made to deployment 8 by a service gateway 936 included in a control plane VCN 916 located in region 1, the call can be sent to deployment 8 in region 1. In this example, control plane VCN 916, or deployment 8 in region 1, cannot be communicatively coupled or otherwise communicate with deployment 8 in region 2.

[0166] 10 is a block diagram 1000 illustrating another example pattern of an IaaS architecture, according to at least one embodiment. A service operator 1002 (e.g., service operator 802 in FIG. 8 ) can be communicatively connected to a secure host tenancy 1004 (e.g., secure host tenancy 804 in FIG. 8 ), which can include a virtual cloud network (VCN) 1006 (e.g., VCN 806 in FIG. 8 ), and a second secure host subnet 1008 (e.g., secure host subnet 808 in FIG. 8 ). VCN 1006 can include an LPG 1010 (e.g., LPG 810 in FIG. 8 ), which can be communicatively connected to an SSH VCN 1012 (e.g., SSH VCN 812 in FIG. 8 ) via an LPG 1010 included in SSH VCN 1012. SSH VCN 1012 can include an SSH subnet 1014 (e.g., SSH subnet 814 in FIG. 8 ), which can be communicatively connected to a control plane VCN 1016 (e.g., control plane VCN 816 in FIG. 8 ) via an LPG 1010 included in control plane VCN 1016, and to a data plane VCN 1018 (e.g., data plane 818 in FIG. 8 ) via an LPG 1010 included in data plane VCN 1018. The control plane VCN 1016 and the data plane VCN 1018 can be included in a service tenancy 1019 (e.g., service tenancy 819 in FIG. 8 ).

[0167] The control plane VCN 1016 may include a control plane DMZ tier 1020 (e.g., control plane DMZ tier 820 of FIG. 8 ) that may include a load balancer (LB) subnet 1022 (e.g., LB subnet 822 of FIG. 8 ), a control plane application tier 1024 (e.g., control plane application tier 824 of FIG. 8 ) that may include an application subnet 1026 (e.g., similar to application subnet 826 of FIG. 8 ), and a control plane data tier 1028 (e.g., control plane data tier 828 of FIG. 8 ) that may include a DB subnet 1030. The LB subnet 1022 included in the control plane DMZ tier 1020 can be communicatively coupled to an application subnet 1026 included in the control plane application tier 1024 and an Internet gateway 1034 (e.g., Internet gateway 834 in FIG. 8 ) that may be included in the control plane VCN 1016, and the application subnet 1026 can be communicatively coupled to a DB subnet 1030 included in the control plane data tier 1028 and a service gateway 1036 (e.g., service gateway in FIG. 8 ) and a network address translation (NAT) gateway 1038 (e.g., NAT gateway 838 in FIG. 8 ). The control plane VCN 1016 can include the service gateway 1036 and the NAT gateway 1038.

[0168] The data plane VCN 1018 may include a data plane application layer 1046 (e.g., data plane application layer 846 of FIG. 8 ), a data plane DMZ layer 1048 (e.g., data plane DMZ layer 848 of FIG. 8 ), and a data plane data layer 1050 (e.g., data plane data layer 850 of FIG. 8 ). The data plane DMZ layer 1048 may include a LB subnetwork 1022 that may be communicatively coupled to a trusted application subnetwork 1060 and an untrusted application subnetwork 1062 of the data plane application layer 1046 and an Internet gateway 1034 included in the data plane VCN 1018. The trusted application subnetwork 1060 may be communicatively coupled to a service gateway 1036 included in the data plane VCN 1018, a NAT gateway 1038 included in the data plane VCN 1018, and a DB subnetwork 1030 included in the data plane data layer 1050. The untrusted application subnet 1062 may be communicatively coupled to a service gateway 1036 included in the data plane VCN 1018 and a DB subnet 1030 included in the data plane data layer 1050. The data plane data layer 1050 may include a DB subnet 1030 that may be communicatively coupled to a service gateway 1036 included in the data plane VCN 1018.

[0169] The untrusted application subnet 1062 may include one or more primary VNICs 1064(1)-(N) communicatively coupled to tenant virtual machines (VMs) 1066(1)-(N). Each tenant VM 1066(1)-(N) may be communicatively coupled to a respective application subnet 1067(1)-(N) that may be included in a respective container egress VCN 1068(1)-(N) that may be included in a respective customer tenancy 1070(1)-(N). Each secondary VNIC 1072(1)-(N) may facilitate communication between the untrusted application subnet 1062 included in the data plane VCN 1018 and the application subnet included in the container egress VCN 1068(1)-(N). Each container egress VCN 1068(1)-(N) may include a NAT gateway 1038 communicatively coupled to the public Internet 1054 (e.g., public Internet 854 of FIG. 8).

[0170] An Internet gateway 1034 included in the control plane VCN 1016 and included in the data plane VCN 1018 may be communicatively coupled to a metadata management service 1052 (e.g., metadata management system 852 of FIG. 8 ), which may be communicatively coupled to the public Internet 1054. The public Internet 1054 may be communicatively coupled to a NAT gateway 1038 included in the control plane VCN 1016 and included in the data plane VCN 1018. A service gateway 1036 included in the control plane VCN 1016 and included in the data plane VCN 1018 may be communicatively coupled to cloud services 1056.

[0171] In one embodiment, the data plane VCN 1018 can be integrated with a customer tenancy 1070. This integration can be useful or desirable in some cases to an IaaS provider's customer, such as when support is desired when executing code. A customer can provide code for execution that may be disruptive, communicate with other customer resources, or otherwise cause undesirable effects. In response, the IaaS provider can determine whether or not to execute the code provided to the IaaS provider by the customer.

[0172] In one example, an IaaS provider customer may be granted temporary network access to the IaaS provider and may request to add functionality to the data plane application layer 1046. The code to perform that functionality may be executed in VMs 1066(1)-(N), and the code may not be configured to run anywhere else on the data plane VCN 1018. Each VM 1066(1)-(N) may be connected to one customer tenancy 1070. Each container 1071(1)-(N) contained in a VM 1066(1)-(N) may be configured to execute this code. In this case, there may be double isolation (e.g., containers 1071(1)-(N) may execute code, and containers 1071(1)-(N) may be at least contained in VMs 1066(1)-(N) contained in the untrusted application subnet 1062), which may help prevent malformed or otherwise undesirable code from damaging the IaaS provider's network or from damaging a different customer's network. Containers 1071(1)-(N) can be communicatively coupled to customer tenancy 1070 and can be configured to send or receive respective data to or from customer tenancy 1070. Containers 1071(1)-(N) cannot be configured to send or receive data to or from any other entities in data plane VCN 1018. Upon completion of code execution, the IaaS provider can disable or otherwise discard containers 1071(1)-(N).

[0173] In one embodiment, the trusted application subnet 1060 can execute code owned or operated by the IaaS provider. In this embodiment, the trusted application subnet 1060 is communicatively coupled to the DB subnet 1030 and can be configured to perform CRUD operations on the DB subnet 1030. The untrusted application subnet 1062 can be communicatively connected to the DB subnet 1030, but in this embodiment, the untrusted application subnet can be configured to perform read operations on the DB subnet 1030. Containers 1071(1)-(N) that can be included in each customer's VMs 1066(1)-(N) and that can execute code from the customer cannot be communicatively coupled to the DB subnet 1030.

[0174] In other embodiments, the control plane VCN 1016 and the data plane VCN 1018 may not be directly communicatively coupled. In this embodiment, there may be no direct communication between the control plane VCN 1016 and the data plane 1018. However, communication may be indirect by at least one method. An LPG 1010 may be constructed by an IaaS provider that may facilitate communication between the control plane VCN 1016 and the data plane VCN 1018. In another example, the control plane VCN 1016 or the data plane VCN 1018 may make a call to a cloud service 1056 through a service gateway 1036. For example, a call from the control plane VCN 1016 to the cloud service 1056 may include a request for a service that may communicate with the data plane VCN 1018.

[0175] 11 is a block diagram 1100 illustrating another example pattern of an IaaS architecture, according to at least one embodiment. A service operator 1102 (e.g., service operator 802 in FIG. 8 ) can be communicatively connected to a secure host tenancy 1104 (e.g., secure host tenancy 804 in FIG. 8 ), which can include a virtual cloud network (VCN) 1106 (e.g., VCN 806 in FIG. 8 ) and a secure host subnet 1108 (e.g., secure host subnet 808 in FIG. 8 ). VCN 1106 can include an LPG 1110 (e.g., LPG 810 in FIG. 8 ), which can be communicatively coupled to an SSH VCN 1112 (e.g., SSH VCN 812 in FIG. 8 ) via an LPG 1110 included in SSH VCN 1112. SSH VCN 1112 can include an SSH subnet 1114 (e.g., SSH subnet 814 in FIG. 8 ), which can be communicatively coupled to a control plane VCN 1116 (e.g., control plane VCN 816 in FIG. 8 ) via an LPG 1110 included in control plane VCN 1116 and to a data plane VCN 1118 (e.g., data plane 818 in FIG. 8 ) via an LPG 1110 included in data plane VCN 1118. The control plane VCN 1116 and the data plane VCN 1118 can be included in a service tenancy 1119 (e.g., service tenancy 819 in FIG. 8 ).

[0176] The control plane VCN 1116 may include a control plane DMZ layer 1120 (e.g., control plane DMZ layer 820 of FIG. 8 ) that may include a LB subnet 1122 (e.g., LB subnet 822 of FIG. 8 ), a control plane application layer 1124 (e.g., control plane application layer 824 of FIG. 8 ) that may include an application subnet 1126 (e.g., application subnet 826 of FIG. 8 ), and a control plane data layer 1128 (e.g., control plane data layer 828 of FIG. 8 ) that may include a DB subnet 1130 (e.g., DB subnet 1030 of FIG. 10 ). The LB subnet 1122 included in the control plane DMZ tier 1120 can be communicatively coupled to an application subnet 1126 included in the control plane application tier 1124 and to an Internet gateway 1134 (e.g., Internet gateway 834 in FIG. 8 ) that may be included in the control plane VCN 1116, and the application subnet 1126 can be communicatively coupled to a DB subnet 1130 included in the control plane data tier 1128 and to a service gateway 1136 (e.g., service gateway in FIG. 8 ) and a network address translation (NAT) gateway 1138 (e.g., NAT gateway 838 in FIG. 8 ). The control plane VCN 1116 can include the service gateway 1136 and the NAT gateway 1138.

[0177] The data plane VCN 1118 may include a data plane application layer 1146 (e.g., data plane application layer 846 in FIG. 8 ), a data plane DMZ layer 1148 (e.g., data plane DMZ layer 848 in FIG. 8 ), and a data plane data layer 1150 (e.g., data plane data layer 850 in FIG. 8 ). The data plane DMZ layer 1148 may include a LB subnetwork 1122 that may be communicatively coupled to a trusted application subnetwork 1160 (e.g., trusted application subnetwork 1060 in FIG. 10 ), an untrusted application subnetwork 1162 (e.g., untrusted application subnetwork 1062 in FIG. 10 ) of the data plane application layer 1146, and an Internet gateway 1134 included in the data plane VCN 1118. The trusted application subnetwork 1160 may be communicatively coupled to a service gateway 1136 included in the data plane VCN 1118, a NAT gateway 1138 included in the data plane VCN 1118, and a DB subnetwork 1130 included in the data plane data layer 1150. The untrusted application subnet 1162 may be communicatively coupled to a service gateway 1136 included in the data plane VCN 1118 and a DB subnet 1130 included in the data plane data layer 1150. The data plane data layer 1150 may include the DB subnet 1130, which may be communicatively coupled to a service gateway 1136 included in the data plane VCN 1118.

[0178] The untrusted application subnet 1162 may include primary VNICs 1164(1)-(N) that may be communicatively coupled to tenant virtual machines (VMs) 1166(1)-(N) that are in the untrusted application subnet 1162. Each tenant VM 1166(1)-(N) may execute code in a respective container 1167(1)-(N) and may be communicatively coupled to an application subnet 1126 that may be included in a data plane application tier 1146 that may be included in a container egress VCN 1168. Each secondary VNIC 1172(1)-(N) may facilitate communication between the untrusted application subnet 1162 included in the data plane VCN 1118 and the application subnet included in the container egress VCN 1168. The container egress VCN may include a NAT gateway 1138 that may be communicatively coupled to the public Internet 1154 (e.g., public Internet 854 of FIG. 8 ).

[0179] An Internet gateway 1134 included in the control plane VCN 1116 and included in the data plane VCN 1118 may be communicatively coupled to a metadata management service 1152 (e.g., metadata management system 852 of FIG. 8 ), which may be communicatively coupled to the public Internet 1154. The public Internet 1154 may be communicatively coupled to a NAT gateway 1138 included in the control plane VCN 1116 and included in the data plane VCN 1118. A service gateway 1136 included in the control plane VCN 1116 and included in the data plane VCN 1118 may be communicatively coupled to cloud services 1156.

[0180] In one example, the pattern illustrated by the architecture of block diagram 1100 of FIG. 11 can be considered an exception to the pattern illustrated by the architecture of block diagram 1000 of FIG. 10 and may be desirable for customers of an IaaS provider when the IaaS provider cannot communicate directly with the customer (e.g., disconnected region). Each container 1167(1)-(N) contained in a VM 1166(1)-(N) for each customer is accessible in real time by that customer. The containers 1167(1)-(N) can be configured to make calls to a respective secondary VNIC 1172(1)-(N) contained in an application subnet 1126 of a data plane application layer 1146 that can be contained in a container egress VCN 1168. The secondary VNICs 1172(1)-(N) can send the calls to a NAT gateway 1138 that can send the calls to the public Internet 1154. In this example, containers 1167(1)-(N) that are accessible to a customer in real time can be isolated from the control plane VCN 1116 and can be isolated from other entities contained within the data plane VCN 1118. Containers 1167(1)-(N) can also be isolated from resources from other customers.

[0181] In another example, a customer can use container 1167(1)-(N) to invoke cloud service 1156. In this example, the customer can execute code in container 1167(1)-(N) that requests a service from cloud service 1156. Container 1167(1)-(N) can send the request to secondary VNIC 1172(1)-(N), which can send the request to a NAT gateway, which can send the request to public Internet 1154. Public Internet 1154 can send the request via Internet gateway 1134 to LB subnet 1122 included in control plane VCN 1116. In response to determining that the request is valid, the LB subnet can send the request to application subnet 1126, which can send the request via service gateway 1136 to cloud service 1156.

[0182] It should be understood that the IaaS architectures 800, 900, 1000, 1100 depicted in the figures may have components other than those depicted, and the depicted embodiments are only some examples of cloud infrastructure systems that may incorporate embodiments of the present disclosure. In some other embodiments, the IaaS systems may have more or fewer components than depicted in the figures, may combine two or more components, or may have a different configuration or arrangement of components.

[0183] In one embodiment, the IaaS system described herein may include a set of application, middleware, and database service offerings that are delivered to customers in a self-service, subscription-based, elastically scalable, reliable, highly available, and secure manner. One example of such an IaaS system is Oracle Cloud Infrastructure (OCI), offered by the assignee of the present application.

[0184] 12 illustrates an example of a computer system 1200 in which various embodiments can be implemented. The system 1200 can be used to implement any of the computer systems described above. As shown, the computer system 1200 includes a processing unit 1204 that communicates with several peripheral subsystems via a bus subsystem 1202. These peripheral subsystems may include a processing acceleration unit 1206, an I / O subsystem 1208, a storage subsystem 1218, and a communication subsystem 1224. The storage subsystem 1218 includes a tangible computer-readable storage medium 1222 and a system memory 1210.

[0185] Bus subsystem 1202 provides a mechanism for allowing the various components and subsystems of computer system 1200 to communicate with each other as desired. Although bus subsystem 1202 is shown diagrammatically as a single bus, alternative embodiments of the bus subsystem may use multiple buses. Bus subsystem 1202 may be any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. For example, such architectures may include a Peripheral Component Interconnect (PCI) bus, which may be implemented as an Industry Standard Architecture (ISA) bus, a MicroChannel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, and a mezzanine bus manufactured in accordance with the IEEE P1386.1 standard.

[0186] A processing unit 1204, which may be implemented as one or more integrated circuits (e.g., conventional microprocessors or microcontrollers), controls the operation of the computer system 1200. One or more processors may be included in the processing unit 1204. These processors may include single-core or multi-core processors. In an embodiment, the processing unit 1204 may be implemented as one or more independent processing units 1232 and / or 1234, with each processing unit including a single-core or multi-core processor. In other embodiments, the processing unit 1204 may be implemented as a quad-core processing unit formed by integrating two dual-core processors on a single chip.

[0187] In various embodiments, the processing unit 1204 may execute various programs in response to program code and may maintain multiple parallel executing programs or processes. At any given time, some or all of the program code being executed may reside within the processor 1204 and / or within the storage subsystem 1218. With appropriate programming, the processor 1204 may provide various functions as discussed above. The computer system 1200 may further include a processing acceleration unit 1206, which may include a digital signal processor (DSP), special purpose processor, and / or the like.

[0188] The I / O subsystem 1208 may include user interface input devices and user interface output devices. User interface input devices may include a keyboard, a pointing device such as a mouse or trackball, a touchpad or touch screen integrated into a display, a scroll wheel, a click wheel, a dial, a button, a switch, a keypad, a voice input device with a voice command recognition system, a microphone, and other types of input devices. User interface input devices may include motion sensing and / or gesture recognition devices, such as, for example, a Microsoft Kinect® motion sensor that allows a user to control and interact with an input device, such as a Microsoft Xbox® 360 game controller, through a natural user interface using gestures and speech commands. User interface input devices may also include eye gesture recognition devices, such as a Google Glass® blink detector, that detects eye activity from a user (e.g., “blinking” when taking a picture and / or selecting a menu) and translates the eye gesture as input to an input device (e.g., Google Glass®). Additionally, the user interface input devices may include a voice recognition sensing device that allows a user to interact with a voice recognition system (eg, the Siri® navigator) via voice commands.

[0189] User interface input devices may include, but are not limited to, three-dimensional (3D) mice, joysticks or pointing sticks, game pads, and graphic tablets, as well as audio / visual devices such as speakers, digital cameras, digital camcorders, portable media players, webcams, image scanners, fingerprint scanners, barcode readers 3D scanners, 3D printers, laser distance meters, and eye-tracking systems. Additionally, user interface input devices may include medical imaging input devices, such as, for example, computed tomography, magnetic resonance imaging, positron emission tomography, and medical ultrasound devices. User interface input devices may also include audio input devices, such as, for example, MIDI keyboards, digital musical instruments, and the like.

[0190] User interface output devices may include a display subsystem, non-visual displays such as indicator lights or audio output devices. The display subsystem may be a flat panel device such as one using a cathode ray tube (CRT), a liquid crystal display (LCD) or a plasma display, a projection device, a touch screen, etc. In general, use of the term "output device" is intended to include any type of possible device and mechanism for outputting information from computer system 1200 to a user or to another computer. For example, user interface output devices may include a variety of display devices that visually convey text, graphics and audio / video information, such as, but not limited to, monitors, printers, speakers, headphones, automobile navigation systems, plotters, audio output devices and modems.

[0191] Computer system 1200 may include a storage subsystem 1218 that provides a tangible, non-transitory computer-readable storage medium for storing software and data structures that provide functionality of embodiments described in this disclosure. The software may include programs, code modules, instructions, scripts, etc. that, when executed by one or more cores or processors of processing unit 1204, provide the functionality described above. Storage subsystem 1218 may also provide a repository for storing data used in accordance with the present disclosure.

[0192] As shown in Figure 12, storage subsystem 1218 may include various components including a system memory 1210, a computer readable storage medium 1222, and a computer readable storage medium reader 1220. The system memory 1210 may store program instructions that are loadable and executable by the processing unit 1204. The system memory 1210 may also store data used during execution of the instructions and / or data generated during execution of the program instructions. A variety of different types of programs may be loaded into the system memory 1210, including, but not limited to, client applications, web browsers, mid-tier applications, relational database management systems (RDBMS), virtual machines, containers, etc.

[0193] The system memory 1210 may also store an operating system 1216. Examples of operating systems 1216 may include Microsoft Windows, Apple Macintosh, and / or Linux operating systems, various commercially available UNIX or UNIX-like operating systems (including, but not limited to, the GNU / Linux operating system, Google Chrome OS, etc.), and / or mobile operating systems, such as iOS, Windows Phone, Android OS, BlackBerry OS, and Palm OS operating systems. In an embodiment in which the computer system 1200 runs one or more virtual machines, the virtual machines, along with their guest operating systems (GOS), may be loaded into the system memory 1210 and executed by one or more processors or cores of the processing unit 1204.

[0194] The system memory 1210 may be provided in different configurations depending on the type of computer system 1200. For example, the system memory 1210 may be volatile memory (such as random access memory (RAM)) and / or non-volatile memory (such as read only memory (ROM), flash memory, etc.). Different types of RAM configurations can be provided, including static random access memory (SRAM), dynamic random access memory (DRAM), etc. In some embodiments, the system memory 1210 may include a basic input / output system (BIOS), which contains the basic routines that help to transfer information between elements within the computer system 1200, such as during start-up.

[0195] Computer-readable storage medium 1222 may represent remote, local, fixed and / or removable storage devices, as well as storage media for temporarily and / or more permanently containing and storing computer-readable information for use by computer system 1200, including instructions executable by processing unit 1204 of computer system 1200.

[0196] The computer readable storage medium 1222 may include any suitable medium known or used in the art, including storage media and communication media, such as, but not limited to, volatile and non-volatile, removable and non-removable media, implemented in any method or technology for storage and / or transmission of information. This may include tangible computer readable storage media, such as RAM, ROM, Electrically Erasable Programmable ROM (EEPROM), Flash memory or other memory technology, CD-ROM, Digital Versatile Disk (DVD) or other optical storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or other tangible computer readable media.

[0197] For example, the computer readable storage medium 1222 may include hard disk drives that read or write non-removable non-volatile magnetic media, magnetic disk drives that read or write removable non-volatile magnetic disks, and optical disk drives that read or write removable non-volatile optical disks such as CD ROMs, DVDs and Blu-Ray disks or other optical media. The computer readable storage medium 1222 may include, but is not limited to, Zip drives, flash memory cards, Universal Serial Bus (USB) flash drives, Secure Digital (SD) cards, DVD disks, digital video tapes, and the like. The computer readable storage medium 1222 may also include solid state drives (SSDs) based on non-volatile memory such as flash memory-based SSDs, enterprise flash drives, solid state ROMs, SSDs based on volatile memory such as solid state RAM, dynamic RAM, static RAM, DRAM-based SSDs, magnetoresistive RAM (MRAM) SSDs, and hybrid SSDs that use a combination of DRAM and flash memory-based SSDs. The disk drives and their associated computer-readable media may provide non-volatile storage of computer-readable instructions, data structures, program modules and other data for the computer system 1200.

[0198] Machine-readable instructions executable by one or more processors or cores of the processing unit 1204 may be stored on a non-transitory computer-readable storage medium. A non-transitory computer-readable storage medium may include physically tangible memory or storage devices, including volatile memory storage devices and / or non-volatile storage devices. Examples of non-transitory computer-readable storage media include magnetic storage media (e.g., disks or tapes), optical storage media (e.g., DVDs, CDs), various types of RAM, ROM or flash memory, hard drives, floppy drives, removable memory drives (e.g., USB drives), or other types of storage devices.

[0199] The communications subsystem 1224 provides an interface to other computer systems and networks. The communications subsystem 1224 serves as an interface for receiving and transmitting data to and from systems other than the computer system 1200. For example, the communications subsystem 1224 may enable the computer system 1200 to connect to one or more devices via the Internet. In some embodiments, the communications subsystem 1224 may include a radio frequency (RF) transceiver component for accessing wireless voice and / or data networks (e.g., using cellular technology, advanced data network technologies such as 3G, 4G, or EDGE (Enhanced Data Rates for Global Evolution), WiFi (IEEE 802.11 family standard, or other mobile communications technologies, or any combination thereof), a global positioning system (GPS) receiver component, and / or other components. In some embodiments, the communications subsystem 1224 may provide a wired network connection (e.g., Ethernet) in addition to or instead of a wireless interface.

[0200] In some embodiments, the communications subsystem 1224 may also receive incoming communications in the form of structured and / or unstructured data feeds 1226, event streams 1228, event updates 1230, etc., on behalf of one or more users who may use the computer system 1200.

[0201] For example, the communications subsystem 1224 can be configured to receive data feeds 1226 in real time from users of social networks and / or other communications services, such as web feeds, such as Twitter® feeds, Facebook® updates, Rich Site Summary (RSS) feeds, and / or real-time updates from one or more third party sources.

[0202] Additionally, the communications subsystem 1224 may be configured to receive data in the form of a continuous data stream, which may include an event stream 1228 of real-time events and / or event updates 1230, which may be of a continuous or unbounded nature with no apparent end. Examples of applications that generate continuous data may include, for example, sensor data applications, financial tickers, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, automobile traffic monitoring, etc.

[0203] The communications subsystem 1224 may also be configured to output structured and / or unstructured data feeds 1226, event streams 1228, event updates 1230, etc. to one or more databases in communication with one or more streaming data source computers coupled to the computer system 1200.

[0204] The computer system 1200 may be one of a variety of types, including a handheld portable device (e.g., an iPhone® mobile phone, an iPad® computing tablet, a PDA), a wearable device (e.g., a Google Glass® head mounted display), a PC, a workstation, a mainframe, a kiosk, a server rack, or any other data processing system.

[0205] Due to the ever-changing nature of computers and networks, the description of the computer system 1200 shown in the figure is intended as an example only. Many other configurations having more or fewer components than the system shown in the figure are possible. For example, customized hardware may also be used, and / or particular elements may be implemented in hardware, firmware, software (including applets), or a combination thereof. Also, connections to other computing devices, such as network input and / or output devices, may be employed. Based on this disclosure and the teachings presented herein, one of ordinary skill in the art will recognize other ways and / or methods of implementing various embodiments.

[0206] Although specific embodiments have been described, various modifications, variations, alternative constructions and equivalents are encompassed within the scope of the disclosure. The embodiments are not limited to operating in one particular data processing environment, but can freely operate in multiple data processing environments. Furthermore, while the embodiments have been described using a particular sequence of transactions and steps, it will be apparent to those skilled in the art that the scope of the disclosure is not limited to the sequence of transactions and steps described. Various features and aspects of the above-described embodiments can be used individually or in combination.

[0207] Also, while the embodiments are described using a particular combination of hardware and software, it should be understood that other combinations of hardware and software are within the scope of the present disclosure. The embodiments can be implemented using only hardware, only software, or a combination thereof. The embodiments can be implemented using a computer program product that includes computer programs / instructions that, when executed by a processor, cause the processor to perform any of the methods described in the present disclosure. The various processes described herein can be performed on the same processor or on different processors in any combination. Thus, when a component or service is described as being configured to perform an operation, such configuration can be achieved, for example, by designing an electronic circuit to perform the operation, or by programming a programmable electronic circuit (such as a microprocessor) to perform the operation, or a combination thereof. The processes can communicate using a variety of techniques, including but not limited to conventional techniques for inter-process communication, and different pairs of processes may use different techniques, or the same pair of processes may use different techniques at different times.

[0208] Accordingly, the specification and drawings are to be regarded in an illustrative and not a restrictive sense. However, it will be apparent that additions, substitutions, deletions and other modifications and alterations may be made thereto without departing from the broader spirit and scope of the appended claims. Thus, although certain disclosed embodiments have been described, they are not intended to be limiting. Various modifications and equivalents are also intended to be encompassed within the scope of the appended claims.

[0209] Use of the terms "a," "an," "the," and similar referents in the context of describing embodiments of the present disclosure (particularly in the context of the appended claims) should be construed to include both the singular and the plural, unless otherwise indicated herein or clearly contradicted by context. The terms "comprising," "having," "including," and "containing" should be construed as open-ended (i.e., meaning "including, but not limited to"), unless otherwise indicated. The term "connected" should be construed as being partly or wholly contained within, attached to, or joined to one another, even if there are intervening elements. The recitation of ranges of values ​​herein is merely intended to serve as a shorthand method of referring individually to each separate value falling within the range, unless otherwise indicated herein, and each separate value is incorporated herein as if it were individually set forth herein. All methods described herein can be performed in any suitable order, unless otherwise indicated herein or clearly contradicted by context. The use of any examples and illustrative language (e.g., "etc.") described herein, unless otherwise required, is intended merely to facilitate a better understanding of the embodiments and does not limit the scope of the disclosure. No language in the specification should be construed as indicating any non-claimed element as essential to the practice of the disclosure.

[0210] Disjunctive phrases such as "at least one of X, Y, or Z" are intended to be interpreted as generally used within the context to indicate that an item, term, etc., can be either X, Y, or Z, or any combination thereof (e.g., X, Y and / or Z), unless specifically stated otherwise. Thus, such disjunctive phrases are generally not intended, and should not imply, that an embodiment requires that at least one of X, at least one of Y, or at least one of Z, respectively, be present.

[0211] Preferred embodiments of the present disclosure are described herein, including the best mode known for carrying out the present disclosure. Variations of these preferred embodiments will become apparent to those skilled in the art upon reading the above description. Those skilled in the art should be able to adopt such variations as appropriate, and the present disclosure may be carried out in a manner different from that specifically described herein. Accordingly, the present disclosure includes all modifications and equivalents of the subject matter recited in the claims appended hereto as permitted by applicable law. Also, unless otherwise indicated herein, any combination of the above-described elements in all its possible variations is encompassed by the present disclosure.

[0212] All references cited in this specification, including publications, patent applications, and patents, are hereby incorporated by reference to the same extent as if each reference was individually and specifically incorporated by reference and was set forth in its entirety herein.

[0213] Although aspects of the disclosure have been described hereinabove with reference to specific embodiments thereof, those skilled in the art will recognize that the disclosure is not limited thereto. Various features and aspects of the disclosure above can be used individually or in combination. Also, the embodiments can be used in any number of environments and applications other than those described herein without departing from the broader spirit and scope of the specification. Accordingly, the specification and drawings should be regarded as illustrative rather than restrictive.

Claims

1. One or more non-transitory computer-readable media having instructions stored thereon that, when executed by one or more processors, cause the system to perform a process, the process comprising: receiving a request to set up a new data center in a region of a cloud services infrastructure, the request including a configuration for the new data center; determining a set of one or more services to be provided by the new data center based at least in part on the configuration; determining seed data for a set of seed instructions for provisioning the set of one or more services at the new data center based at least in part on the set of one or more services; generating the set of seed instructions for provisioning the set of one or more services at the new data center based at least in part on the seed data; and providing the set of seed instructions to provision the new data center with a seed infrastructure to provide the set of one or more services.

2. 10. The one or more non-transitory computer-readable media of claim 1, wherein using the set of seed instructions to provision the new data center occurs before one or more control planes are configured for the new data center.

3. 3. The one or more non-transitory computer-readable media of claim 2, wherein the one or more control planes include an identity control plane, a compartment control plane, or a limit control plane.

4. 11. The one or more non-transitory computer-readable media of any preceding claim, wherein the seed data includes a bare metal cloud operator access tenancy (BOAT) pre-sync file, a tenancy creation mechanism file, and a state file for the new data center, and generating the set of seed instructions includes combining the BOAT pre-sync file, the tenancy creation mechanism file, and the state file.

5. 13. The one or more non-transitory computer-readable media of any preceding claim, wherein the region is a new region.

6. The one or more non-transitory computer-readable media of claim 5 , wherein the new region is within a new realm.

7. 11. The one or more non-transitory computer-readable media of any preceding claim, wherein the request indicates a region type for the new data center, and the seed data for the set of seed instructions is further determined based at least in part on the indicated region type.

8. determining the seed data includes identifying one or more groups of seed data stored in a data store corresponding to the set of one or more stored services, the data store storing a plurality of groups of seed data, each group of the plurality of groups of seed data being associated with a corresponding service provided by a data center; Generating the set of seed instructions includes: retrieving the one or more groups of seed data from the data store; and combining the one or more groups of seed data to generate the set of seed instructions.

9. 10. The one or more non-transitory computer-readable media of claim 8, wherein the request indicates a region type for the new data center, each of the groups of seed data being further associated with a corresponding region type, and the one or more groups of seed data being further identified based at least in part on the indicated region type for the new data center.

10. 10. The one or more non-transitory computer-readable media of claim 8 or claim 9, wherein the set of one or more services includes a first service and a second service, identifying the one or more groups of seed data includes identifying a first group of seed data corresponding to the first service and a second group of seed data corresponding to the second service, retrieving the one or more groups of seed data includes retrieving the first group of seed data and the second group of seed data from the data store, and combining the one or more groups of seed data includes combining the first group of seed data and the second group of seed data to generate the set of seed instructions.

11. 13. The one or more non-transitory computer-readable media of any preceding claim, wherein provisioning the set of seed instructions in the new data center is performed without the use of application programming interface (API) calls.

12. 11. The one or more non-transitory computer-readable media of any preceding claim, wherein determining the seed data includes determining a Bare Metal Cloud Operator Access Tenancy (BOAT) presync file to include in the set of seed instructions, the BOAT presync file including a cloud identifier for the new data center, and generating the set of seed instructions includes extracting the BOAT presync file from a BOAT presync.

13. 11. The one or more non-transitory computer-readable media of any preceding claim, wherein determining the seed data includes determining a tenancy creation facility file to include in the set of seed instructions, the tenancy creation facility file including administrative policies for the new data center, and generating the set of seed instructions includes retrieving the tenancy creation facility file from a tenancy creation facility.

14. 11. The one or more non-transitory computer-readable media of any preceding claim, wherein determining the seed data includes determining a state file to include in the set of seed instructions, the state file defining one or more services for the new data center, and generating the set of seed instructions includes retrieving the state file from an object store.

15. The instructions, when executed by the one or more processors, cause the system to:

5. The one or more non-transitory computer-readable media of any preceding claim, further comprising causing the seed infrastructure to perform further processing including assigning a special indication that the seed data is part of the new data center configuration process.

16. The instructions, when executed by the one or more processors, cause the system to:

16. The one or more non-transitory computer-readable media of claim 15, further processing including transitioning the seed infrastructure from the special display to a normal display based on the configuration of one or more services for the new data center, the normal display indicating that the seed infrastructure can be used in the same manner as the one or more services.

17. 1. A method comprising: receiving a request to set up a new data center in a region of a cloud services infrastructure, the request including a configuration for the new data center, the method further comprising: determining a set of one or more services to be offered by the new data center based at least in part on the configuration; determining, based at least in part on the set of one or more services, whether reference seed instructions corresponding to the set of one or more services are stored; generating a set of seed instructions for provisioning the set of one or more services at the new data center based on whether the reference seed instructions corresponding to the set of one or more services are stored; providing the set of seed instructions for provisioning the new data center to provide the set of one or more services.

18. 20. The method of claim 17, wherein the request includes a request to configure the new data center in a new region within a new realm of the cloud services infrastructure, the set of one or more services corresponding to the new data center in the new region within the new realm.

19. a memory storing a configuration for the data center; a processor, the processor comprising: receiving a request to establish a new data center within a region of a cloud services infrastructure, the request including a configuration for the new data center; storing the configuration for the new data center in the memory; determining a set of one or more services to be provided by the new data center based at least in part on the configuration; determining seed data for a set of seed instructions for provisioning of the set of one or more services to the new data center based at least in part on the set of one or more services; generating the set of seed instructions based at least in part on the seed data; providing said set of seed instructions for provisioning said set of one or more services.

20. determining the seed data includes identifying one or more groups of seed data corresponding to the set of one or more services stored in a data store, the data store storing a plurality of groups of seed data, each group of the plurality of groups of seed data being associated with a corresponding service offered by a data center; Generating the set of seed instructions includes: retrieving the one or more groups of seed data corresponding to the set of one or more services from the data store; and combining the one or more groups of seed data to generate the set of seed instructions.

Images