User Interface for Critical Path Resources

Abstract

The present embodiment relates to determining a critical path that identifies an order for bootstrapping a subset of resources in a datacenter under construction. A cloud infrastructure orchestration service (CIOS) can identify a collective set of capabilities individually associated with resources to be bootstrapped by the CIOS in the datacenter under construction from one or more configuration files. The CIOS can identify, for each respective capability, a first set of capabilities on which issuing the respective capability depends. A user input identifies a selected flock. For the selected flock, one or more unissued capabilities corresponding to at least one of the one or more capabilities associated with the selected flock can be identified, and a ranking for those capabilities can be derived. A visualization is generated that identifies a portion of the unissued capabilities corresponding to the selected flock, and the unissued capabilities are identified and arranged according to the ranking.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This PCT application is based on U.S. Provisional Patent Application No. 63 / 308,003, filed February 8, 2022, entitled "Techniques for Bootstrapping a Region Build," U.S. Provisional Patent Application No. 63 / 312,814, filed February 22, 2022, entitled "Techniques for Implementing Virtual Data Centers," U.S. Provisional Patent Application No. 63 / 314,982, filed February 28, 2022, entitled "User Interface for a Critical Path Resources," U.S. Provisional Patent Application No. 63 / 347,235, filed May 31, 2022, entitled "User Interface for Critical Path Resources," and U.S. Provisional Patent Application No. 63 / 347,235, filed May 31, 2022, entitled "User Interface for Critical Path Resources." This application claims priority to U.S. patent application Ser. No. 18 / 163,266, filed Feb. 1, 2023, entitled "Microbial and Microbial Resources," the disclosures of which are incorporated herein by reference in their entireties for all purposes.

[0002] FIELD OF THEINVENTION The disclosed technology relates to constructing data centers in cloud computing environments in an automated manner. More specifically, the present disclosure describes a number of user interfaces that may be constructed and utilized to present information regarding the availability of and / or dependencies between capabilities that represent units of change within the construction. [Background technology]

[0003] background Today, cloud infrastructure services utilize many individual services to build a datacenter (e.g., to bootstrap various resources into a datacenter in a particular geographic region). In some examples, a region is a logical abstraction that corresponds to a local geographic area in which one or more datacenters are (or will be) located. Building a datacenter may include provisioning and configuring infrastructure resources (e.g., for various services) and deploying code to these resources. Operations to build a datacenter may be collectively referred to as performing a "region build." Any suitable number of datacenters may be included in a region, and thus a region build may include operations to build multiple datacenters. When resources are bootstrapped into a datacenter, various capabilities may be published to indicate their availability.

[0004] Conventional tools for building regions require significant manual effort because bootstrapping operations for one service may depend on other features and / or services of the region that may not yet be available. For example, to bootstrap an application, both an object storage application and a cloud identity service may first need to be available in the region. However, in some instances, without the implementation of such dependent resources, the application may not be able to be bootstrapped or have its capabilities issued. As the number of service teams and regions increases, the tasks performed to orchestrate provisioning and deployment increase dramatically. Substantial reliance on manual effort to bootstrap services and / or build regions may be very time-consuming, introduce risks, and may not scale well.

[0005] To readily identify the discussion of any particular element or act, the most significant digit or digits in a reference number refer to the number of the figure in which that element is first introduced. [Brief description of the drawings]

[0006] [Figure 1] FIG. 1 is a block diagram of an environment in which a cloud infrastructure orchestration service (CIOS) 102 may operate to dynamically provide bootstrap services in a region, according to at least one embodiment. [Diagram 2] FIG. 1 is a block diagram illustrating an environment and method for building a Virtual Bootstrap Environment (ViBE) according to at least one embodiment. [Diagram 3] 1 is a block diagram illustrating an environment and method for bootstrapping a service into a target region utilizing ViBE, according to at least one embodiment. [Figure 4] 1 is a block diagram illustrating another exemplary CIOS in accordance with at least one embodiment. [Diagram 5] 1 is a block diagram illustrating capability dependencies in a CIOS, according to at least one embodiment. [Figure 6] FIG. 1 is a block diagram of an exemplary capability visualization generation system according to at least one embodiment. [Figure 7] 1 is a block diagram illustrating an example visualization of on-deck capabilities (e.g., capabilities for which all dependencies have been satisfied) in accordance with at least one embodiment. [Figure 8] FIG. 1 is a block diagram illustrating an example visualization of a blocked capability that depends on one or more unissued capabilities, according to at least one embodiment. [Figure 9] FIG. 1 is a block diagram illustrating an example method for generating a visualization of capabilities according to at least one embodiment. [Figure 10]1 is a block diagram illustrating an example mapping of dependencies for a selected block in a CIOS, according to at least one embodiment. [Figure 11] 1 is an example visualization illustrating a critical path for unblocking a specified resource, according to at least one embodiment. [Figure 12] 1 is a flow process for generating a visualization of a critical path for issuing dependent capabilities to unblock a flock of resources, according to at least one embodiment. [Figure 13] FIG. 1 is a block diagram illustrating one pattern for implementing a cloud infrastructure as a service system, according to at least one embodiment. [Figure 14] FIG. 1 is a block diagram illustrating another pattern for implementing a cloud infrastructure as a service system in accordance with at least one embodiment. [Figure 15] FIG. 1 is a block diagram illustrating another pattern for implementing a cloud infrastructure as a service system in accordance with at least one embodiment. [Figure 16] FIG. 1 is a block diagram illustrating another pattern for implementing a cloud infrastructure as a service system in accordance with at least one embodiment. [Figure 17] FIG. 1 is a block diagram illustrating an exemplary computer system in accordance with at least one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] Detailed Description In the following description, for purposes of explanation, specific details are set forth in order to provide a thorough understanding of certain embodiments. It will be apparent, however, that various embodiments may be practiced without these specific details. The drawings and description are not intended to be limiting. The term "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any embodiment or design described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments or designs.

[0008] An example of an automated data center (region) construction infrastructure The adoption of cloud services has shown a rapid increase in recent years. Now, various types of cloud services are offered by various different cloud service providers (CSPs). The term cloud service is generally used to refer to services or functions made available by a CSP to a user or customer on demand (e.g., via a subscription model) using the systems and infrastructure (cloud infrastructure) offered by the CSP. Typically, the servers and systems that make up the CSP's infrastructure used to provide cloud services to the customer are separate from the customer's own on-premise operated servers and systems. Thus, the customer can utilize the cloud services offered by the CSP without having to purchase separate hardware and software resources for the service. Cloud services are designed to provide subscribing customers with easy, scalable and on-demand access to applications and computing resources without the customer having to invest in procuring the infrastructure used to provide the service or functionality. Various different types or models of cloud services may be offered, such as software as a service (SaaS), platform as a service (PaaS), infrastructure as a service (IaaS), and others. A customer can subscribe to one or more cloud services offered by a CSP. A customer can be any entity, such as an individual, an organization, or a business.

[0009] As indicated above, a CSP is responsible for providing the infrastructure and resources used to provide cloud services to subscribing customers. The resources provided by a CSP can include both hardware and software resources. These resources can include, for example, computational resources (e.g., virtual machines, containers, applications, processors), memory resources (e.g., databases, data stores), network resources (e.g., routers, host machines, load balancers), identities, and other resources. In one implementation, the resources provided by a CSP to provide a set of cloud services CSPs are organized into a datacenter. A datacenter may be configured to provide a particular set of cloud services. The CSP is responsible for outfitting the datacenter with the infrastructure and resources used to provide that particular set of cloud services. A CSP may build one or more datacenters.

[0010] Data centers provided by a CSP may be hosted in different regions. A region is a local geographic area and may be identified by a region name. Regions are generally independent of each other and can be separated by large distances, such as across countries or even continents. Regions are grouped into realms. Examples of regions for a CSP may include US West, US East, Australia East, Australia Southeast, etc.

[0011] A region may include one or more data centers, the data centers being located within a certain geographic area corresponding to the region. As an example, the data centers in a region may be located in cities within the region. For example, for a particular CSP, a data center in the US West region may be located in San Jose, California, a data center in the US East region may be located in Ashburn, Virginia, a data center in the Australia East region may be located in Sydney, Australia, a data center in the Australia Southeast region may be located in Melbourne, Australia, etc.

[0012] Data centers within a region may be organized into one or more availability domains, which are used for high availability and disaster recovery purposes. An availability domain may include one or more data centers within a region. Availability domains within a region are isolated from each other, fault tolerant, and designed to make it highly unlikely that data centers in multiple availability domains will fail simultaneously. For example, availability domains within a region may be constructed to make it highly unlikely that a failure in one availability domain within a region will affect the availability of data centers in other availability domains within the same region.

[0013] When a customer or subscriber subscribes to or contracts with one or more services provided by a CSP, the CSP creates a tenancy for the customer. A tenancy is like an account created for a customer. In one implementation, a tenancy for a customer exists in one realm and can access all regions that belong to that realm. Thus, the customer's users can access the services subscribed by the customer under this tenancy.

[0014] As indicated above, CSPs build or deploy data centers to provide cloud services to their customers. As the CSP's customer base grows, the CSP typically builds new data centers in new regions or increases the capacity of existing data centers to serve the growing demands of the customers and provide better service to the customers. Preferably, the data center is built in geographic proximity to the location of the customers served by the data center. The geographical proximity between the data center and the customers served by the data center leads to more efficient utilization of resources and faster and more reliable service to the customers. Thus, the CSP typically builds new data centers in new regions in geographic areas that are geographically close to the customers served by the data center. For example, for a growing customer base in Germany, the CSP may build one or more data centers in new regions in Germany.

[0015] Building a datacenter (or multiple datacenters) in a region is sometimes referred to as building a region. The term "region construction" is used to refer to building one or more datacenters in a region. Building a datacenter in a region requires provisioning or creating a new set of resources needed or used to provide the set of services that the datacenter is configured to provide. The end result of the region construction process is the creation of a datacenter in the region that is capable of providing the set of services intended for that datacenter and includes the set of resources used to provide the set of services.

[0016] Building a new datacenter in a region is a very complex activity that requires extensive coordination between various bootstrapping activities. At a high level, it requires the execution and coordination of various tasks, such as identifying the set of services to be provided by the datacenter, identifying the various resources required to provide the set of services, generating, provisioning, and deploying the identified resources, and properly wiring the resources so that they can be used in the intended manner. Each of these tasks has further subtasks that need to be coordinated, further increasing the complexity. Due to this complexity, currently, building a datacenter in a region requires several manually initiated or manually controlled tasks that require careful manual coordination. As a result, the task of building a new region (i.e., building one or more datacenters in a region) is very time-consuming. It can take time, e.g., months, to build a datacenter. In addition, the process is highly error-prone and sometimes requires several iterations before the desired configuration of the datacenter is achieved, which further increases the time it takes to build a datacenter. These limitations and problems severely limit the ability of CSPs to grow computing resources in a timely manner in response to growing customer needs.

[0017] Technical effects This disclosure describes techniques for shortening construction time, reducing computing resource waste, and reducing the risks associated with building one or more data centers in a region. Instead of the weeks and months required in the past to build a data center in a region, the techniques described herein can be used to build a new data center in a region in a relatively significantly shorter time with less risk of error than traditional approaches.

[0018] Disclosed herein is a cloud infrastructure orchestration service (CIOS) configured to bootstrap (e.g., provision and deploy) services to a new datacenter based on a predefined configuration file that identifies resources (e.g., infrastructure components and software to be deployed) for implementing a given change to the datacenter. The CIOS can parse and analyze the configuration file (e.g., flock configuration) to identify dependencies between resources, execution targets, phases, and flocks. The CIOS may generate specific data structures from the analysis and use these data structures to drive operations and manage the order in which services are bootstrapped into regions. The CIOS may utilize these data structures to identify when the CIOS can bootstrap services, when the bootstrap is blocked and / or when the bootstrap operations associated with previously blocked services can be resumed. Advantageously, the CIOS can identify circular dependencies in the data structures and perform operations to eliminate / resolve these circular dependencies prior to task execution. Using these techniques, the CIOS substantially reduces the risk of executing tasks prior to the availability of resources on which those tasks depend.

[0019] Using the techniques disclosed herein, CIOS can optimize parallel processing to perform changes to datacenters while ensuring that tasks are not initiated until the functionality they depend on is available in the regions, thereby allowing CIOS to perform region construction more efficiently, which significantly reduces the time required to construct a datacenter and the wasted computational resource usage seen in conventional approaches.

[0020] The present embodiment relates to processing capability data for a CIOS and generating a visualization that provides insight into the capabilities of the CIOS. For example, a multi-flock orchestrator (e.g., multi-flock orchestrator 106) can process the capability data to determine, for each capability, a number of capabilities that need to be issued for the capability to issue (or "dependent capabilities") and a number of capabilities on which each capability depends. The dependencies for each capability can be summed to derive a rank for each capability. The rank can specify a number of other capabilities that can be unblocked in response to the issuance of each capability. A first portion of the visualization can identify the capabilities along with all issued dependent capabilities arranged by rank for the identified capability.

[0021] The embodiment can further derive a rank for every capability along with one or more dependent capabilities that are not issued. A second portion of the visualization can provide the capabilities along with one or more dependent capabilities that are not issued arranged by rank. The second portion can also specify a number of steps to issue each capability (e.g., a number of dependent capabilities that need to be issued). Such a visualization can parse a number of capability CIOS and arrange the capabilities based on the dependencies associated with each capability. The visualization can be used to efficiently issue capabilities by allocating resources and building new regions in the CIOS.

[0022] Further, the present embodiment can derive a critical path that provides an efficient listing of resources for issuing to enable issuance of a selected capability or flock of capabilities. For example, a flock having multiple capabilities can be selected. Further, each capability can include multiple dependent capabilities, each having a corresponding issuance status (e.g., issued, ready to be issued, requires issuance of other capabilities). The multi-flock orchestrator described herein can derive any suitable number of paths for issuing capabilities across the CIOS to enable issuance (or "unblocking") of all capabilities for the selected flock. Further, the multi-flock orchestrator can be one path from the identified paths that has the greatest efficiency in unblocking capabilities for the selected flock. The critical path can be displayed in a user interface to enable efficient allocation of resources for performing construction tasks by the CIOS.

[0023] definition A "region" is a logical abstraction that corresponds to a geographic location. A region may include any suitable number of one or more execution targets. In some embodiments, an execution target may correspond to a data center.

[0024] An "Execution Target" refers to the smallest unit of change for executing a release. A "Release" refers to an expression of intent to orchestrate a particular change to a service (e.g., deploy version 8, "add internal DNS records", etc.). For most services, an execution target represents an "instance" of the service. A service can be bootstrapped into one or more execution targets. An execution target may be associated with a set of devices (e.g., a data center).

[0025] "Bootstrapping" is intended to mean the collective tasks associated with provisioning and deploying any suitable number of resources (e.g., infrastructure components, artifacts, etc.) corresponding to a service.

[0026] "Service" means functionality provided by a set of resources. The set of resources for a service includes any suitable combination of infrastructure, platform, or software (e.g., applications) hosted by a cloud provider that can be configured to provide the functionality of the service. A service can be made available to users over the Internet.

[0027] "Artifact" means an infrastructure component or code that is deployed to a Kubernetes engine cluster, which may include software (e.g., applications), configuration information for the infrastructure component (e.g., configuration files), etc.

[0028] "Flock configuration" means a configuration file (or set of configuration files) that describes the set of all resources (e.g., infrastructure components and artifacts) associated with a service. A flock configuration may contain declarative statements that specify one or more aspects that correspond to a desired state of the resources of the service.

[0029] "Service State" means a current snapshot of all resources (e.g., infrastructure resources, artifacts, etc.) associated with a service. The service state indicates the state corresponding to the provisioning and / or deployment tasks associated with the service resources.

[0030] IaaS provisioning (or "provisioning") means acquiring computers or virtual hosts for use, as well as installing the necessary libraries or services on them. The expression "provisioning a device" means developing a device into a state where it can be utilized by an end user for their specific use. A device that has undergone a provisioning process may be called a "provisioned device". Preparing a provisioned device (installing libraries and daemons) may be part of provisioning. This preparation is different from deploying a new application or a new version of an application on a prepared device. In most cases, deployment does not include provisioning, which may need to be performed first. Once prepared, the device may be called an "infrastructure component".

[0031] IaaS deployment (or "deployment") refers to the process of providing and / or installing a new application or a new version of an application on provisioned infrastructure components. Once the infrastructure components are provisioned (e.g., acquired, assigned, prepared, etc.), additional software may be deployed (e.g., provided and installed on the infrastructure components). After provisioning and deployment are complete, the infrastructure components may be referred to as "resources." Examples of resources may include, but are not limited to, virtual machines, databases, object storage, block storage, load balancers, etc.

[0032] A "capability" identifies a unit of functionality associated with a service. A unit can be part or all of the functionality provided by a service. For example, a capability can be published that indicates that a resource is available for authorization / authentication processing (e.g., a subset of the functionality provided by the resource). As another example, a capability can be published that indicates that the complete functionality of a service is available. Capabilities can be used to identify functionality on which a resource or service depends and / or features of the resource or service that are available for use.

[0033] "Virtual Bootstrap Environment" (ViBE) refers to a virtual cloud network that is provisioned in an overlay of an existing region (e.g., a "host region"). Once provisioned, the ViBE is connected to the new region using a communication channel (e.g., an IPsec tunnel VPN). Certain required core services (or "seed" services) can be provisioned in the ViBE, such as a deployment orchestrator, public key infrastructure (PKI) services, etc. These services can provide the capabilities required to bring hardware online, establish a chain of trust to the new region, and deploy the remaining services in the new region. Utilizing a virtual bootstrap environment can prevent circular dependencies between bootstrap resources by utilizing resources of the host region. Services can be staged and tested in the ViBE before the physical region (e.g., the target region) is available.

[0034] "Cloud Infrastructure Orchestration Services" (CIOS) may mean a system configured to manage provisioning and deployment operations for any suitable number of services as part of a region deployment.

[0035] A multi-flock orchestrator (MFO) can be a computing component (e.g., a service) that coordinates events among components of the CIOS to provision and deploy services to a target region (e.g., a new region). The MFO tracks relevant events for each service in the region configuration and takes action in response to these events.

[0036] "Host Region" means a region that hosts a Virtual Bootstrap Environment (ViBE). A host region may be used to bootstrap a ViBE.

[0037] "Target Region" means the region being constructed. "Publishing a capability" means "publishing" as used in a "publisher-subscriber" computing design or otherwise to provide an indication that a particular capability is available (or unavailable). Capabilities are "published" (e.g., collected by a capability service, provided to a capability service, pushed, pulled, etc.) to provide an indication that functionality of a resource / service is available. In some embodiments, capabilities may be published / sent via an event, notification, data transmission, function call, API call, etc. An event (or other notification / data transmission, etc.) indicating availability of a particular capability can be broadcast / addressed (e.g., published) to the capability service.

[0038] A "capability service" may be a flock configured to model dependencies between different flocks. Capability services may be provided within a cloud infrastructure orchestration service and may define which capabilities, services, and features are available in a region.

[0039] A "Real-time Regional Data Distributor" (RRDD) can be a service or system configured to manage regional data that can be injected into a flock configuration to dynamically generate execution targets for new regions.

[0040] In some examples, techniques for implementing a cloud infrastructure orchestration service (CIOS) are described herein. Such techniques can be configured to manage bootstrapping (e.g., provisioning and deploying software to) infrastructure components within a cloud environment (e.g., a region), as briefly described above. In some examples, the CIOS can include computing components (e.g., CIOS central and CIOS regional, both of which are described in more detail below) that can be configured to manage bootstrapping tasks (provisioning and deployment) for a given service, and a multi-flock orchestrator (also described in more detail below) configured to initiate / manage region construction (e.g., bootstrapping operations corresponding to multiple services).

[0041] CIOS enables region construction and worldwide infrastructure provisioning and code deployment with minimal manual runtime effort from service teams (e.g., beyond initial approval and / or physical transportation of hardware, in some examples). High-level responsibilities of CIOS include, but are not limited to, orchestrating region construction, providing users with a view of the current state of resources managed by CIOS (e.g., in a region, across regions, worldwide, etc.), and managing bootstrap operations to bootstrap resources within a region.

[0042] CIOS may provide view reconciliation, where a view of a desired state (e.g., desired configuration) of a resource may be reconciled with the current / actual state (e.g., current configuration) of the resource. In some examples, view reconciliation may include obtaining state data to identify which resources are actually operating and their current configuration and / or state. Reconciliation can be performed at various granularities, such as at the service level.

[0043] The CIOS can perform plan generation, where differences between desired and current states of resources are identified. Part of the plan generation can include identifying operations that need to be performed to bring resources from their current state to the desired state. In some examples, the CIOS can present the generated plant to a user for approval. In these examples, the CIOS can mark the plan as approved or rejected based on user input from the user. Thus, the user can spend less time judging the plan, and the plan is more accurate because it is machine generated. The plan is almost too detailed for human consumption. However, the CIOS can provide this data through a very advanced user interface (UI).

[0044] In some examples, the CIOS can handle change management execution by executing an approved plan. Once an execution plan is generated and approved, the engineer may no longer need to be involved in change management until the CIOS initiates the rollback. The CIOS can handle rollback to a previous service version by generating a plan to revert the service to a previous (e.g., pre-release) state (e.g., when the CIOS detects a service health degradation while executing).

[0045] CIOS can measure service health by monitoring alarms and running integration tests. CIOS can help teams quickly define rollback behavior in the event of service degradation and can execute the rollback behavior later. CIOS can generate and display plans and track approvals. CIOS can combine provisioning and deployment functions in one system that coordinates these tasks across region construction. CIOS also supports discovery of flocks (e.g., service resources such as flock configurations corresponding to any suitable number of services), artifacts, resources, and dependencies. CIOS can discover dependencies between execution tasks at all levels (e.g., resource level, execution target level, phase level, service level, etc.) by static analysis (e.g., including parsing and processing the contents) of one or more configuration files. Using these dependencies, CIOS can generate various data structures from these dependencies, which can be used to drive task execution (e.g., tasks related to provisioning infrastructure resources and deployment of artifacts across regions).

[0046] FIG. 1 is a block diagram of an environment 100 in which a cloud infrastructure orchestration service (CIOS) may operate to dynamically provide bootstrap services in a region, according to at least one embodiment. CIOS 102 may include, but is not limited to, the following components: real-time regional data distributor (RRDD) 104, multi-flock orchestrator (MFO) 106, CIOS central 108, CIOS regional 110, and capability service 112. Specific functionality of CIOS central 108 and CIOS regional 110 is provided in more detail in U.S. patent application Ser. No. 17 / 016,754, entitled "Techniques for Deploying Infrastructure Resources with a Declarative Provisioning Tool," the entire contents of which are incorporated in their entirety for all purposes. In some embodiments, any suitable combination of components of CIOS 102 may be provided as a service. In some embodiments, some portion of CIOS 102 may be deployed to a region (e.g., a data center represented by host region 103). In some embodiments, CIOS 102 may include any suitable number of cloud services (not shown in FIG. 1), as discussed in further detail in U.S. patent application Ser. No. 17 / 016,754 and below with respect to FIGS. 2 and 3.

[0047] The real-time regional data distributor (RRDD) 104 may be configured to maintain and provide region data identifying realms, regions, execution targets, and availability domains. In some cases, the region data may be in any suitable format (e.g., JSON format, data object / container, XML, etc.). The region data maintained by the RRDD 104 may include any suitable number of subsets of data that may be individually referenceable by a corresponding identifier. For example, an identifier such as "all_regions" may be associated with a data structure (e.g., a list, structure, object, etc.) that includes metadata for all defined regions. As another example, an identifier such as "realm" may be associated with a data structure that identifies metadata for a number of realms and a set of regions corresponding to each realm. In general, the region data may maintain any suitable attributes of one or more realms, regions, availability domains (ADs), execution targets (ETs), etc., such as identifiers, DNS extensions, states (e.g., state of the region), etc. The RRDD 104 may be configured to manage region state as part of the region data. The region state may include any suitable information indicative of a state of bootstrap within the region. For example, some example region states may include "initial," "construction," "production," "dormant," or "decommissioned." The "initial" state may indicate a region that has not yet been bootstrapped. The "construction" state may indicate that bootstrapping of one or more flocks in the region has begun. The "production" state may indicate that bootstrapping is complete and the region is being prepared for validation. The "dormant" state may indicate that CIOS Central 108 or CIOS Regional 110 has paused internal interaction with the regional stack, possibly due to operational issues. The "decommissioned" state may indicate that the region is being decommissioned and is likely to become unavailable and / or will not be contacted again.

[0048] CIOS central 108 may be configured to provide any suitable number of user interfaces by which a user (e.g., user 109) may interact with CIOS 102. For example, a user may make changes to region data through a user interface provided by CIOS central 108. CIOS central 108 may additionally provide various interfaces that allow a user to view changes made to flock configurations and / or artifacts, generate and view plans, approve / reject plans, view status in plan execution (e.g., corresponding to infrastructure provisioning, deployment, region construction, and / or desired states of any suitable number of resources managed by CIOS 102). CIOS central 108 may implement a control plane configured to manage any suitable number of CIOS regional 110 instances. CIOS central 108 may provide one or more user interfaces for representing region data, allowing user 109 to view and / or modify the region data. CIOS central 108 may be configured to invoke functions of RRDD 104 through any suitable number of interfaces. In general, CIOS central 108 may be configured to manage region data, either directly or indirectly (e.g., via RRDD 104). CIOS central 108 may be configured to compile a flock configuration to inject the region data as variables within the flock configuration.

[0049] Each instance of CIOS regional 110 may correspond to a module configured to perform bootstrapping tasks associated with one service of the region. CIOS regional 110 may receive desired state data from CIOS central 108. In some embodiments, the desired state data may include a flock configuration that declares (e.g., via a declarative statement) a desired state of resources associated with the service. CIOS central 108 may maintain current state data that indicates any suitable aspect of the current state of resources associated with the service. In some embodiments, CIOS regional 110 may identify that changes are required for one or more resources by comparison of the desired state data and the current state data. For example, CIOS regional 110 may determine that one or more infrastructure components need to be provisioned, one or more artifacts are deployed, or any suitable changes are required to the resources of the service to bring the state of those resources into agreement with the desired state. As CIOS regional 110 performs the bootstrapping operation, CIOS regional 110 may publish data that indicates various capabilities of the resources as they become available. A "capability" identifies a unit of functionality associated with a service. A unit can be part or all of the functionality provided by a service. For example, a capability can be published that indicates that a resource is available for authorization / authentication processing (e.g., a subset of the functionality provided by the resource). As another example, a capability can be published that indicates that the complete functionality of a service is available. Capabilities can be used to identify functionality on which a resource or service depends and / or features of the resource or service that are available for use.

[0050] The capability service 112 is configured to maintain capability data that indicates 1) which capabilities of various services are currently available, 2) whether any resources / services are waiting for a particular capability, 3) which particular resources and / or services are waiting for a given capability, or any suitable combination of the above. The capability service 112 may provide an interface by which the capability service may be requested. The capability service 112 may provide one or more interfaces (e.g., application programming interfaces) that allow the capability data to be sent to the MFO 106 and / or CIOS regional 110 (e.g., each instance of the CIOS regional 110). In some embodiments, any suitable component or module of the MFO 106 and / or CIOS regional 110 may be configured to request the capability data from the capability service 112.

[0051] In some embodiments, a multi-flock orchestrator (MFO) 106 may be configured to drive the region construction effort. In some embodiments, the MFO 106 may manage information describing which flocks / flock configuration versions and / or artifact versions are utilized to bootstrap a given service in a region (or to make a unit of change to a target region). In some embodiments, the MFO 106 may be configured to monitor (or otherwise be notified of) changes to the region data managed by the real-time regional data distributor 104. In some embodiments, region construction may be triggered by the MFO 106 upon receiving an indication that the region data has changed. In some embodiments, the MFO 106 may collect various flock configurations and artifacts used for region construction. Some or all of the flock configurations may be configured to be region agnostic; that is, the flock configuration may not explicitly identify which region the flock is bootstrapped to. In some embodiments, the MFO 106 may trigger a data injection process through which the collected flock configuration is recompiled (e.g., by the CIOS central 108). During the recompilation, an operation may be performed (e.g., by the CIOS central 108) to cause the region data maintained by the real-time regional data distributor 104 to be injected into the configuration file. The flock configuration can reference the region data through variables / parameters without requiring a hard-coded identification of the region data. The flock configuration can be dynamically modified at runtime with this data injection, rather than the region data being hard-coded, thus making it more difficult to change.

[0052] The multi-flock orchestrator 106 may perform static flock analysis, where the flock configuration is parsed to identify dependencies between resources, execution targets, phases, and flocks, and in particular to identify circular dependencies that need to be removed. In some embodiments, the MFO 106 may generate any suitable number of data structures based on the identified dependencies. These data structures (e.g., directed acyclic graphs, linked lists, etc.) may be utilized by the cloud infrastructure orchestration service 102 to drive operations to perform region construction. For example, these data structures may collectively define the order in which services are bootstrapped within a region. An example of such a data structure is discussed further below with respect to the construction dependency graph 338 of FIG. 3. When circular dependencies (e.g., service A requires service B and vice versa) exist and are identified through static flock analysis and / or graphs, the MFO may be configured to notify any appropriate service teams that changes are required to the corresponding flock configuration to correct these circular dependencies. The MFO 106 can be configured to traverse one or more data structures to manage the order in which services are bootstrapped into regions. The MFO 106 can identify (e.g., using data obtained from the capabilities service 112) the capabilities available within a given region at any given time. The MFO 106 can use this data to identify when a service can be bootstrapped, when bootstrap is blocked, and / or when a bootstrap operation associated with a previously blocked service can be resumed. Based on this traversal, the MFO 106 can execute various releases in which instructions are sent by the MFO 106 to the CIOS central 108 to execute a bootstrap operation corresponding to any suitable number of flocking configurations.In some examples, the MFO 106 may be configured to identify that one or more flock configurations may require multiple releases due to circular dependencies found in the graph, and as a result, the MFO 106 may send multiple sets of instructions to the CIOS central 108 for a given flock configuration to break the circular dependencies identified in the graph.

[0053] In some embodiments, a user may request that a new region (e.g., target region 114) be built. This may require bootstrapping resources corresponding to various services. In some embodiments, target region 114 may not be communicatively available (and / or secure) at the time the region build request is initiated. Rather than deferring bootstrapping until such time that target region 114 is available and configured to perform the bootstrap operation, CIOS 102 may initiate region build using a virtual bootstrap environment 116. The virtual bootstrap environment (ViBE) 116 may be an overlay network hosted by host region 103 (an existing region that has been previously configured with a core set of services and is communicatively available and secure). MFO 106 may leverage the resources of host region 103 to bootstrap resources to ViBE 116 (commonly referred to as "building a ViBE"). For example, MFO 106 can provide instructions through CIOS Central 108 to cause an instance of CIOS regional 110 in a host region (e.g., host region 103) to bootstrap another instance of CIOS regional in ViBE 116. Once the CIOS regional in ViBE becomes available for processing, bootstrapping of services for the target region 114 can continue in ViBE 116. When the target region 114 is available to perform the bootstrap operation, previously bootstrapped services in ViBE 116 may be moved to the target region 114. Utilizing these techniques, CIOS 102 can significantly increase the speed at which regions are built by dramatically reducing the need for any manual input and / or configuration to be provided.

[0054] 2 is a block diagram illustrating an environment 200 and method for building a virtual bootstrap environment (ViBE) 202 (an example of ViBE 116 of FIG. 1) according to at least one embodiment. ViBE 202 represents a virtual cloud network that is provisioned in an overlay of an existing region (e.g., hosted region 204, an example of hosted region 103 of FIG. 1 and in the embodiments is a hosted region service enclave). ViBE 202 represents an environment in which services can be staged for a target region (e.g., a region under construction such as target region 114 of FIG. 1) before the target region is available.

[0055] To bootstrap a new region (e.g., target region 114 of FIG. 1), a core set of services may be bootstrapped. These core set of services may exist in the host region 204, but do not yet exist in ViBE (nor in the target region). These required core services provide the functionality needed to provision devices, establish a chain of trust to the new region, and deploy the remaining services (e.g., negative locks) within the region. ViBE 202 may be a tenancy that is deployed in the host region 204. This can be thought of as a virtual region.

[0056] Once the target region is available to provide bootstrap operations, ViBE 202 can be connected to the target region, allowing services in ViBE to interact with services and / or infrastructure components of the target region. This allows for the deployment of production level services instead of a self-contained seed service as in previous systems, requiring a connection to the target region over the Internet. Traditionally, seed services are deployed as part of a container collection and are used to bootstrap the dependencies needed to build the region. Using the existing region's infrastructure / tooling, resources may be bootstrapped (e.g., provisioned and deployed) into ViBE 202 and connected to the service enclaves of the region (e.g., host region 204) to provision hardware and deploy services until the target region is self-sufficient and can be directly communicated with. Utilizing ViBE 202 allows for the dependencies and services needed to be able to provision / prepare infrastructure and deploy software while utilizing the host region's resources to break circular dependencies of core services.

[0057] A multi-flock orchestrator (MFO) 206 may be configured to perform operations to build (e.g., configure) ViBE 202. MFO 206 may obtain applicable flock configurations corresponding to various resources to be bootstrapped into a new region (in this case, a ViBE region, ViBE 202). For example, MFO 206 may obtain a flock configuration (e.g., a "ViBE flock configuration") that identifies aspects of bootstrapping capability services 208 and workers 210. As another example, MFO 206 may obtain another flock configuration corresponding to bootstrapping domain name services (DNS) 212 into ViBE 202.

[0058] In step 1, MFO 206 may instruct CIOS central 214 (e.g., an example of CIOS central 108 and CIOS central 214 of FIGS. 1 and 2, respectively). For example, MFO 206 may send a request (e.g., including a ViBE flock configuration) to request bootstrap of capability services 208 and workers 210 that do not yet exist in ViBE 202 at this time. In some embodiments, CIOS central 214 may have access to all flock configurations. Thus, in some examples, MFO 206 may send an identifier for a ViBE flock configuration rather than the file itself, which CIOS central 214 may independently obtain from storage (e.g., from DB 308 or flock DB 312 of FIG. 3).

[0059] In step 2, CIOS central 214 may provide the ViBE flock configuration via a corresponding request to CIOS regional 216. CIOS regional 216 may parse the ViBE flock configuration to identify and perform specific infrastructure provisioning and deployment operations in step 3.

[0060] In some embodiments, CIOS regional 216 may utilize additional corresponding services for provisioning and deployment. For example, in step 4, CIOS regional 216 CIOS regional may instruct deployment orchestrator 218 (e.g., an example of core services or other write, build and deploy application software of host region 204) to execute instructions that, in turn, bootstrap capability service 208 and worker 210 within ViBE 202.

[0061] In step 5, a capability may be sent to capability service 208 (from CIOS regional 216, deployment orchestrator 218 via worker 210, or otherwise) indicating that resources corresponding to the ViBE block are available. Capability service 208 may persist this data. In some embodiments, capability service 208 adds this information to a list it maintains of capabilities available with ViBE. For example, the capability provided to capability service 208 in step 5 may indicate that capability service 208 and worker 210 are available for processing.

[0062] In step 6, the MFO 206 may identify that the capability indicates that the capability service 208 and worker 210 are available based on receiving or obtaining data (an identifier corresponding to the capability) from the capability service 208.

[0063] In step 7, as a result of receiving / obtaining the data in step 6, MFO 206 may instruct CIOS Central 214 to bootstrap a DNS service (e.g., DNS 212) on ViBE 202. The instructions may identify or include a particular flock setting that corresponds to the DNS service.

[0064] In step 8, CIOS central 214 may instruct CIOS regional 216 to deploy DNS 212 to ViBE 202. In some embodiments, the DNS flocking configuration for DNS 212 is provided by CIOS central 214.

[0065] In step 9, worker 210, now deployed in ViBE 202, may be assigned by CIOS regional 216 to the task of deploying DNS 212. The worker may run a declarative infrastructure provisioner in the form described above in connection with FIG. 3 to identify the set of operations that need to be performed to deploy DNS 212 (e.g., by comparing the flock configuration (desired state) with the current state of the (now non-existent) resources associated with the flock).

[0066] At step 10, deployment orchestrator 218 may instruct worker 210 to deploy DNS 212 according to the operations identified at step 9. As shown, worker 210 proceeds to execute operations to deploy DNS 212 to ViBE 202 at step 11. At step 12, worker 210 notifies capability service 208 that DNS 212 is available on ViBE 202. MFO 206 may subsequently identify that the ViBE flock configuration and resources associated with the DNS flock configuration are available and may proceed to bootstrap any appropriate number of additional resources to ViBE.

[0067] After steps 1-12 are completed, the process for building ViBE202 can be considered complete, and ViBE202 can be considered built.

[0068] FIG. 3 is a block diagram illustrating an environment 300 and method for bootstrapping a service to a target region utilizing ViBE according to at least one embodiment.

[0069] In step 1, user 302 may utilize any suitable user interface provided by CIOS central 304 (an example of CIOS central 108 and CIOS central 214 of FIGS. 1 and 2, respectively) to modify region data. For example, user 302 may create a new region into which a number of services are bootstrapped.

[0070] In step 2, CIOS central 304 may perform an operation to send the changes to RRDD 306 (e.g., an example of RRDD 104 in FIG. 1). In step 3, RRDD 306 may store the received region data in database 308, a data store configured to store region data including any suitable identifiers, attributes, states, etc., such as region, AD, realm, ET, etc. In some embodiments, updater 307 may be utilized to store the region data in database 308 or any suitable data store, from which such updates may be accessible (e.g., to a service team). In some embodiments, updater 307 may be configured to notify (e.g., via any suitable electronic notification) of updates made to database 308.

[0071] In step 4, the MFO 310 (an example of MFOs 106 and 206 in FIGS. 1 and 2, respectively) may detect changes in the region data. In some embodiments, the MFO 310 may be configured to poll the RRDD 306 for changes in the region data. In some embodiments, the RRDD 306 may be configured to publish or otherwise notify the MFO 310 of the region changes.

[0072] In step 5, detecting a change in the region data may trigger MFO 310 to retrieve a version set (e.g., a version set associated with a particular identifier, such as a "golden version set" identifier) ​​to identify a specific version for each flock (e.g., service) to be bootstrapped into the new region and a specific version for each artifact corresponding to that flock. The version set may be retrieved from DB 312. As flocks develop and change, the versions for their corresponding settings and artifacts used for region construction may change. These changes may be persisted in flock DB 312 so that MFO 310 may identify which versions of flock settings and artifacts to use to use to build a region (e.g., ViBE region, target region / non-ViBE region, etc.). Flock settings (e.g., all versions of flock settings) and / or artifacts (e.g., all versions of artifacts) may be stored in DB 308, DB 312, or any suitable data store accessible to CIOS Central 304 and / or MFO 310.

[0073] In step 6, the MFO 310 may request the CIOS Central 304 to recompile each of the flock settings associated with the version set with the current region data. In some embodiments, the request may indicate versions for each flock setting and / or artifacts that correspond to those flock settings.

[0074] In step 7, the CIOS Central 304 may obtain the current regional data from the DB 308 (e.g., directly or via the real-time regional data distributor 306) and retrieve any appropriate flock settings and artifacts according to the version requested by the MFO 310.

[0075] In step 8, CIOS Central 304 may recompile the flock configuration with the region data obtained in step 7 to inject the flock configuration with the current region data. CIOS Central 304 may return the compiled flock configuration to MFO 310. In some embodiments, CIOS Central 304 may simply indicate that compilation has occurred, or MFO 310 may access the recompiled flock configuration via RRDD 306.

[0076] In step 9, the MFO 310 may perform a static analysis of the recompiled flock configuration. As part of the static analysis, the MFO 310 may parse the flock configuration (e.g., using libraries associated with a declarative infrastructure provisioner (e.g., Terraform, etc.)) to identify dependencies between flocks. From the identified analysis and dependencies, the MFO 310 may generate a build dependency graph 338. The build dependency graph 338 may be an acyclic, directed graph that identifies the order in which flocks are bootstrapped into new regions (and / or the order in which changes indicated in the flock configuration are applied). Each node in the graph may correspond to bootstrapping any appropriate portion of a particular flock. The particular bootstrap order may be identified based at least in part on the dependencies. In some embodiments, the dependencies may be represented as attributes of the nodes and / or may be indicated via edges of the graph connecting the nodes. The MFO 310 may traverse the graph (e.g., beginning at a start node) to drive the region build operation.

[0077] In some embodiments, MFO 310 may utilize a cycle detection algorithm to detect the existence of a cycle (e.g., service A depends on service B and vice versa). MFO 310 may identify orphaned capability dependencies. For example, MFO 310 may identify orphaned nodes in construction dependency graph 338 that are not connected to any other nodes. MFO 310 may identify erroneously issued capabilities (e.g., when a capability is issued too early and the corresponding functionality is not actually available yet). MFO 310 may detect from the graph that there are one or more instances that issue the same capability. In some embodiments, any suitable number of these errors may be detected, and MFO 310 (or another suitable component, such as CIOS Central 304) may be configured to notify or otherwise present this information to a user (e.g., via an electronic notification, a user interface, etc.). In some embodiments, MFO 310 may be configured to force the removal / recreation of resources to break circular dependencies, and may again provide instructions to CIOS Central 304 to perform bootstrap operations for those resources and / or corresponding block configurations.

[0078] The initiating node may correspond to bootstrapping the ViBE flock, and the second node may correspond to bootstrapping the DNS. Steps 10-15 correspond to deploying (via deployment orchestrator 317, an example of deployment orchestrator 218 in FIG. 2) the ViBE flock to ViBE 316 (e.g., an example of ViBE 116 and 202 in FIGS. 1 and 2, respectively). That is, steps 10-15 in FIG. 3 generally correspond to steps 1-6 in FIG. 2. Upon notification that capabilities corresponding to the ViBE flock to be deployed exist (e.g., indicating that capability service 318 and worker 320, corresponding to capability service 208 and worker 210 in FIG. 2, are available), MFO 310 resumes traversing construction dependency graph 338 to identify the next operation to be executed.

[0079] For example, MFO 310 may continue to traverse construction dependency graph 338 to identify a DNS block to be deployed. Steps 16-21 may be performed to deploy DNS 322 (an example of DNS 212 in FIG. 2). These operations may generally correspond to steps 7-12 in FIG. 2.

[0080] At step 21, a capability indicating that DNS 322 is available may be stored. Upon detecting this capability, MFO 310 may resume traversal of construction dependency graph 338. In this traversal, MFO 310 may inform ViBE 316 that any appropriate portions of an instance of CIOS regional (e.g., an instance of CIOS regional 314) are deployed to ViBE 316. In some embodiments, steps 16-21 may be substantially repeated with respect to deploying CIOS regional (ViBE) 326 (an instance of CIOS regional 314, CIOS regional 110 in FIG. 1) and worker 328 to ViBE 316. The capability may be sent to capability service 318 that CIOS regional (ViBE) 326 is available.

[0081] Upon detecting that CIOS Regional (ViBE) 326 is available, MFO 310 may resume traversal of construction dependency graph 338. In this traversal, MFO 310 may identify that a deployment orchestrator (e.g., deployment orchestrator 330, an example of deployment orchestrator 317) is deployed to ViBE 316. In some embodiments, steps 16-21 may be substantially repeated with respect to deploying deployment orchestrator 330. Information identifying capabilities may be sent to capability service 318 indicating that deployment orchestrator 330 is available.

[0082] Once the deployment orchestrator 330 is deployed, ViBE 316 may be considered available to process subsequent requests. Upon detecting that the deployment orchestrator 330 is available, MFO 310 may instruct subsequent bootstrap requests to be routed to the ViBE component rather than utilizing the host region component (the component of host region 322). Thus, MFO 310 may continue to traverse the construction dependency graph 338 at each node instructing flock deployment to ViBE 316 via CIOS Central 304. CIOS Central 304 may request CIOS Regional (ViBE) 326 to deploy resources according to the flock configuration.

[0083] At some point during this process, the target region 334 may become available. An indication that the target region is available may be discernible from region data for the target region 334 provided by the user 302 (e.g., as an update to the region data). The availability of the target region 334 may depend on establishing a network connection between the target region 334 and an external network (e.g., the Internet). The network connection may be supported over a public network (e.g., the Internet) but may use software security tools (e.g., IPsec) to provide one or more encrypted tunnels (e.g., IPsec tunnels such as tunnel 336) from ViBE 316 to the target region 334. As used herein, "IPsec" refers to a suite of protocols for authenticating and encrypting network traffic over networks that use the Internet Protocol (IP) and may include one or more available implementations of the suite of protocols (e.g., Openswan, Libreswan, strongSwan, etc.). The network may connect ViBE 316 to a service enclave of the target region 334.

[0084] Prior to establishing the IPsec tunnel, the initial network connection to the target region 334 may be on a sufficient connection (e.g., an out-of-band VPN tunnel) to allow bootstrapping of networking services until IPsec can be deployed on assets (e.g., bare metal assets) in the target region 334. To bootstrap the network resources of the target region 334, the deployment orchestrator 330 may deploy an IPsec gateway on the assets in the target region 334. The deployment orchestrator 330 may then deploy a VPN host in the target region 334 configured to terminate the IPsec tunnel from ViBE 316. A service in ViBE 316 (e.g., deployment orchestrator 330, Service A, etc.) can establish an IPsec connection with the VPN host in the target region 334, and the bootstrap operation from ViBE 316 to the target region 334 may begin.

[0085] In some embodiments, the bootstrap operation may begin by services in ViBE 316 provisioning resources in the target region 334 to support hosting instances of core services when deployed from ViBE 316. For example, a host provisioning service may provision a hypervisor on infrastructure (e.g., bare metal hosts) in the target region 334 to allocate computing resources for VMs. Once the host provisioning service completes the allocation of physical resources in the target region 334, the host provisioning service may publish information indicating a capability indicating that the physical resources in the target region 334 have been allocated. The capability may be published to the capability service 318 (e.g., by worker 328) via CIOS regional (ViBE) 326.

[0086] Once the hardware allocation for the target region 334 has been established and posted to the capability service 318, CIOS regional (ViBE) 326 can orchestrate the deployment of instances of core services from ViBE 316 to the target region 334. This deployment may be similar to the process described above for building ViBE 316, but using ViBE components (e.g., CIOS regional (ViBE) 326, worker 328, deployment orchestrator 330) instead of host region 332 service enclave components. The deployment operation may generally correspond to steps 16-21 described above.

[0087] As a service is deployed from ViBE 316 to target region 334, a DNS record associated with the service may correspond to an instance of the service in ViBE 316. The DNS record associated with the service may be updated later to complete the deployment of the service to target region 334. In other words, the instance of the service in ViBE 316 may continue to receive traffic (e.g., requests) to the service until the DNS record is updated. The service may be partially deployed into target region 334 and may publish information (e.g., to capability service 318) indicating the capabilities that the service is partially deployed in. For example, a service running in ViBE 316 may be deployed into target region 334 with corresponding compute instances, load balancers, and associated applications and other software, but may need to wait for database data to move to target region 334 before being fully deployed. The DNS record (e.g., managed by DNS 322) may still be associated with the service in ViBE 316. Once the data movement for the service is complete, the DNS record may be updated to point to the operational service deployed in target region 334. The deployed service in target region 334 may then receive traffic (eg, requests) for the service, while the instance of the service in ViBE 316 may no longer receive traffic for the service.

[0088] A. System Overview A cloud infrastructure orchestration service (CIOS) can implement multiple functions / applications using computing resources located across one or more regions (e.g., data centers). Additionally, as more computing resources are added to the CIOS, additional functions / applications can be added to the CIOS. The functions / applications implemented in the CIOS as described herein can be referred to as "capabilities." To implement (or "issue") a capability in the CIOS, resources (e.g., computer resources, software implementation resources) can be allocated to implement the capability.

[0089] Furthermore, a capability in CIOS can depend on one or more other capabilities. In addition, a first capability can be a dependency for a second capability, such that implementation of the second capability is only possible after issuing the first capability. For example, a load balancer capability can depend on an object store capability and a cloud identity service capability. Thus, in this example, to implement the load balancer capability, both the object store capability and the cloud identity service capability need to be issued (indicating that the object store capability and the identity service capability are available).

[0090] In many CIOS infrastructure instances, a large number of capabilities can be issued. Furthermore, each capability can include different dependencies, and each capability can unblock the issuance of any number of other capabilities. Tracking dependencies for a large number of capabilities can be very time and resource consuming. Furthermore, issuing capabilities requires a variety of resources (e.g., computing resources, operator interaction, software development). Thus, there is a need to efficiently allocate resources for issuing capabilities, for example, to efficiently implement a service or build a new region.

[0091] The present embodiment relates to processing capability data for CIOS and generating visualizations that provide insight into CIOS capabilities. For example, a multi-flock orchestrator (e.g., multi-flock orchestrator 106) can process the capability data to determine, for each capability, a number of capabilities that need to be issued for the capability to issue (or "dependent capabilities") and a number of capabilities on which each capability depends. The dependencies for each capability can be summed to derive a rank for each capability. The rank can specify a number of other capabilities that can be unblocked in response to the issuance of each capability. A first portion of the visualization can identify the capabilities along with all issued dependent capabilities arranged by rank for the identified capability.

[0092] The embodiment may further derive a rank for every capability along with one or more dependent capabilities that are not issued. A second portion of the visualization may provide the capabilities along with one or more dependent capabilities that are not issued arranged by rank. The second portion may also specify a number of steps to issue each capability (e.g., a number of dependent capabilities that need to be issued). Such a visualization may parse a number of capability CIOS and arrange the capabilities based on the dependencies associated with each capability. The visualization may be used to efficiently issue capabilities by allocating resources and building new regions in the CIOS.

[0093] Further, the present embodiment can derive a critical path that provides an efficient listing of resources for issuing to enable issuance of a selected capability or flock of capabilities. For example, a flock having multiple capabilities can be selected. Further, each capability can include multiple dependent capabilities, each having a corresponding issuing status (e.g., issued, ready to be issued, requires issuance of other capabilities). The multi-flock orchestrator described herein can derive multiple paths for issuing capabilities across the CIOS to enable issuance (or "unblocking") of all capabilities for the selected flock. Further, the multi-flock orchestrator can identify a path from the multiple paths that has the greatest efficiency in unblocking capabilities for the selected flock. The critical path can be provided to the client for use in efficiently allocating resources in issuing capabilities in the CIOS.

[0094] A CIOS can include multiple regions. Each region can provide a grouping of computing resources, for example, within a geographical proximity or within a data center. The computing resource portion of the CIOS can implement multiple capabilities. Resources can be assigned to flocks that provide the grouping of resources in the CIOS.

[0095] 4 provides a block diagram illustrating another example CIOS (e.g., CIOS 400, an example of CIOS 100 of FIG. 1). As shown in FIG. 4, CIOS 400 can include multiple regions 402A, 402B. Each region 402A, 402B can include multiple computing instances 404A-N, 406A-N. Additionally, a new region 402C can be added to CIOS 400 to add additional computing resources (e.g., computing instances 408A-N) to CIOS 400.

[0096] The multi-flock orchestrator 412 can process configuration data (e.g., one or more flock configuration files) to identify dependencies between capabilities. For example, the multi-flock orchestrator 412 can perform any suitable number of parsings of the flock configuration files (e.g., configuration data 413) described above to identify dependencies between capabilities / bootstrap tasks. From these identified capabilities, the multi-flock orchestrator 412 can generate a ranking to prioritize the issuance of capabilities (and / or the execution of corresponding bootstrap tasks) in the CIOS 400. In some embodiments, the multi-flock orchestrator 412 can generate a construction dependency graph (e.g., construction dependency graph 338) from any suitable number of flock configuration files to drive an order of bootstrap task execution within a region, task execution across regions, etc. The multi-flock orchestrator 412 can traverse the construction dependency graph to identify the bootstrap tasks to be executed and the order in which those tasks are executed. Upon reaching a node in the graph (e.g., a node corresponding to a flock / set of resources to be bootstrapped), the multi-flock orchestrator 412 may identify the bootstrap task to be executed and the capabilities on which execution of the node's corresponding bootstrap task depends. If the current node's task depends on one or more other capabilities that have been issued, the multi-flock orchestrator 412 may perform operations to identify whether these other capabilities have been issued. The construction dependency graph may begin with one or more nodes that have no dependencies on other capabilities. As changes are made within the regions corresponding to these nodes, the multi-flock orchestrator may proceed in traversing the construction dependency graph until it reaches a node that depends on a capability that has not yet been issued.

[0097] As resources are bootstrapped into the CIOS (e.g., infrastructure components are provisioned, software artifacts are deployed, etc.), new capabilities can be added to the CIOS. As resources are added and / or units of change are performed, various capabilities can be published (e.g., by the capability service 410 based on input data received from the CIOS regional 314 described above) to indicate the currently available functionality within the region. As a non-limiting example, a flock configuration file that specifies a unit of change (e.g., a set of resources to be bootstrapped into a region) may be associated with one or more capabilities. During the execution of these changes (or upon completion of these changes), one or more capabilities may be published. A given capability (and corresponding flock configuration file) can be associated with any suitable number of capabilities upon which its publication depends. Similarly, the publication of any suitable number of other capabilities may depend on the publication of a given capability. For a capability to be published, all capabilities it depends on may need to be published. Thus, in some embodiments, a unit of change corresponding to a flock configuration file may not be executed until all capabilities it depends on are available (as indicated by those capabilities being published).

[0098] Input data from regions 402A-C can be provided to capability service 410 (e.g., an example of capability service 112 of FIG. 1, 208 of FIG. 2, and / or 318 of FIG. 3, respectively). The input data may indicate capabilities currently available in the region as determined by another CIOS component (e.g., CIOS regional 314 of FIG. 3). Capability service 410 can obtain the input data across regions 402A-C and process the input data to derive capability data. This capability data may indicate one or more capabilities currently available with regions 402A-C. In some embodiments, capability service 410 may receive input data from any suitable component (e.g., CIOS regional 110 of FIG. 1) when a bootstrapping task is completed for each flock (e.g., a set of resources to be bootstrapped according to a given flock configuration file) or at any suitable time. The capability service 410 may provide, publish, and / or broadcast corresponding capability data to any suitable component of the CIOS via broadcasting the capability data according to a predetermined schedule and / or in response to a request received from another component of the CIOS.

[0099] For example, capability data can be provided to the multi-flock orchestrator 412 from the capability service 410. In some embodiments, the multi-flock orchestrator 412 can request this information from the capability service 410 at any appropriate time. Using the capability data provided by the capability service 410, the multi-flock orchestrator 412 can identify when capabilities have been issued that correspond to nodes whose bootstrap tasks were previously restricted by waiting for the issuance of these capabilities. Upon identifying that these capabilities are now available, the multi-flock orchestrator 412 can authorize and / or initiate execution of the bootstrap task for that node (e.g., by instructing the CIOS central 304 accordingly). The multi-flock orchestrator 412 can then proceed traversing the construction dependency graph until it hits another node that depends on one or more capabilities that have not yet been issued. The capability service 410 may manage a list of published capabilities, each of which indicates resources available within a respective unit of functionality and / or region. The multi-flock orchestrator 412 may utilize the capability data received / obtained from the capability service 410 to drive the traversal of the construction dependency graph and, consequently, the execution of the bootstrap tasks within the regions.

[0100] As described above, capabilities may include multiple dependencies across various regions or phases in a CIOS. FIG. 5 is a block diagram 500 illustrating capability dependencies in a CIOS. As shown in FIG. 5, multiple regions (e.g., region 1 and region 2) may include various phases. For example, a first region may include an application phase 502 and an infrastructure phase 504. Each phase may be associated with provisioning infrastructure and / or deploying software. A flock configuration file may include information related to one or more phases. In this example, two flock configuration files (not shown) may be associated with an infrastructure phase 508 and an application phase 506, respectively. The infrastructure phase 508 may be associated with provisioning infrastructure resources, and the application phase 506 may be associated with deploying software to these resources. Further, in this example, a second region may be associated with an application phase 506 and an infrastructure phase 508.

[0101] Each phase defined in a flock configuration file may depend on a number of dependent capabilities 510A and a number of capabilities (e.g., issued capabilities 510B) that are issued during execution of the bootstrap task corresponding to the phase. For example, provisioning of a third capability 518B in the first region infrastructure 504 causes the issuance of capability 3 518B (as identified in the corresponding flock configuration file). However, the flock configuration file corresponding to the region 1 infrastructure 504, capability 3 518B, or either one may depend on another capability available in region 1. For example, capability 3 518B, the region 1 infrastructure 504, and the flock configuration files associated with both may depend on capabilities such as CI identity compartment capability 520A, CI identity policy capability 520B, and tenancy creator 520C, as shown in FIG. 5. Thus, in this example, in order to issue capability 3 518B (and / or to begin execution of the bootstrap tasks associated with region 1 infrastructure 504), dependent capabilities 520A-C may first need to be issued.

[0102] Further, in many instances, in order to publish a capability, dependent capabilities may need to be published across multiple phases in multiple regions. For example, dependent capabilities 514A-C may first need to be published by application 506 in region 2 before publishing FOO service 512 (and / or before executing the bootstrap tasks associated with a flock configuration file that specifies the publication of FOO service 512). Furthermore, capabilities 516A-C may first need to be published before dependent capability 1 514C can be published by infrastructure 508 in region 2. Capability 516C (associated with application 502 in region 1 during which resources specified in the corresponding flock configuration file are bootstrapped in region 1) may further be published in application 502 in region 1 and may depend on capabilities 518A, 518B. Capability 3 518B may be published in infrastructure 504 in region 1 and may depend on capabilities 520A-C. Thus, to publish a service (eg, FOO service 512), multiple dependent capabilities may need to be published across multiple phases in multiple regions of the CIOS.

[0103] If the dependent capability 514C is not published by the region 2 infrastructure 508, the FOO service 512 may not be able to be published (and / or the bootstrap tasks identified based on the flock configuration file describing the publication of the FOO service 512 may be restricted from being executed). Furthermore, if capability 2 516C and capability 1 514C are not published, the publication of the FOO service 512 may require two publication steps (e.g., a first step to publish capability 2 516C and a second step to publish capability 1 514C). However, in many instances, resources (e.g., computing resources, virtual machines, etc.) may need to be allocated to publish capabilities. In particular, resources need to be allocated to efficiently publish capabilities in the CIOS because the services bootstrapped by the CIOS may include a large number of capabilities.

[0104] 6 is a block diagram of an example capability visualization generation system 600. As shown in FIG. 6, the capability visualization generation system 600 can include a capability service 610 (e.g., capability service 410 of FIG. 4) and a multi-flock orchestrator 612 (e.g., multi-flock orchestrator 412 of FIG. 4). The capability service 610 can obtain input data from computing instances in the CIOS and process the input data. For example, the capability service 610 can process the input data to identify capabilities, flocks, phases, status of each capability, etc., which can be included as part of the capability data provided to the multi-flock orchestrator 612.

[0105] The multi-floc orchestrator 612 can obtain capability data from the capability service 610 and process the capability data to generate one or more visualizations (e.g., visualization 614) of various aspects of the capability data as described herein. The multi-floc orchestrator 612 can include any of the capability dependency identification and tracking subsystem 602, the capability ranking subsystem 604, the capability path priority subsystem 606, and the visualization subsystem 608. Although FIG. 6 illustrates the multi-floc orchestrator 612 as providing the visualization 614 using the visualization subsystem 608, in some embodiments, the visualization 614 and the visualization subsystem 608 operate as part of or are provided by the CIOS central 108 of FIG. 1.

[0106] As described above, the multi-flock orchestrator 612 (e.g., capability dependency identification and tracking subsystem 602) can process configuration data (e.g., one or more flock configuration files) and identify dependencies between capabilities and dependencies between capability / bootstrap tasks based at least in part on any suitable number of parsings of the flock configuration files described above (of which the configuration data of FIG. 6 is an example). In some embodiments, a given flock configuration file can indicate any suitable number of capabilities on which that flock (the capabilities of the flock) depend. In some embodiments, the flock configuration file may indicate one or more capabilities that are to be issued when bootstrapping the resources of the flock is completed. Dependency information for a capability can be derived by identifying metadata for the capability (e.g., capability type, region or phase for the capability) and identifying (e.g., from the flock configuration file) capabilities that need to be issued before issuing the specified capability (or capabilities) of the flock can begin. In some examples, the capability dependency identification and tracking subsystem 602 can process aspects of each capability and track requests / calls to each dependent capability to identify the dependent capabilities.

[0107] The capability dependency identification and tracking subsystem 602 may generate a build dependency graph (e.g., build dependency graph 338) from any suitable number of flock configuration files to drive, among other things, the ordering of bootstrap task execution within regions, across regions, etc. The capability dependency identification and tracking subsystem 602 may traverse the build dependency graph to identify the bootstrap tasks to be executed and the order in which those tasks are executed. Upon reaching a node in the graph (e.g., a node corresponding to a flock / set of resources to be bootstrapped), the capability dependency identification and tracking subsystem 602 may identify the bootstrap tasks to be executed and the capabilities on which the execution of the node's corresponding bootstrap tasks depends. If the current node's tasks depend on one or more other capabilities being issued, the capability dependency tracking subsystem 602 may perform operations to identify whether these other capabilities have been issued. For example, the capability dependency identification and tracking subsystem 602 may identify whether these other capabilities have been issued from any previously received capability data (e.g., capability data received from the capability service 610). The capability dependency identification and tracking subsystem 602 may use the capabilities identified in that data to track capability availability within the region and determine whether to maintain its position or progress with the traversal of the construction dependency graph.

[0108] Additionally, the capability dependency tracking subsystem 602 can aggregate dependency information for each identified capability. The aggregated dependency information can be processed to derive insights into the capability data, such as identifying unissued capabilities that, when issued, would allow the ability to issue the greatest number of other capabilities. The capability dependency tracking subsystem 602 can generate capability dependency data for each capability that specifies the dependencies corresponding to those capabilities and the status of each capability in the CIOS (e.g., issued / available, not yet issued / available, etc.). The capability dependency data can be provided to the capability ranking subsystem 604.

[0109] The capability ranking subsystem 604 can process the capability-dependent data and assign ranks to the capabilities for use in generating one or more visualizations of the capability data. The rank for each capability can indicate the impact of issuing each capability to enable the issuance of other capabilities.

[0110] For example, three dependent capabilities may depend on the issuance of a first unissued capability (e.g., the three capabilities require the first unissued capability to be issued before them), and two other capabilities may depend on the second unissued capability. In this example, the first unissued capability may be assigned a score and / or a first rank, and the second unissued capability may be assigned a second score and / or a second rank that is lower than the first score and / or the first rank. These ranks and / or scores may identify the first unissued capability as being of higher priority than the second unissued capability based at least in part on the first unissued capability unblocking a larger number (e.g., three) of dependent capabilities. In other words, the first unissued capability may be determined to be of higher priority. Because the number of capabilities that depend on the first unissued capability is greater than the number of capabilities that depend on the second unissued capability, the ranking (or scoring) provided by capability ranking subsystem 604 can indicate a priority when allocating resources to issue capabilities in the CIOS.

[0111] In some examples, the capability ranking subsystem 604 can rank capabilities based on the status of all dependencies. For example, a first capability can be unissued but dependent on multiple capabilities that have already been issued. Thus, the first capability can be allocated resources and issued. Furthermore, a second capability can be unissued and dependent on one or more dependent capabilities that have not been issued. Thus, the capabilities on which the second capability depends may need to be issued first before the second capability is issued. These capabilities can be ranked (or scored) based on such differences. For example, a first portion of capabilities including capabilities for which all dependencies are satisfied (e.g., capabilities that depend only on capabilities that have already been issued) can be ranked with a first assigned rank (e.g., a particular rank, a particular score, etc.). Additionally, a second portion of capabilities that depend on one or more currently unissued capabilities can be ranked with a second assigned rank (e.g., a different rank, a different score, etc.) that is different from the first assigned rank. The second portion of capabilities can also be assigned a number corresponding to the number of capabilities that are required to be issued first before issuing each capability of the second portion. The number of capabilities that are required to be issued before a given capability can be referred to as an "issuance step." For example, a capability can be assigned and / or associated with five issue steps, indicating that five other capabilities must be issued before issuing the given capability. Generating visualizations with different rankings for portions of capabilities is further described below with reference to Figures 7 and 8.

[0112] The capability path priority subsystem 606 can process the capability dependency data (e.g., as derived from the capability dependency identification and tracking subsystem 602) to derive multiple paths for unblocking a specified capability or set of capabilities and select a critical path that maximizes efficiency when allocating resources to issue capabilities to unblock a selected capability, service, or flock. Deriving a critical path for a selected capability / flock is described in more detail below with respect to Figures 9-11. In response to deriving a critical path for a selected capability / flock, the visualization subsystem 608 can generate one or more visualizations (e.g., visualization 1000) showing the critical path.

[0113] The visualization subsystem 608 can generate one or more visualizations showing various aspects of the capability data. For example, the visualization can include a table (e.g., table 700 in FIG. 7 ) showing flocks (units of change that, when executed, cause one or more corresponding capabilities to be issued) ranked by a first ranking. As another example, the visualization can include a table (e.g., table 800 in FIG. 8 ) showing a second portion of flocks that depend on one or more currently unissued capabilities ranked by a second ranking and specifying a number of issuance steps for each capability. The visualization can include a number of other data fields, such as the flock, phase, team, or any suitable attribute or aspect associated with the capability, and a list of all dependent capabilities for the capability. As another example, the visualization subsystem 608 can generate one or more visualizations showing a critical path, such as visualization 1100 in FIG. 11 . Although the visualization subsystem 608 is shown as operating as part of the multi-flock orchestrator 612, in some embodiments, the visualization subsystem 608 operates as part of the CIOS central 108 of FIG.

[0114] The visualizations (e.g., visualization 614, including any suitable combination of visualizations 700, 800, and / or 1100 of FIG. 7, FIG. 8, and FIG. 11, respectively) can be provided to a client device for further processing and review. For example, resources can be efficiently allocated to issue capabilities according to rankings assigned to the capabilities in the CIOS. In some embodiments, user input can be received in visualizations 700, 800, and / or 1100 (e.g., example user interfaces) to initiate a release (e.g., to initiate a bootstrap task execution for one or more flocks). In some embodiments, user input cannot be provided for entries in the visualizations that correspond to capabilities that depend on at least one unissued capability. In some embodiments, user input can be received in visualizations 700, 800, and / or 1100 to modify the order in which capabilities are issued (e.g., the order in which bootstrap task execution is performed for one or more flocks corresponding to these capabilities).

[0115] B. On-Deck Capability Visualization Overview As described above, capability data for the CIOS can be processed (e.g., by the multi-flock orchestrator 612 of FIG. 6) to assign ranks to the various capabilities based on the dependency information for each capability. Additionally, one or more visualizations can show the capabilities arranged by the rankings assigned to the capabilities. For example, a first visualization can show a capability for which all capabilities on which it depends have already been issued. FIG. 7 is a block diagram illustrating an example visualization of on-deck capabilities (capabilities for which all dependencies have been satisfied, capabilities that depend on capabilities that have not been issued), according to at least one embodiment.

[0116] As shown in FIG. 7, the on-deck visualization 700 may include a table or other similar display of various fields of data. For example, the on-deck visualization 700 may detail aspects of a number of capabilities (e.g., capabilities corresponding to flocks) for which all corresponding dependencies have been satisfied (e.g., issued), arranged by a ranking assigned to each capability. The capabilities depicted within the visualization 700 indicate capabilities that are prepared to be issued since each capability on which the corresponding flock depends has already been issued. These capabilities may be referred to as "on-deck capabilities." For example, capabilities C1-C5 (associated with executing a unit of change corresponding to flock 1 in region 1) may be considered to be on-deck capabilities based at least in part on identifying that all capabilities on which flock 1 in region 1 depends have already been issued.

[0117] As shown in FIG. 7, multiple fields 702-712 can be provided for multiple capabilities / flocks. Each row in the visualization 700 can correspond to a particular flock (e.g., a unit of change that, when executed, causes one or more capabilities to be issued). For example, a first row 714A (corresponding to flock 1 in region 1) can provide details specific to the first flock (e.g., a unit of change that, when executed, causes capabilities C1-C5 to be issued in region 1), and a second row 714B can provide details specific to the second flock (e.g., another unit of change that causes capabilities C1-C5 to be issued in region 2). In some embodiments, each of the fields 702-712 can correspond to a particular flock that does not depend on unissued capabilities. Each of the entries 714A-D (e.g., corresponding to a respective flock / phase / region) can be arranged according to a ranking. For example, Flock 1 in region 1 and Flock 1 in region 2 may be ranked highest by having a greater number of capabilities issued than the number of capabilities issued by releasing Flock 2 in region 2 or Flock 3 in region 3.

[0118] A ranking can be assigned to each capability based on dependency information for each capability. For example, a respective ranking (e.g., ranking 702) can be assigned to a flock or set of capabilities corresponding to a flock based on a number of other capabilities that can be issued (or "unblocked") in response to issuance of that set of capabilities corresponding to the flock. Other factors, such as the team, flock, or region specific to each capability, can be weighted when assigning a ranking to each capability. The ranking can indicate efficiency in allocating resources to issue capabilities in the CIOS. As shown in FIG. 7, capability 714A can be ranked highest based at least in part on a determination that issuing capability 714A (as opposed to any of capabilities 714B, 714C, or 714D) causes the greatest number of other capabilities to be issued.

[0119] In addition to the ranking assigned for each entry 714A-D, other details may be provided. For example, for each entry, a team may be specified (e.g., in the column corresponding to team 704). The team may include a classification or grouping of computing resources for each type of capability. Exemplary teams may relate to telemetry, domain name services (DNS), storage, identity, or any other service / application that may be implemented in a CIOS. As another example, a flock 706 may be associated with each capability. As noted above, a flock may correspond to any suitable attribute that corresponds to one or more resources, capabilities, phases, regions, teams, or units of change (e.g., a unit of change that requires provisioning an infrastructure component, deploying a software artifact in an infrastructure component).

[0120] For example, visualization 700 may specify a phase (e.g., via a column corresponding to phase 708), a region (e.g., region 710), and a capability (e.g., capability 712) generated by executing a unit of change. For example, entry 714A may correspond to executing a unit of change corresponding to flock 1 in region 1, resulting in issuing capabilities C1, C2, C3, C4, and C5. In some examples, visualization 700 may provide specific details for each change / flock, such as the capability generated by the execution of the change and / or a number of other capabilities that are unblocked in response to issuing each capability. Entries 714A-D may be arranged in visualization 700 according to a ranking 702 for each flock. The capabilities when arranged in visualization 700 may provide insight into allocating resources for issuing capabilities.

[0121] The capability visualization can further rank capabilities that have one or more unmet dependencies. For example, to issue a first capability, a second capability may need to be issued first. Thus, the capability visualization can further rank capabilities that have one or more unissued dependent capabilities.

[0122] 8 is a block diagram illustrating an example visualization 800 of blocked capabilities that depend on one or more unissued capabilities. As described above, entries 816A-D illustrated in visualization 800 correspond to blocks and / or corresponding sets of capabilities that depend on one or more unissued capabilities. For example, to unblock the capability corresponding to entry 816A, one step is identified specifying that one other capability needs to be issued in order to issue the capability corresponding to entry 816A (e.g., capabilities C13, C14, C15, and C16).

[0123] In visualization 800, a second ranking 802 can rank entries corresponding to each flock / capability that depends on one or more unissued capabilities to provide insight into future capabilities that will be issued. Similar to ranking 702, ranking 802 can rank capabilities using one or more factors, such as, for example, a number of other capabilities that are unlocked in response to the issuance of each capability. Visualization 800 can further specify teams 804, flocks 806, phases 808, regions 810, and a number of capabilities 814 that each flock generates to provide more details for each flock and / or capability.

[0124] C. Flow process for generating capability visualizations in cloud infrastructure services In some examples, a method for generating a visualization of capabilities in a cloud infrastructure service is provided. Figure 9 is a block diagram 900 illustrating an example method for generating a visualization of capabilities. In some embodiments, the method of Figure 9 may be performed by the multi-flock orchestrator 106, 208, 318, 412, or 612 of Figures 1-4 and 6, respectively.

[0125] At 902, the method may include obtaining one or more flock configuration files corresponding to one or more respective flocks. The one or more flock configuration files may identify a number of capabilities associated with each flock (e.g., changes corresponding to services, applications, resources, etc.) that may be implemented in the cloud infrastructure service. For example, a multi-flock orchestrator (e.g., multi-flock orchestrators 412 and 612 of FIGS. 4 and 6, respectively) may obtain a number of flock configuration files corresponding to changes to be made in a given region / datacenter. The multi-flock orchestrator may perform any suitable number of parsing of the flock configuration files to identify capabilities on which the flock (and capabilities issued by releasing the flock / bootstrapping resources associated with the flock) depend and / or capabilities that depend on capabilities issued in connection with releasing the flock / bootstrapping resources associated with the flock.

[0126] At 904, each identified capability may be processed individually. Processing each capability may include identifying, at 906, a first number of capabilities on which issuing the identified capability depends. Processing each capability may include identifying, at 908, a second number of capabilities that may be issued in response to issuing the identified capability. In some embodiments, identifying the second number of capabilities may include performing any suitable number of parsing of any suitable number of other flock configuration files (e.g., associated with issuing other capabilities). The second number of capabilities may include all capabilities that are unblocked (e.g., releasable) in response to issuing each corresponding capability. Or in other words, the second number of capabilities may include capabilities that may be prepared for issuing in response to issuing each corresponding capability. The unblocked capabilities may be those that are determined not to depend on capabilities that are not currently issued. In other words, an unblocked capability is one that depends only on capabilities that have already been issued. The combination of the first number of capabilities and the second number of capabilities can allow for generating a mapping of dependency data for each capability (e.g., construction dependency graph 338 of FIG. 3).

[0127] At 910, the method may include deriving a rank (or order) for issuing capabilities (and / or an execution order for executing bootstrap tasks associated with the corresponding flock) based on the first number of capabilities and the second number of capabilities. The rank (or order) may be based on a total number of capabilities that are unblocked in response to issuing the capabilities. Furthermore, in some examples, the ranking for each capability may be divided into multiple categories of capabilities (e.g., a first category for all capabilities with all dependencies satisfied (all capabilities on which it depends have been issued) and a second category for capabilities with one or more dependencies unsatisfied (at least one capability on which it depends remains unissued)). A capability ranking subsystem (e.g., capability ranking subsystem 604) of the multi-flock orchestrator 412 may rank the capabilities as described herein.

[0128] The rank of each capability can be assigned based on a number of factors. For example, for each capability, a first weight can be derived based on a number of dependent capabilities for each capability (e.g., capabilities on which the given capability depends). Another exemplary weight can include a number of capabilities (and corresponding bootstrap tasks executed) that may be allowed to be issued in response to issuing the given capability. Other weights can be derived for capabilities, such as factors based on blocks specific to each capability or the issue status of each capability. The assigned rank for each capability can specify the efficiency in issuing resources in the CIOS in response to issuing each corresponding capability.

[0129] At 912, a visualization showing insight into the capabilities can be generated. For example, the visualization can include a first portion identifying capabilities for which all dependencies have been met / resolved and arranged by derived rank (or score) (e.g., capabilities that do not depend on capabilities that are not currently issued). An exemplary first portion of the visualization can include an on-deck visualization 700 in FIG. 7. The on-deck visualization 700 shows capabilities that are prepared to be issued (referred to as "on-deck capabilities"). The visualization can further include a second portion specifying capabilities that depend on one or more unissued dependent capabilities and arranged by derived rank (or score) for each capability, and a number of capabilities that need to be issued to enable / allow issuance of each capability (e.g., issue steps). An exemplary second portion of the visualization can include a blocked capabilities visualization 800 in FIG. 8. The visualizations 700 and 800 can arrange capabilities based on efficiency in allocating resources to build new regions in a cloud infrastructure service.

[0130] In some examples, a visualization can be displayed at a client device. Computing resources can be allocated to issue capabilities based on the visualization. In some examples, the first portion or the second portion of the visualization can include a table. The table can include, for each capability displayed in the visualization, a corresponding rank, a flock of resources, a region in the CIOS, and any other capabilities that can be issued in response to issuing each capability.

[0131] As described above, the multi-flock orchestrator can periodically obtain updated capability data. For example, the multi-flock orchestrator can identify a change in the status of a capability from an unpublished status to a published status (e.g., via data provided by the capability service described above). The visualization can then be updated to show an updated ranking generated by the multi-flock orchestrator for the identified capability. The updated ranking can reflect any changes in status to the identified capability (e.g., dependent capabilities for a particular capability being published). The visualization can be updated based on the updated ranking for the identified capability.

[0132] Thereafter, a first capability (e.g., a blocked capability) previously shown in the second portion of the visualization (e.g., showing a capability / block that depends on at least one unissued capability) can be included in / moved to the first portion of the updated visualization based at least on time until it is determined that all capabilities on which the first capability depends have been satisfied (e.g., issued) and / or the first capability does not depend on any unissued capabilities.

[0133] D. Identify the critical path for a capability or flock As described above, a capability can be associated with multiple dependent capabilities (e.g., capabilities on which a given capability depends) and multiple capabilities that are unblocked in response to issuing the capability (e.g., releasable, not dependent on unissued capabilities, etc.). Thus, to issue a capability, all dependent capabilities may need to be issued first. Furthermore, as part of building a CIOS or adding computing resources to a CIOS, a new capability (or a flock containing multiple capabilities) can be added. A flock can include a unit of change corresponding to one or more capabilities that are issued (e.g., when the unit of change is executed / implemented) to provide new functionality to the CIOS.

[0134] A capability included in a flock can be associated with different capabilities whose issuance is required in order to issue the capability in the flock. Thus, to issue all capabilities in a flock, many different capabilities on which the capabilities of the flock may depend may need to be issued first. However, identifying an order in which the bootstrap operations corresponding to these capabilities are efficiently performed in the CIOS can be difficult due to the large number of capabilities utilized.

[0135] In one embodiment, capability data for a CIOS (e.g., a set of flock configuration changes corresponding to each unit of change associated with bootstrapping a resource in a region) can be processed to derive a critical path that efficiently unblocks a selected flock and / or one or more capabilities in the CIOS. A critical path can specify a list of capabilities (called "dependent capabilities") that, when issued, will unblock the flock, allowing the capabilities associated with the flock to be issued. A critical path may include capabilities associated with multiple flocks. For example, capability A of flock 1 may depend on capability B of flock 2, which in turn depends on capability C of flock 3. A critical path for capability A may indicate that capability C must be released before capability B, which in turn must be released before capability A.

[0136] In response to obtaining a selection of a capability or a flock including multiple capabilities, a ranking can be assigned to all dependent capabilities for the selected capability / flock. The ranking can specify a priority or order for issuing dependencies to efficiently unblock the selected capability flock. Multiple factors, such as the number of selected capabilities that are unblocked in response to issuing each dependent capability, the number of selected capabilities that depend on each dependent capability, etc., can be weighted in deriving a ranking (or order) for each dependent capability. In response to deriving a ranking (or order) for all dependent capabilities, a critical path can be derived that provides a mapping of capabilities that need to be issued to unblock all capabilities associated with a particular capability and / or flock. The critical path can be provided in a visualization that specifies details related to each capability that is issued to unblock the specified capability or flock. The visualization can provide insight into efficiently allocating resources to unblock the selected capability or flock.

[0137] As described above, a resource flock can include multiple capabilities with various dependencies. The dependent capabilities can be located across various flock configuration files in the CIOS. Figure 10 is a block diagram 1000 illustrating an example mapping of dependencies for a selected flock in the CIOS.

[0138] As shown in FIG. 10, a specified flock 1002 may be provided. The flock 1002 may be selected by a client, for example, as part of a query for a critical path to unblock the flock 1002. The flock 1002 may be associated with a number of capabilities that are issued in performing the changes corresponding to the flock 1002. Further, each capability of the flock may include inherent dependent capabilities (e.g., capabilities that are required to be issued first before the capability can be issued). Additionally, an issuance status of each capability and / or dependent capability associated with and / or corresponding to the flock may be identified. Exemplary issuance statuses include, but are not limited to, issued, prepared to be issued, and not issued.

[0139] Additionally, the selected flock 1002 (or capability) may be associated with multiple dependent capabilities. For example, as shown in FIG. 10, the selected flock 1002 (or capability) may depend on multiple other flocks / capabilities (e.g., flocks 1004A-F, 1006A-C, 1008A, and 1008B). Additionally, the capabilities that the flock 1002 depends on may include various statuses. For example, the first set of flocks 1008A and 1008B may depend on one or more currently unissued capabilities. Additionally, the second set of flocks 1004A-F may be ready to be issued (e.g., by not having a dependency on a currently unissued capability). Additionally, the third set of flocks 1006A-C may correspond to issued capabilities. The status of each flock (and / or each capability corresponding to the flock) can be used to map dependency information for each selected capability and derive a critical path as described herein.

[0140] As described above, a critical path for unblocking a flock (corresponding to the selected flock 1002) can be derived based on dependency information for capabilities as obtained from a flock configuration file associated with the flock (and / or additional flock configuration files corresponding to these dependencies). As an illustrative example, a first flock can be selected for issuance. The first flock (and / or the capabilities of the first flock) can depend on capability A, capability B, and capability C. Each of these capabilities is unpublished. Dependency information can be derived for each capability to identify that capability A depends on unpublished capability D, capability B depends on unpublished capability D and unpublished capability E, and capability C depends on unpublished capability D, unpublished capability E, and unpublished capability F. The dependencies on each capability (e.g., dependencies on unpublished capabilities D, E, and F) can be used to assign a rank to each capability. For example, in assigning ranks, it may be determined that unissued capability D unblocks capability A and needs to be issued in order for capabilities A-C to issue. Thus, capability D is assigned a first rank. Additionally, capability E needs to be issued in order for capabilities A and B to issue, and may be assigned a second rank (by identifying that issuing capability E unblocks fewer capabilities than issuing capability D). The issuance of capability F is only required for capability B to issue, and may be assigned a third rank by determining that capability F does not unblock other capabilities and / or unblocks a fewer number of capabilities (or at least fewer than those that depend on capabilities D or E).A critical path for unblocking the first flock can be derived based on the rankings for the dependent capabilities. For example, the critical path for unblocking the first flock can include issuing capability D, then issuing capability E, then issuing capability F. The critical path can be used, for example, to allocate resources for issuing the dependent capabilities to unblock the selected flock.

[0141] As noted above, the critical path may be provided as part of a visualization. FIG. 11 is an example visualization (e.g., critical path visualization 1100) illustrating a critical path for unblocking a specified resource. The visualization 1100 may be displayed in response to a selection of the flock 1002 of FIG. 10. As shown in FIG. 11, the critical path visualization 1100 may specify the selected resource 1102 (e.g., Flock 1). Additionally, the critical path visualization 1100 may include details related to the flocks and / or capabilities on which the selected resource depends. In particular, the critical path visualization 1100 may provide a critical path for releasing the flock and / or issuing a corresponding capability to unblock the selected resource 1102 (e.g., Flock 1). For example, the critical path visualization 1100 may provide entries 1114A-D corresponding to at least some of the flocks 1004A-F, 1006A-C, 1008A, and 1008B of FIG. 10 that are included in the critical path and arranged by ranking. For example, the entries 1114A-D may be arranged based on a path ranking 1104 for each capability 1114A-D, where the path ranking 1104 indicates a critical path for unblocking a selected resource. In some embodiments, the visualization 1100 is scrollable to view any suitable number of dependent capabilities. The critical path shown in FIG. 11 shows that the flock corresponding to entry 1114A is released / bootstrapped first, followed by the flocks corresponding to entries 1114B, 1114C, and 1114D, respectively.

[0142] The visualization 1110 may also include additional details related to each capability on which a selected resource (e.g., flock 1) depends. For example, for each dependent capability, a team 1106, a flock 1108, a phase 1110, and / or a region 1112 may be provided for each capability 1114A-D.

[0143] E. A flow process for generating a visualization of the critical path to issue dependent capabilities to unblock resource flocks 12 is a flow process 1200 for generating a visualization of a critical path for issuing dependent capabilities to unblock a flock of resources. For example, a multi-flock orchestrator (e.g., multi-flock orchestrator 412 of FIG. 4) can execute the process as described herein.

[0144] At 1202, a selection of a flock of resources can be detected (e.g., via one or more user interfaces provided by the CIOS (e.g., by CIOS Central)). An example selection may include selecting flock 1 as shown at 1102 in FIG. 11. In some embodiments, selecting a flock can include obtaining a query for a particular flock from a client (e.g., via a client device). In some examples, detecting the selection of a flock can be from an interaction with a visualization of the flock in the CIOS. For example, a flock (e.g., flock 1002) can be selected from visualization 1000 of FIG. 10 that includes a plurality of various flocks. In some embodiments, a capability can be selected for processing as described herein. In some embodiments, selecting flock 1002 can be performed by selecting flock 1002, which can then navigate the user to critical path visualization 1100 of FIG. 11.

[0145] At 1204, one or more capabilities associated with a number of flocks (including any suitable number of flocks including the selected flock) may be identified (e.g., from one or more parsing of a flock configuration file corresponding to the flocks). The capabilities may relate to services or applications that can be implemented within the cloud infrastructure service.

[0146] As noted above, a flock may include one or more capabilities that are individually associated with other capabilities that must first be issued in order to issue each capability of the flock. Any suitable number of flock configuration files may be processed to identify the capabilities on which each respective capability / flock depends. These identified dependencies between capabilities and / or flocks may then be used to derive a critical path for the selected flock.

[0147] At 1206, each capability identified from the one or more flock configuration files can be processed individually. Processing each capability can include identifying, at 1208, a first number of capabilities on which issuing the identified capability depends. Each of the first number of capabilities can include dependent capabilities that are required to be issued prior to issuance of each corresponding capability. Processing each capability can include identifying, at 1210, a second number of capabilities that can be issued as a result of issuing the identified capability. The second number of capabilities can include all capabilities that are unblocked by issuance of each corresponding capability. The combination of the first number of capabilities and the second number of capabilities can enable generating a mapping of dependency data for each capability (e.g., the construction dependency graph 338 of FIG. 3 ).

[0148] As mentioned above, a flock of a resource may include multiple capabilities. Furthermore, each selected capability may be associated with one or more dependent capabilities that need to be issued to unblock each selected capability. For example, a first set of capabilities as retrieved at 1208 may be identified for each capability on which the selected flock ultimately depends. The selected flock / capability may depend on a second flock / capability (as identified from a corresponding flock configuration file), which may depend on a third flock / capability (as identified from a flock configuration file corresponding to the second flock / capability), and so on. A superset of all capabilities corresponding to any suitable number of flocks / capabilities on which one or more capabilities of the selected flock ultimately depend. Additionally, at 1212, all unissued capabilities may be identified from the first set of flocks / capabilities. These include only unissued capabilities on which the selected flock ultimately depends. The unissued capabilities identified in 1212 can be ranked and assigned as part of a critical path for unblocking the selected capabilities, as described below.

[0149] At 1214, a rank for each of the unissued capabilities can be derived. The unissued dependent capabilities can be assigned a rank based on a number of weighting factors to each of the unissued dependent capabilities. For example, a first weighting factor can include a number of selected capabilities that are unblocked in response to issuance of each unissued capability of the first set of capabilities. A second weighting factor can include a number of selected capabilities that depend on each unissued capability of the first set of capabilities. Other weighting factors can be applied to each unissued capability and can be used to assign a rank to each unissued capability as described herein. The rankings assigned to each unissued dependent capability can be used to prepare for execution of bootstrap operations corresponding to these flocks and / or capabilities for the critical path to unblock the selected flocks.

[0150] At 1216, a visualization (e.g., visualization 1100 of FIG. 11) can be generated from the unissued capabilities arranged according to the derived rank for each unissued capability. The visualization can specify a critical path in unblocking the resource. Further, the unissued dependent capabilities arranged by ranking can maximize efficiency in issuing capabilities to unblock the selected block. The visualization can be provided to a client device for the client to allocate resources to unblock the selected block of resources.

[0151] F. Exemplary Embodiments A first exemplary embodiment provides a method for identifying dependencies between capabilities of a cloud computing environment being constructed. The method may include identifying, by a cloud infrastructure orchestration service from one or more configuration files, a collective set of capabilities that are individually associated with services or applications to be bootstrapped by the cloud infrastructure orchestration service within the cloud computing environment being constructed. The method may include identifying, for each respective capability of the collective set of capabilities, by the cloud infrastructure orchestration service, a first set of capabilities on which publishing the respective capability depends. The method may include generating, by the cloud infrastructure orchestration service, a visualization including a first portion specifying a first subset of capabilities of the collective set of capabilities that do not depend on unpublished capabilities based at least in part on identifying the first set of capabilities, and a second portion specifying a second subset of capabilities of the collective set of capabilities that depend on one or more currently unpublished capabilities.

[0152] In some embodiments, the second portion further specifies a respective number of unissued capabilities for each capability. In some embodiments, issuance of each capability may be dependent on issuance of a respective number of unissued capabilities. In some embodiments, the first portion or the second portion of the visualization includes, for each specified capability in the first portion or the second portion, a respective table showing a corresponding flock and a corresponding region.

[0153] In some embodiments, the method may include identifying one or more updates to the published status of individual capabilities of the collective set of capabilities from capability data obtained from the capability service. The method may include generating an updated ranking based at least in part on the one or more updates. The method may include generating an updated visualization that includes an updated first portion that identifies a third subset of capabilities of the collective set of capabilities that do not depend on capabilities that have not been published according to the one or more updates, and an updated second portion that identifies the third subset of capabilities of the collective set of capabilities that depend on the set of capabilities that have not been currently published according to the one or more updates.

[0154] In some embodiments, a first capability initially included in the second portion of the visualization is newly included in the updated first portion of the updated visualization in response to identifying, from one or more updates, that a second capability on which the first capability depends has been published.

[0155] In some embodiments, the method may include assigning a ranking to the second subset of capabilities based at least in part on identifying a number of the first set of capabilities that correspond to each of the second subset of capabilities. In some embodiments, the ranking is generated based at least in part on identifying, for each respective capability of the collective set of capabilities, a second set of capabilities that can be issued in response to issuing the respective capability, and a second visualization is generated to show the second set of capabilities.

[0156] A second method (e.g., a method for determining a critical path that identifies an order for bootstrapping a subset of resources in a datacenter under construction by a cloud infrastructure orchestration service) is disclosed. In some embodiments, the method includes identifying, by the cloud infrastructure orchestration service from one or more configuration files, a collective set of capabilities that are individually associated with resources to be bootstrapped by the cloud infrastructure orchestration service in the datacenter under construction. The method may include identifying, for each respective capability of the collective set of capabilities, by the cloud infrastructure orchestration service a first set of capabilities on which publishing the respective capability depends. The method may include identifying a user input that identifies a selected flock. The method may include identifying, for the selected flock, by the cloud infrastructure orchestration service, one or more unpublished capabilities that correspond to at least one of the one or more capabilities associated with the selected flock. The method may include ranking the unpublished capabilities. The method may include generating a visualization that identifies at least a portion of the unpublished capabilities that correspond to the selected flock. In some embodiments, unpublished capabilities may be identified and arranged according to a ranking.

[0157] In some embodiments, the ranking of the unissued capabilities corresponding to the selected flocks specifies at least a critical path that identifies an order for bootstrapping a subset of resources within the data center under construction.

[0158] In some embodiments, the method may include causing a display of a visualization at a client device, where computing resources are allocated to bootstrap resources corresponding to the unpublished capabilities based at least in part on the ranking.

[0159] In some embodiments, the visualization includes a table that includes, for each unissued capability, the assigned rank, the corresponding block, and the corresponding region in the cloud infrastructure service.

[0160] In some embodiments, the method may include identifying, from the capability data obtained from the capability service, one or more updates to the publishing status of individual capabilities of the unpublished capabilities. The method may include updating the visualization to identify the previously unpublished capabilities as being published based at least in part on the one or more updates.

[0161] In some embodiments, the method may include deriving updated rankings for remaining unissued capabilities identified as not being issued after receiving the one or more updates, and updating the visualization such that the remaining unissued capabilities are arranged within the visualization according to the updated rankings.

[0162] In some embodiments, the ranking is derived based at least in part on identifying, for each unissued capability, a set of capabilities that can be issued in response to issuing the respective unissued capability.

[0163] Another embodiment relates to a cloud computing system that includes one or more processors and one or more memories storing computer-executable instructions that, when executed by the one or more processors, cause a computing device, component, or service (e.g., any suitable combination of the components of the CIOS of FIG. 1) to perform any suitable combination of the methods disclosed herein.

[0164] Yet another embodiment relates to a non-transitory computer-readable medium storing computer-executable instructions that, when executed by one or more processors of a cloud computing system, cause a computing device, component, or service (e.g., any suitable combination of the components of the CIOS of FIG. 1) to perform any suitable combination of the methods disclosed herein.

[0165] G.IaaS overview As mentioned above, Infrastructure as a Service (IaaS) is one particular type of cloud computing. IaaS can be configured to provide virtualized computing resources over a public network (e.g., the Internet). In the IaaS model, a cloud computing provider can host infrastructure components (e.g., servers, storage devices, network nodes (e.g., hardware), deployment software, platform virtualization (e.g., hypervisor layer), etc.). In some cases, an IaaS provider may supply various services (e.g., billing, monitoring, logging, load balancing, and clustering, etc.) to accompany these infrastructure components. Thus, these services may be policy-driven, so that an IaaS user may be able to implement policies to drive load balancing to maintain application availability and performance.

[0166] In some examples, an IaaS customer may access resources and services over a wide area network (WAN), such as the Internet, and can use the cloud provider's services to install the remaining elements of the application stack. For example, a user can log into an IaaS platform to create virtual machines (VMs), install an operating system (OS) on each VM, deploy middleware such as databases, create storage buckets for workloads and backups, and even install enterprise software on the VMs. The customer can then use the provider's services to perform a variety of functions, including balancing network traffic, troubleshooting application issues, monitoring performance, managing disaster recovery, and the like.

[0167] In most cases, the cloud computing model requires the participation of a cloud provider, which may be, but does not have to be, a third-party service that specializes in providing (offering, renting, selling) IaaS. An entity may choose to deploy a private cloud and become its own provider of infrastructure services.

[0168] In some examples, IaaS deployment is the process of adding a new application or a new version of an application to a provisioned application server, etc. It may also include the process of provisioning the server (e.g., installing libraries, daemons, etc.). This is often managed by the cloud provider below the hypervisor layer (e.g., server, storage, network hardware, and virtualization). Thus, the customer may be responsible for handling the (OS), middleware, and / or application deployment (e.g., in self-service virtual machines (e.g., that can be spun up on demand)).

[0169] In some instances, IaaS provisioning may refer to acquiring a computer or virtual host for use and then installing required libraries or services on it. In most cases, deployment does not include provisioning, and provisioning may not need to be performed first.

[0170] In some cases, there are two distinct challenges for IaaS provisioning. First, there is the initial challenge of provisioning an initial set of infrastructure before anything is operational. Second, there is the challenge of evolving the existing infrastructure once everything is provisioned (e.g., adding new services, modifying services, removing services, etc.). In some cases, these two challenges may be solved by allowing the configuration of the infrastructure to be defined declaratively. In other words, the infrastructure (e.g., which components are required and how they interact) can be defined by one or more configuration files. Thus, the overall topology of the infrastructure (e.g., which resources depend on which, how they each work together) can be described declaratively. In some instances, once the topology is defined, workflows can be generated that generate and / or manage the different components described in the configuration files.

[0171] In some examples, the infrastructure may have many interconnected elements. For example, there may be one or more virtual private clouds (VPCs) (e.g., potentially on-demand pools of configurable and / or shared computing resources), also known as a core network. In some examples, there may also be one or more inbound / outbound traffic groups and one or more virtual machines (VMs) that are provisioned to define how the network's inbound and / or outbound traffic is set up. Other infrastructure elements, such as load balancers, databases, etc., may also be provisioned. As more infrastructure elements are desired and / or added, the infrastructure may grow incrementally.

[0172] In some examples, continuous deployment techniques may be employed to enable deployment of infrastructure code across various virtual computing environments. In addition, the described techniques may enable infrastructure management within these environments. In some examples, a service team may write code that is desired to be deployed to one or more, but often many, different production environments (e.g., across various different geographic locations, sometimes across the globe). However, in some examples, the infrastructure onto which the code will be deployed must first be set up. In some examples, provisioning may be done manually, provisioning tools may be utilized to provision the resources, and / or deployment tools may be utilized to deploy the code once the infrastructure is provisioned.

[0173] 13 is a block diagram 1300 illustrating an example pattern of an IaaS architecture, according to at least one embodiment. A service operator 1302 can be communicatively coupled to a secure host tenancy 1304, which can include a virtual cloud network (VCN) 1306 and a secure host subnet 1308. In some examples, the service operator 1302 can use one or more client computing devices, which can be portable handheld devices (e.g., iPhone®, mobile phone, iPad®, computing tablet, personal digital assistant (PDA)) or wearable devices (e.g., Google Glass® head-mounted display) running software such as Microsoft Windows Mobile® and / or various mobile operating systems such as iOS, Windows Phone, Android, BlackBerry 15, Palm OS, and enabled for Internet, email, short message service (SMS), Blackberry®, or other communications protocols. Alternatively, the client computing devices can be general purpose personal computers including, for example, personal and / or laptop computers running various versions of Microsoft Windows, Apple Macintosh, and / or Linux operating systems. The client computing devices can be, for example, workstation computers running any of the various commercially available UNIX or UNIX-like operating systems, including, without limitation, various GNU / Linux operating systems, such as Google Chrome OS.Alternatively, or in addition, the client computing devices may be any other electronic device, such as a thin-client computer, an Internet-enabled gaming system (e.g., a Microsoft Xbox gaming console with or without a Kinect® gesture input device), and / or a personal messaging device, capable of communicating over a network that has access to the VCN 1306 and / or the Internet.

[0174] The VCN 1306 can include a local peering gateway (LPG) 1310 that can be communicatively coupled to a secure shell (SSH) VCN 1312 via an LPG 1310 included in the SSH VCN 1312. The SSH VCN 1312 can include an SSH subnet 1314, which can be communicatively coupled to a control plane VCN 1316 via an LPG 1310 included in the control plane VCN 1316. The SSH VCN 1312 can also be communicatively coupled to a data plane VCN 1318 via the LPG 1310. The control plane VCN 1316 and the data plane VCN 1318 can be included in a service tenancy 1319, which can be owned and / or operated by the IaaS provider.

[0175] The control plane VCN 1316 can include a control plane demilitarized zone (DMZ) tier 1320 that serves as a perimeter network (e.g., the portion of an enterprise network between the enterprise intranet and an external network). DMZ-based servers may have limited responsibility and may help contain breaches. Additionally, the DMZ tier 1320 can include one or more load balancer (LB) subnets 1322, a control plane app tier 1324 that can include app subnets 1326, a control plane data tier 1328 that can include database (DB) subnets 1330 (e.g., a front-end DB subnet and / or a back-end DB subnet). The LB subnet 1322 included in the control plane DMZ tier 1320 can be communicatively coupled to an app subnet 1326 included in the control plane app tier 1324 and an Internet gateway 1334 that can be included in the control plane VCN 1316, and the app subnet 1326 can be communicatively coupled to a DB subnet 1330, a service gateway 1336, and a network address translation (NAT) gateway 1338 included in the control plane data tier 1328. The control plane VCN 1316 can include the service gateway 1336 and the NAT gateway 1338.

[0176] The control plane VCN 1316 can include a data plane mirror Apri Tier 1340, which can include an app subnet 1326. The app subnet 1326 included in the data plane mirror Apri Tier 1340 can include a virtual network interface controller (VNIC) 1342 on which a compute instance 1344 can run. The compute instance 1344 can communicatively couple the app subnet 1326 of the data plane mirror Apri Tier 1340 to the app subnet 1326, which can be included in the data plane Apri Tier 1346.

[0177] The data plane VCN 1318 can include a data plane A-Tier 1346, a data plane DMZ tier 1348, and a data plane data tier 1350. The data plane DMZ tier 1348 can include a LB subnet 1322 that can be communicatively coupled to an app subnet 1326 of the data plane A-Tier 1346 and an Internet gateway 1334 of the data plane VCN 1318. The app subnet 1326 can be communicatively coupled to a service gateway 1336 of the data plane VCN 1318 and a NAT gateway 1338 of the data plane VCN 1318. The data plane data tier 1350 can also include a DB subnet 1330 that can be communicatively coupled to the app subnet 1326 of the data plane A-Tier 1346.

[0178] The Internet gateways 1334 of the control plane VCNs 1316 and data plane VCNs 1318 can be communicatively coupled to a metadata management service 1352, which can be communicatively coupled to the public Internet 1354. The public Internet 1354 can be communicatively coupled to a NAT gateway 1338 of the control plane VCNs 1316 and data plane VCNs 1318. The service gateways 1336 of the control plane VCNs 1316 and data plane VCNs 1318 can be communicatively coupled to cloud services 1356.

[0179] In some examples, the service gateways 1336 of the control plane VCN 1316 and the data plane VCN 1318 can make application programming interface (API) calls to the cloud services 1356 without traversing the public Internet 1354. The API calls from the service gateway 1336 to the cloud services 1356 can be one-way. The service gateway 1336 can make the API calls to the cloud services 1356, and the cloud services 1356 can send the requested data to the service gateway 1336. However, the cloud services 1356 may not initiate the API calls to the service gateway 1336.

[0180] In some examples, secure host tenancy 1304 can be directly connected to service tenancy 1319, which may otherwise be isolated. Secure host subnet 1308 can communicate with SSH subnet 1314 through LPG 1310, which may enable bidirectional communication on an otherwise isolated system. Connecting secure host subnet 1308 to SSH subnet 1314 may give secure host subnet 1308 access to other entities in service tenancy 1319.

[0181] The control plane VCN 1316 may enable users of the service tenancy 1319 to set up or otherwise provision desired resources. The desired resources provisioned in the control plane VCN 1316 may be deployed or otherwise used in the data plane VCN 1318. In some examples, the control plane VCN 1316 can be isolated from the data plane VCN 1318, and the data plane mirror Apri tier 1340 of the control plane VCN 1316 can communicate with the data plane Apri tier 1346 of the data plane VCN 1318 via a VNIC 1342 that can be included in the data plane mirror Apri tier 1340 and the data plane Apri tier 1346.

[0182] In some examples, a user, or customer, of the system may make a request, e.g., create, read, update, or delete (CRUD) operations, over the public internet 1354, which may communicate the request to a metadata management service 1352. The metadata management service 1352 may communicate the request to the control plane VCN 1316 through an internet gateway 1334. The request may be received by a LB subnet 1322 included in the control plane DMZ tier 1320. The LB subnet 1322 may determine that the request is valid, and in response to this determination, the LB subnet 1322 may send the request to an app subnet 1326 included in the control plane app tier 1324. If the request is validated and requests a call to the public internet 1354, the call to the public internet 1354 may be sent to a NAT gateway 1338, which may make the call to the public internet 1354. Memory that may be desired to be stored with the request may be stored in the DB subnet 1330.

[0183] In some examples, the data plane mirror ApriTia 1340 can facilitate direct communication between the control plane VCN 1316 and the data plane VCN 1318. For example, it may be desired that changes to the configuration, updates, or other appropriate modifications be applied to the resources included in the data plane VCN 1318. Through the VNIC 1342, the control plane VCN 1316 can communicate directly with the resources included in the data plane VCN 1318, thereby enabling it to perform the changes to the configuration, updates, or other appropriate modifications thereon.

[0184] In some embodiments, the control plane VCN 1316 and the data plane VCN 1318 can be included in the service tenancy 1319. In this case, a user or customer of the system may not own or operate either the control plane VCN 1316 or the data plane VCN 1318. Instead, an IaaS provider may own or operate the control plane VCN 1316 and the data plane VCN 1318, both of which may be included in the service tenancy 1319. This embodiment may allow for network isolation that may prevent users or customers from interacting with other users' or other customers' resources. This embodiment may also allow users or customers of the system to store databases privately, without having to rely on the public internet 1354 for storage, which may not have the desired level of threat protection.

[0185] In other embodiments, the LB subnet 1322 included in the control plane VCN 1316 can be configured to receive signals from the service gateway 1336. In this embodiment, the control plane VCN 1316 and the data plane VCN 1318 may be configured to be called by the IaaS provider's customers without calling the public Internet 1354. The IaaS provider's customers may desire this embodiment because databases used by the customers may be stored in the service tenancy 1319, which may be controlled by the IaaS provider and may be isolated from the public Internet 1354.

[0186] FIG. 14 is a block diagram 1400 illustrating another example pattern of an IaaS architecture, according to at least one embodiment. A service operator 1402 (e.g., service operator 1302 of FIG. 13) can be communicatively coupled to a secure host tenancy 1404 (e.g., secure host tenancy 1304 of FIG. 13), which can include a virtual cloud network (VCN) 1406 (e.g., VCN 1306 of FIG. 13) and a secure host subnet 1408 (e.g., secure host subnet 1308 of FIG. 13). The VCN 1406 can include a local peering gateway (LPG) 1410 (e.g., LPG 1310 of FIG. 13), which can be communicatively coupled to a secure shell (SSH) VCN 1412 (e.g., SSH VCN 1312 of FIG. 13) via an LPG 1310 included in the SSH VCN 1412. SSH VCN 1412 can include an SSH subnet 1414 (e.g., SSH subnet 1314 in FIG. 13), which can be communicatively coupled to a control plane VCN 1416 (e.g., control plane VCN 1316 in FIG. 13) via an LPG 1410 included in the control plane VCN 1416. The control plane VCN 1416 can be included in a service tenancy 1419 (e.g., service tenancy 1319 in FIG. 13), and the data plane VCN 1418 (e.g., data plane VCN 1318 in FIG. 13) can be included in a customer tenancy 1421, which can be owned or operated by a user or customer of the system.

[0187] The control plane VCN 1416 may include a control plane DMZ tier 1420 (e.g., control plane DMZ tier 1320 of FIG. 13 ) which may include a LB subnet 1422 (e.g., LB subnet 1322 of FIG. 13 ), a control plane Aplica tier 1424 (e.g., control plane Aplica tier 1324 of FIG. 13 ) which may include an app subnet 1426 (e.g., app subnet 1326 of FIG. 13 ), and a control plane data tier 1428 (e.g., control plane data tier 1328 of FIG. 13 ) which may include a database (DB) subnet 1430 (e.g., similar to the DB subnet 1330 of FIG. 13 ). The LB subnet 1422 included in the control plane DMZ tier 1420 can be communicatively coupled to an app subnet 1426 included in the control plane app tier 1424 and an Internet gateway 1434 (e.g., Internet gateway 1334 of FIG. 13 ) that can be included in the control plane VCN 1416, and the app subnet 1426 can be communicatively coupled to a DB subnet 1430, a service gateway 1436 (e.g., service gateway of FIG. 13 ), and a network address translation (NAT) gateway 1438 (e.g., NAT gateway 1338 of FIG. 13 ) included in the control plane data tier 1428. The control plane VCN 1416 can include the service gateway 1436 and the NAT gateway 1438.

[0188] The control plane VCN 1416 can include a data plane mirror Apri tier 1440 (e.g., data plane mirror Apri tier 1340 of FIG. 13 ), which can include an app subnet 1426. The app subnet 1426 included in the data plane mirror Apri tier 1440 can include a virtual network interface controller (VNIC) 1442 (e.g., VNIC of 1342) on which a compute instance 1444 (e.g., similar to the compute instance 1344 of FIG. 13 ) can run. The compute instance 1444 can facilitate communication between the app subnet 1426 of the data plane mirror Apri tier 1440 and a supplement subnet 1426 that can be included in the data plane Apri tier 1446 (e.g., data plane Apri tier 1346 of FIG. 13 ) via the VNIC 1442 included in the data plane mirror Apri tier 1440 and the VNIC 1442 included in the data plane Apri tier 1446.

[0189] An Internet gateway 1434 included in the control plane VCN 1416 can be communicatively coupled to a metadata management service 1452 (e.g., metadata management service 1352 of FIG. 13), which can be communicatively coupled to a public Internet 1454 (e.g., public Internet 1354 of FIG. 13). The public Internet 1454 can be communicatively coupled to a NAT gateway 1438 included in the control plane VCN 1416. A service gateway 1436 included in the control plane VCN 1416 can be communicatively coupled to cloud services 1456 (e.g., cloud services 1356 of FIG. 13).

[0190] In some examples, the data plane VCN 1418 can be included in the customer tenancy 1421. In this case, the IaaS provider may provide a control plane VCN 1416 for each customer, and the IaaS provider may set up a unique compute instance 1444 included in the service tenancy 1419 for each customer. Each compute instance 1444 can enable communication between the control plane VCN 1416 included in the service tenancy 1419 and the data plane VCN 1418 included in the customer tenancy 1421. The compute instance 1444 allows resources provisioned in the control plane VCN 1416 included in the service tenancy 1419 to be deployed or otherwise used in the data plane VCN 1418 included in the customer tenancy 1421.

[0191] In other examples, an IaaS provider customer may have a database that resides in customer tenancy 1421. In this example, control plane VCN 1416 may include a data plane mirror Apri Tier 1440, which may include app subnet 1426. Data plane mirror Apri Tier 1440 may reside in data plane VCN 1418, but data plane mirror Apri Tier 1440 may not reside in data plane VCN 1418. That is, data plane mirror Apri Tier 1440 may have access to customer tenancy 1421, but data plane mirror Apri Tier 1440 may not reside in data plane VCN 1418 or be owned or operated by an IaaS provider customer. Data plane mirror Apri Tier 1440 may be configured to make calls to data plane VCN 1418, but may not be configured to make calls to any entities included in control plane VCN 1416. A customer may wish to deploy or otherwise use resources in a data plane VCN 1418 that are provisioned in a control plane VCN 1416, but the data plane mirror AppLitera 1440 can facilitate the customer's desired deployment or other use of the resources.

[0192] In some embodiments, the IaaS provider's customer can apply filters to the data plane VCN 1418. In this embodiment, the customer can determine which data plane VCNs 1418 can access, and the customer may limit access from the data plane VCN 1418 to the public Internet 1454. The IaaS provider may not be able to apply filters or otherwise control the access of the data plane VCN 1418 to any external networks or databases. Applying filters and controls by the customer to the data plane VCN 1418 included in the customer tenancy 1421 can help isolate the data plane VCN 1418 from other customers and the public Internet 1454.

[0193] In some embodiments, cloud services 1456 can be called by service gateway 1436 to access services in control plane VCN 1416 or data plane VCN 1418 that may not exist on the public Internet 1454. The connection between cloud services 1456 and control plane VCN 1416 or data plane VCN 1418 may not be live or continuous. Cloud services 1456 may exist in different networks owned or operated by an IaaS provider. Cloud services 1456 may be configured to receive calls from service gateway 1436 and may not be configured to receive calls from the public Internet 1454. Some cloud services 1456 may be isolated from other cloud services 1456, and control plane VCN 1416 may be isolated from cloud services 1456 that may not be in the same region as control plane VCN 1416. For example, control plane VCN 1416 may be located in “Region 1” and cloud service “Deployment 13” may be located in Region 1 and Region 2. When a call is made to deployment 13 by service gateway 1436 included in control plane VCN 1416 located in region 1, the call may be sent to deployment 13 in region 1. In this example, control plane VCN 1416, or deployment 13 in region 1, may not be communicatively coupled to or otherwise in communication with deployment 13 in region 2.

[0194] FIG. 15 is a block diagram 1500 illustrating another example pattern of an IaaS architecture, according to at least one embodiment. A service operator 1502 (e.g., service operator 1302 of FIG. 13) can be communicatively coupled to a secure host tenancy 1504 (e.g., secure host tenancy 1304 of FIG. 13), which can include a virtual cloud network (VCN) 1506 (e.g., VCN 1306 of FIG. 13) and a secure host subnet 1508 (e.g., secure host subnet 1308 of FIG. 13). The VCN 1506 can include an LPG 1510 (e.g., LPG 1310 of FIG. 13), which can be communicatively coupled to an SSH VCN 1512 (e.g., SSH VCN 1312 of FIG. 13) via an LPG 1510 included in the SSH VCN 1512. SSH VCN 1512 can include an SSH subnet 1514 (e.g., SSH subnet 1314 in FIG. 13), which can be communicatively coupled to a control plane VCN 1516 (e.g., control plane VCN 1316 in FIG. 13) via an LPG 1510 included in the control plane VCN 1516, and to a data plane VCN 1518 (e.g., data plane 1318 in FIG. 13) via an LPG 1510 included in the data plane VCN 1518. The control plane VCN 1516 and the data plane VCN 1518 can be included in a service tenancy 1519 (e.g., service tenancy 1319 in FIG. 13).

[0195] The control plane VCN 1516 may include a control plane DMZ tier 1520 (e.g., control plane DMZ tier 1320 of FIG. 13 ) which may include a load balancer (LB) subnet 1522 (e.g., LB subnet 1322 of FIG. 13 ), a control plane Aplica tier 1524 (e.g., control plane Aplica tier 1324 of FIG. 13 ) which may include an app subnet 1526 (e.g., similar to the app subnet 1326 of FIG. 13 ), and a control plane data tier 1528 (e.g., control plane data tier 1328 of FIG. 13 ) which may include a DB subnet 1530. The LB subnet 1522 included in the control plane DMZ tier 1520 can be communicatively coupled to an app subnet 1526 included in the control plane app tier 1524 and an Internet gateway 1534 (e.g., Internet gateway 1334 of FIG. 13 ) that can be included in the control plane VCN 1516, and the app subnet 1526 can be communicatively coupled to a DB subnet 1530 included in the control plane data tier 1528, a service gateway 1536 (e.g., service gateway of FIG. 13 ), and a network address translation (NAT) gateway 1538 (e.g., NAT gateway 1338 of FIG. 13 ). The control plane VCN 1516 can include the service gateway 1536 and the NAT gateway 1538.

[0196] The data plane VCN 1518 can include a data plane A-P-Tier 1546 (e.g., data plane A-P-Tier 1346 of FIG. 13), a data plane DMZ tier 1548 (e.g., data plane DMZ tier 1348 of FIG. 13), and a data plane data tier 1550 (e.g., data plane data tier 1350 of FIG. 13). The data plane DMZ tier 1548 can include a LB subnet 1522, which can be communicatively coupled to a trusted app subnet 1560 and an untrusted app subnet 1562 of the data plane A-P-Tier 1546, and an Internet gateway 1534 included in the data plane VCN 1518. The trusted app subnet 1560 can be communicatively coupled to a service gateway 1536 included in the data plane VCN 1518, a NAT gateway 1538 included in the data plane VCN 1518, and a DB subnet 1530 included in the data plane data tier 1550. The untrusted app subnet 1562 can be communicatively coupled to a service gateway 1536 included in the data plane VCN 1518 and a DB subnet 1530 included in the data plane data tier 1550. The data plane data tier 1550 can include a DB subnet 1530 that can be communicatively coupled to a service gateway 1536 included in the data plane VCN 1518.

[0197] The untrusted app subnet 1562 can include one or more primary VNICs 1564(1)-(N) that can be communicatively coupled to tenant virtual machines (VMs) 1566(1)-(N). Each tenant VM 1566(1)-(N) can be communicatively coupled to a respective app subnet 1567(1)-(N) that can be included in a respective container egress VCN 1568(1)-(N) that can be included in a respective customer tenancy 1570(1)-(N). Each secondary VNIC 1572(1)-(N) can facilitate communication between the untrusted app subnet 1562 included in the data plane VCN 1518 and the app subnet included in the container egress VCN 1568(1)-(N). Each container egress VCN 1568(1)-(N) may include a NAT gateway 1538 that may be communicatively coupled to the public Internet 1554 (e.g., the public Internet 1354 in FIG. 13).

[0198] An Internet gateway 1534 included in the control plane VCN 1516 and the data plane VCN 1518 can be communicatively coupled to a metadata management service 1552 (e.g., metadata management system 1352 of FIG. 13 ), which can be communicatively coupled to the public Internet 1554. The public Internet 1554 can be communicatively coupled to a NAT gateway 1538 included in the control plane VCN 1516 and the data plane VCN 1518. A service gateway 1536 included in the control plane VCN 1516 and the data plane VCN 1518 can be communicatively coupled to cloud services 1556.

[0199] In some embodiments, data plane VCN 1518 can be integrated with customer tenancies 1570. This integration may be useful or desirable for an IaaS provider's customer in some cases, such as when they may want support when executing code. The customer may provide code to execute that may be disruptive, may communicate with other customer resources, or may otherwise produce undesirable effects. In response, the IaaS provider may determine whether to execute the code provided to the IaaS provider by the customer.

[0200] In some examples, a customer of an IaaS provider may grant temporary network access to the IaaS provider and may request a function to be attached to data plane tier app 1546. Code to execute the function may be executed in VMs 1566(1)-(N), and the code may be configured to not execute anywhere else in data plane VCN 1518. Each VM 1566(1)-(N) may be connected to one customer tenancy 1570. Each container 1571(1)-(N) included in VM 1566(1)-(N) may be configured to execute code. In this case, there may be dual isolation (e.g., containers 1571(1)-(N) executing code, which may be contained in at least VMs 1566(1)-(N) contained in untrusted app subnet 1562), which may help prevent erroneous or otherwise unwanted code from damaging the IaaS provider's network or damaging a different customer's network. Containers 1571(1)-(N) may be communicatively coupled to customer tenancy 1570 and configured to send or receive data from customer tenancy 1570. Containers 1571(1)-(N) may not be configured to send or receive data from any other entity in data plane VCN 1518. Once the code execution is complete, the IaaS provider may kill or otherwise discard containers 1571(1)-(N).

[0201] In some embodiments, trusted app subnet 1560 may execute code that may be owned or operated by the IaaS provider. In this embodiment, trusted app subnet 1560 may be communicatively coupled to DB subnet 1530 and may be configured to perform CRUD operations on DB subnet 1530. Untrusted app subnet 1562 may be communicatively coupled to DB subnet 1530, but in this embodiment, untrusted app subnet 1562 may be configured to perform read operations on DB subnet 1530. Containers 1571(1)-(N), which may be included in each customer's VMs 1566(1)-(N) and may execute code from the customer, may not be communicatively coupled to DB subnet 1530.

[0202] In other embodiments, the control plane VCN 1516 and the data plane VCN 1518 may not be directly communicatively coupled. In this embodiment, there may not be direct communication between the control plane VCN 1516 and the data plane VCN 1518. However, communication may occur indirectly through at least one method. An LPG 1510 may be established by an IaaS provider, and the LPG 1510 may facilitate communication between the control plane VCN 1516 and the data plane VCN 1518. In another example, the control plane VCN 1516 or the data plane VCN 1518 may make a call to a cloud service 1556 through the service gateway 1536. For example, a call from the control plane VCN 1516 to the cloud service 1556 may include a request for a service that may communicate with the data plane VCN 1518.

[0203] FIG. 16 is a block diagram 1600 illustrating another example pattern of an IaaS architecture, according to at least one embodiment. A service operator 1602 (e.g., service operator 1302 of FIG. 13) can be communicatively coupled to a secure host tenancy 1604 (e.g., secure host tenancy 1304 of FIG. 13), which can include a virtual cloud network (VCN) 1606 (e.g., VCN 1306 of FIG. 13) and a secure host subnet 1608 (e.g., secure host subnet 1308 of FIG. 13). The VCN 1606 can include an LPG 1610 (e.g., LPG 1310 of FIG. 13), which can be communicatively coupled to an SSH VCN 1612 (e.g., SSH VCN 1312 of FIG. 13) via an LPG 1610 included in the SSH VCN 1612. SSH VCN 1612 can include an SSH subnet 1614 (e.g., SSH subnet 1314 in FIG. 13), which can be communicatively coupled to a control plane VCN 1616 (e.g., control plane VCN 1316 in FIG. 13) via an LPG 1610 included in the control plane VCN 1616, and to a data plane VCN 1618 (e.g., data plane VCN 1318 in FIG. 13) via an LPG 1610 included in the data plane VCN 1618. The control plane VCN 1616 and the data plane VCN 1618 can be included in a service tenancy 1619 (e.g., service tenancy 1319 in FIG. 13).

[0204] The control plane VCN 1616 may include a control plane DMZ tier 1620 (e.g., control plane DMZ tier 1320 of FIG. 13) which may include a LB subnet 1622 (e.g., LB subnet 1322 of FIG. 13), a control plane Aplica tier 1624 (e.g., control plane Aplica tier 1324 of FIG. 13) which may include an App subnet 1626 (e.g., App subnet 1326 of FIG. 13), and a control plane data tier 1628 (e.g., control plane data tier 1328 of FIG. 13) which may include a DB subnet 1630 (e.g., DB subnet 1530 of FIG. 15). The LB subnet 1622 included in the control plane DMZ tier 1620 can be communicatively coupled to an app subnet 1626 included in the control plane app tier 1624 and an Internet gateway 1634 (e.g., Internet gateway 1334 in FIG. 13 ) that can be included in the control plane VCN 1616, and the app subnet 1626 can be communicatively coupled to a DB subnet 1630, a service gateway 1636 (e.g., service gateway in FIG. 13 ), and a network address translation (NAT) gateway 1638 (e.g., NAT gateway 1338 in FIG. 13 ) included in the control plane data tier 1628. The control plane VCN 1616 can include the service gateway 1636 and the NAT gateway 1638.

[0205] The data plane VCN 1618 can include a data plane A-P-Tier 1646 (e.g., data plane A-P-Tier 1346 of FIG. 13), a data plane DMZ tier 1648 (e.g., data plane DMZ tier 1348 of FIG. 13), and a data plane data tier 1650 (e.g., data plane data tier 1350 of FIG. 13). The data plane DMZ tier 1648 can include a trusted app subnet 1660 (e.g., trusted app subnet 1560 of FIG. 15) and an untrusted app subnet 1662 (e.g., untrusted app subnet 1562 of FIG. 15) of the data plane A-P-Tier 1646, as well as a LB subnet 1622 that can be communicatively coupled to an Internet gateway 1634 included in the data plane VCN 1618. The trusted app subnet 1660 may be communicatively coupled to a service gateway 1636 included in the data plane VCN 1618, a NAT gateway 1638 included in the data plane VCN 1618, and a DB subnet 1630 included in the data plane data tier 1650. The untrusted app subnet 1662 may be communicatively coupled to a service gateway 1636 included in the data plane VCN 1618 and a DB subnet 1630 included in the data plane data tier 1650. The data plane data tier 1650 may include a DB subnet 1630 that may be communicatively coupled to a service gateway 1636 included in the data plane VCN 1618.

[0206] The untrusted app subnet 1662 can include primary VNICs 1664(1)-(N) that can be communicatively coupled to tenant virtual machines (VMs) 1666(1)-(N) that reside within the untrusted app subnet 1662. Each tenant VM 1666(1)-(N) can execute code in a respective container 1667(1)-(N) and can be communicatively coupled to an app subnet 1626 that can be included in a data plane app tier 1646 that can be included in a container egress VCN 1668. Each secondary VNIC 1672(1)-(N) can facilitate communication between the untrusted app subnet 1662 included in the data plane VCN 1618 and the app subnet included in the container egress VCN 1668. The container egress VCN can include a NAT gateway 1638 that can be communicatively coupled to the public Internet 1654 (e.g., public Internet 1354 of FIG. 13).

[0207] An Internet gateway 1634 included in the control plane VCN 1616 and the data plane VCN 1618 can be communicatively coupled to a metadata management service 1652 (e.g., metadata management service 1352 of FIG. 13 ), which can be communicatively coupled to the public Internet 1654. The public Internet 1654 can be communicatively coupled to a NAT gateway 1638 included in the control plane VCN 1616 and the data plane VCN 1618. A service gateway 1636 included in the control plane VCN 1616 and the data plane VCN 1618 can be communicatively coupled to cloud services 1656.

[0208] In some examples, the pattern illustrated by the architecture of block diagram 1600 of FIG. 16 may be considered an exception to the pattern illustrated by the architecture of block diagram 1500 of FIG. 15 and may be desirable for an IaaS provider's customer when the IaaS provider cannot directly communicate with the customer (e.g., disconnected region). Each container 1667(1)-(N) included in VM 1666(1)-(N) for each customer can be accessed in real time by the customer. The containers 1667(1)-(N) may be configured to make calls to a respective secondary VNIC 1672(1)-(N) included in app subnet 1626 of data plane app tier 1646 that may be included in container egress VCN 1668. The secondary VNIC 1672(1)-(N) can send the call to NAT gateway 1638, which may send the call to public Internet 1654. In this example, containers 1667(1)-(N) that may be accessed in real time by customers may be isolated from control plane VCN 1616 and may be isolated from other entities included in data plane VCN 1618. Containers 1667(1)-(N) may be isolated from resources from other customers.

[0209] In another example, a customer may use containers 1667(1)-(N) to call cloud service 1656. In this example, the customer may execute code in containers 1667(1)-(N) that requests a service from cloud service 1656. Containers 1667(1)-(N) can send the request to secondary VNICs 1672(1)-(N), which can send the request to a NAT gateway, which can send the request to public Internet 1654. Public Internet 1654 can send the request to LB subnet 1622 included in control plane VCN 1616 via Internet gateway 1634. In response to determining that the request is valid, LB subnet can send the request to app subnet 1626, which can send the request to cloud service 1656 via service gateway 1636.

[0210] It should be appreciated that the illustrated IaaS architectures 1300, 1400, 1500, 1600 may have components other than those shown. Additionally, the illustrated embodiments are only some examples of cloud infrastructure systems that may incorporate disclosed embodiments. In some other embodiments, the IaaS systems may have more or fewer components than shown, may combine two or more components, or may have a different configuration or arrangement of components.

[0211] In one embodiment, the IaaS system described herein may include a suite of application, middleware, and database service offerings that are delivered to customers in a self-service, subscription-based, elastically scalable, reliable, highly available, and secure manner. An example of such an IaaS system is Oracle Cloud Infrastructure (OCI), offered by the present assignee.

[0212] 17 illustrates an exemplary computer system 1700 upon which various embodiments may be implemented. The system 1700 may be used to implement any of the computer systems described above. As shown, the computer system 1700 includes a processing unit 1704 that communicates with a number of peripheral subsystems via a bus subsystem 1702. These peripheral subsystems may include a processing acceleration unit 1706, an I / O subsystem 1708, a storage subsystem 1718, and a communication subsystem 1724. The storage subsystem 1718 includes a tangible computer readable storage medium 1722 and a system memory 1710.

[0213] Bus subsystem 1702 provides a mechanism for allowing the various components and subsystems of computer system 1700 to communicate with each other as intended. Although bus subsystem 1702 is shown diagrammatically as one bus, alternative embodiments of the bus subsystem may utilize multiple buses. Bus subsystem 1702 may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. For example, such architectures may include an Industry Standard Architecture (ISA) bus, a MicroChannel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus, which may be implemented as a mezzanine bus manufactured in accordance with the IEEE P 1386.1 standard.

[0214] Processing unit 1704, which may be implemented as one or more integrated circuits (e.g., conventional microprocessors or microcontrollers), controls the operation of computer system 1700. One or more processors may be included in processing unit 1704. These processors may include single-core or multi-core processors. In some embodiments, processing unit 1704 may be implemented as one or more independent processing units 1732 and / or 1734 with single-core or multi-core processors included in each processing unit. In other embodiments, processing unit 1704 may be implemented as a quad-core processing unit formed by integrating two dual-core processors into one chip.

[0215] In various embodiments, the processing unit 1704 can execute various programs in response to program code and can maintain multiple simultaneously executing programs or processes. At any given time, some or all of the program code being executed can reside in the processor 1704 and / or in the memory subsystem 1718. With appropriate programming, the processor 1704 can provide various functions as described above. The computer system 1700 may additionally include a processing acceleration unit 1706, which can include a digital signal processor (DSP), a special purpose processor, and / or the like.

[0216] The I / O subsystem 1708 may include user interface input devices and user interface output devices. User interface input devices may include a keyboard, a pointing device such as a mouse or trackball, a touchpad or touch screen integrated into a display, a scroll wheel, a click wheel, a dial, a button, a switch, a keypad, an audio input device with a voice command recognition system, a microphone, and other types of input devices. User interface input devices may include motion sensing and / or gesture recognition devices such as a Microsoft Kinect® motion sensor that allows a user to control and interact with an input device such as a Microsoft Xbox® 360 game controller through a natural user interface using gestures and voice instructions. User interface input devices may also include eye gesture recognition devices such as a Google Glass® blink detector that detects eye movements from a user (e.g., "blinking" when taking a picture and / or selecting a menu) and translates the eye gestures as input to an input device (e.g., Google Glass®). Additionally, the user interface input devices may include a voice recognition sensing device that allows a user to interact with a voice recognition system (e.g., the Siri® Navigator) through voice commands.

[0217] User interface input devices may also include, but are not limited to, three-dimensional (3D) mice, joysticks or pointing sticks, game pads and graphic tablets, and audio / visual devices such as speakers, digital cameras, digital camcorders, portable media players, webcams, image scanners, fingerprint scanners, barcode readers 3D scanners, 3D printers, laser range finders, and eye gaze detection devices. In addition, user interface input devices may also include, for example, medical image input devices such as computed tomography, magnetic resonance imaging, positron emission tomography, and medical ultrasound devices. User interface input devices may also include, for example, audio input devices such as MIDI keyboards, digital musical instruments, and the like.

[0218] The user interface output devices may include a display subsystem, indicator lights, or non-visual displays such as audio output devices. The display subsystem may be a cathode ray tube (CRT), a flat panel device such as a flat panel device using a liquid crystal display (LCD) or a plasma display, a projection device, a touch screen, etc. In general, use of the term "output device" is intended to include all possible types of devices and mechanisms for outputting information from computer system 1700 to a user or to another computer. For example, user interface output devices may include, without limitation, various display devices that visually convey text, graphics, and audio / video information, such as monitors, printers, speakers, headphones, automobile navigation systems, plotters, voice output devices, and modems.

[0219] Computer system 1700 may include a storage subsystem 1718 that provides a tangible, non-transitory computer-readable storage medium for storing software and data structures that provide the functionality of the embodiments described in this disclosure. The software may include programs, code modules, instructions, scripts, etc. that, when executed by one or more cores or processors of processing unit 1704, provide the functionality described above. Storage subsystem 1718 may also provide a repository for storing data used in accordance with the present disclosure.

[0220] 17, the storage subsystem 1718 can include various components including a system memory 1710, a computer readable storage medium 1722, and a computer readable storage medium reader 1720. The system memory 1710 may store program instructions loadable and executable by the processing unit 1704. The system memory 1710 may store data used during the execution of the instructions and / or data generated during the execution of the program instructions. A variety of different types of programs may be loaded into the system memory 1710 including, but not limited to, client applications, web browsers, mid-tier applications, relational database management systems (RDBMS), virtual machines, containers, and the like.

[0221] System memory 1710 may store operating system 1716. Examples of operating system 1716 may include various versions of Microsoft Window, Apple Macintosh, and / or Linux operating systems, various commercially available UNIX or UNIX-like operating systems (including, without limitation, various GNU / Linux operating systems, Google Chrome OS, etc.) and / or mobile operating systems, such as iOS, Windows Phone, Android OS, BlackBerry OS, and Palm OS operating systems. In certain implementations in which computer system 1700 runs one or more virtual machines, the virtual machine with a guest operating system (GOS) may be loaded into system memory 1710 and executed by one or more processors or cores of processing unit 1704.

[0222] The system memory 1710 may be provided in different configurations depending on the type of computer system 1700. For example, the system memory 1710 may be volatile memory (e.g., random access memory (RAM)) and / or non-volatile memory (e.g., read only memory (ROM), flash memory, etc.). Different types of RAM configurations may be provided, including static random access memory (SRAM), dynamic random access memory (DRAM), and others. In some implementations, the system memory 1710 may include a basic input / output system (BIOS) containing the basic routines that help to transfer information between elements within the computer system 1700, such as during start-up.

[0223] Computer-readable storage media 1722 may represent remote, local, fixed, and / or removable storage devices and media for temporarily and / or more permanently containing and storing computer-readable information for use by computer system 1700, including instructions executable by processing unit 1704 of computer system 1700.

[0224] The computer readable storage medium 1722 may include any suitable medium known or used in the art, including storage media and communication media, such as, but not limited to, volatile and non-volatile, removable and non-removable media, implemented in any method or technology for storing and / or transmitting information. This may include tangible computer readable storage media, such as RAM, ROM, Electronically Erasable Programmable ROM (EEPROM), flash memory or other memory technology, CD-ROM, Digital Versatile Disk (DVD) or other optical storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or other tangible computer readable media.

[0225] For example, the computer readable storage medium 1722 may include a hard disk drive that reads from or writes to a non-removable, non-volatile magnetic medium, a magnetic disk drive that reads from or writes to a removable, non-volatile magnetic disk, an optical disk drive that reads from or writes to a removable, non-volatile optical disk or other optical medium, such as a CD-ROM, DVD, and Blu-Ray® disk. The computer readable storage medium 1722 may include, but is not limited to, a Zip® drive, a flash memory card, a Universal Serial Bus (USB) flash drive, a Secure Digital (SD) card, a DVD disk, a digital video tape, and the like. The computer readable storage medium 1722 may also include a flash memory-based SSD, an enterprise flash drive, a solid-state drive (SSD) based on non-volatile memory such as solid-state ROM, a solid-state RAM, a dynamic RAM, a static RAM, a DRAM-based SSD, an SSD based on volatile memory such as a magnetoresistive RAM (MRAM) SSD, a hybrid SSD that uses a combination of DRAM and flash memory mace SSD. The disk drives and their associated computer-readable media may provide non-volatile storage of computer-readable instructions, data structures, program modules, and other data for computer system 1700.

[0226] Machine-readable instructions executable by one or more processors or cores of the processing unit 1704 may be stored in a non-transitory computer-readable storage medium. A non-transitory computer-readable storage medium may include physically tangible memory or storage devices, including volatile memory storage devices and / or non-volatile storage devices. Examples of non-transitory computer-readable storage media include magnetic storage media (e.g., disks or tapes), optical storage media (e.g., DVDs, CDs), various types of RAM, ROM, or flash memory, hard drives, floppy drives, removable memory drives (e.g., USB drives), or other types of storage devices.

[0227] The communications subsystem 1724 provides an interface to other computer systems and networks. The communications subsystem 1724 serves as an interface for receiving data from other systems and transmitting data from the computer system 1700 to other systems. For example, the communications subsystem 1724 may enable the computer system 1700 to be connected to one or more devices via the Internet. In some embodiments, the communications subsystem 1724 may include a radio frequency (RF) transceiver component for accessing wireless voice and / or data networks (e.g., using cellular technology, advanced data network technologies such as 3G, 4G, or EDGE (Enhanced Data Rates for Global Evolution), WiFi (IEEE 1502.11 family of standards, or other mobile communications technologies, or any combination thereof), a global positioning system (GPS) receiver component, and / or other components). In some embodiments, the communications subsystem 1724 may provide a wired network connection (e.g., Ethernet) in addition to or instead of a wireless interface.

[0228] In some embodiments, the communications subsystem 1724 may receive incoming communications in the form of structured and / or unstructured data feeds 1726, event streams 1728, event updates 1730, etc., on behalf of one or more users who may be using the computer system 1700.

[0229] For example, the communications subsystem 1724 may be configured to receive data feeds 1726 in real time from users of social networks and / or other communications services, such as web feeds such as Twitter® feeds, Facebook® updates, Rich Site Summary (RSS) feeds, and / or real-time updates from one or more third-party information sources.

[0230] Additionally, the communications subsystem 1724 may be configured to receive data in the form of a continuous data stream, which may include an event stream 1728 of real-time events and / or event updates 1730, which may be continuous or essentially open-ended. Examples of applications that generate continuous data may include, for example, sensor data applications, financial tickers, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, automobile traffic monitoring, etc.

[0231] The communications subsystem 1724 may be configured to output structured and / or unstructured data feeds 1726, event streams 1728, event updates 1730, etc. to one or more databases that may communicate with one or more streaming data source computers coupled to the computer system 1700.

[0232] The computer system 1700 can be one of a variety of types, including a handheld portable device (e.g., an iPhone® mobile phone, an iPad® computing tablet, a PDA), a wearable device (e.g., a Google Glass® head-mounted display), a PC, a workstation, a mainframe, a kiosk, a server rack, or any other data processing system.

[0233] Due to the ever-changing nature of computers and networks, the illustrated description of computer system 1700 is intended as a specific example only. Many other configurations having more or fewer components than the illustrated system are possible. For example, customized hardware may be used and / or particular elements may be implemented in hardware, firmware, software (including applets), or a combination. Furthermore, connections to other computing devices, such as network input / output devices, may be employed. Based on the disclosure and teachings provided herein, one of ordinary skill in the art will recognize other ways and / or methods for implementing the various embodiments.

[0234] Although specific embodiments have been described, various modifications, variations, alternative configurations, and equivalents are within the scope of the disclosure. The embodiments are not limited to operation in one particular data processing environment, but are free to operate in multiple data processing environments. Although the embodiments have been described with a particular sequence of transactions and steps, it should be apparent to one skilled in the art that the scope of the disclosure is not limited to the described sequence of transactions and steps. Various features and aspects of the above-described embodiments may be used individually or in combination.

[0235] Furthermore, while embodiments are described using a particular combination of hardware and software, it should be recognized that other combinations of hardware and software are within the scope of the present disclosure. The embodiments may be implemented solely in hardware, solely in software, or using a combination thereof. The various processes described herein can be implemented on the same processor or on different processors in any combination. Thus, where a component or module is described as being configured to perform a certain operation, such configuration can be achieved, for example, by designing an electronic circuit to perform the operation, by programming a programmable electronic circuit (such as a microprocessor) to perform the operation, or in any combination thereof. The processors can communicate using a variety of techniques, including but not limited to conventional techniques for inter-process communication, and different pairs of processes may use different techniques, or the same pair of processes may use different techniques at different times.

[0236] The embodiments may be implemented by using a computer program product comprising computer programs / instructions which, when executed by a processor, cause the processor to perform any of the methods described in the disclosure.

[0237] Accordingly, the specification and drawings are to be regarded in an illustrative rather than restrictive sense. It will be apparent, however, that additions, subtractions, deletions, and other modifications and changes may be made thereto without departing from the broader spirit and scope set forth in the claims. Thus, although certain disclosed embodiments have been described, they are not intended to be limiting. Various modifications and equivalents are intended to be within the scope of the following claims.

[0238] In the context of describing the disclosed embodiments (particularly in the context of the claims that follow), the use of the terms "a," "an," and "the" and similar referents should be construed to cover both the singular and the plural, unless otherwise indicated herein or clearly contradicted by context. The terms "comprising," "having," "including," and "containing" should be construed as open-ended terms (i.e., meaning "including, but not limited to"), unless otherwise noted. The term "connected" should be construed as partially or wholly contained within, attached to, or joined to, even if there is something intervening. The recitation of ranges of values ​​herein is merely intended to serve as a shorthand method of referring to each separate value contained within the range, unless otherwise indicated herein, and each separate value is incorporated into the specification as if it were individually recited herein. All methods described herein can be performed in any suitable order, unless otherwise indicated herein or clearly contradicted by context. The use of any and all examples or exemplary language (e.g., "etc.") provided herein is intended merely to better illuminate the embodiments and does not pose a limitation on the scope of the disclosure unless otherwise claimed. No language in the specification should be construed as indicating any non-claimed element as essential to the practice of the disclosure.

[0239] Disjunctive language, such as the phrase "at least of X, Y, or Z," is intended to be understood in context as being generally used to indicate that an item, term, etc. may be either X, Y, or Z, or any combination thereof (e.g., X, Y and / or Z), unless specifically stated otherwise. Thus, such disjunctive language is not intended, and should not generally imply, that an embodiment requires that at least one of X, at least one of Y, or at least one of Z, respectively, be present.

[0240] Preferred embodiments of the present disclosure are described herein, including the best mode known for carrying out the disclosure. Variations of these preferred embodiments may become apparent to those skilled in the art upon reading the above description. Those skilled in the art should be able to adopt such variations as appropriate, and the disclosure may be carried out in forms other than those specifically described herein. Accordingly, this disclosure includes all modifications and equivalents of the subject matter recited in the claims appended hereto as permitted by applicable law. Moreover, any combination of the above-described elements in all possible variations thereof is encompassed by the disclosure, unless otherwise indicated herein.

[0241] All references cited in this specification, including publications, patent applications, and patents, are hereby incorporated by reference to the same extent as if each reference was individually and specifically indicated to be incorporated by reference and was set forth in its entirety herein.

[0242] In the foregoing specification, aspects of the disclosure are described in relation to specific embodiments thereof, but those skilled in the art will recognize that the disclosure is not limited thereto. Various features and aspects of the above-described disclosure may be used individually or in combination. Moreover, the embodiments can be utilized in any number of environments and applications beyond those described herein without departing from the broader spirit and scope of the specification. Accordingly, the specification and drawings should be regarded as illustrative rather than restrictive.

Claims

1. 1. A method for determining a critical path by a cloud infrastructure orchestration service that identifies an order for bootstrapping a subset of resources within a data center under construction, the method comprising: identifying, by the cloud infrastructure orchestration service from one or more configuration files, a collective set of capabilities individually associated with resources to be bootstrapped by the cloud infrastructure orchestration service within the data center under construction; identifying, for each respective capability in the collective set of capabilities, a first set of capabilities on which the cloud infrastructure orchestration service depends to publish the respective capability; Identifying a user input identifying a selected flock; identifying, for the selected flock, one or more unpublished capabilities by the cloud infrastructure orchestration service corresponding to at least one of the one or more capabilities associated with the selected flock; ranking the unissued capabilities; and generating a visualization that identifies at least a portion of the unissued capabilities corresponding to the selected flock, wherein the unissued capabilities are identified and arranged according to the ranking.

2. 2. The method of claim 1, wherein the ranking of the unissued capabilities corresponding to the selected flock specifies at least the critical path that identifies the order for bootstrapping the subset of resources within the data center under construction.

3. 10. The method of claim 1, further comprising: generating a display of the visualization at a client device, wherein computing resources are allocated to bootstrap resources corresponding to the unpublished capabilities based at least in part on the ranking.

4. The method of claim 1 , wherein the visualization includes a table including, for each unissued capability, an assigned rank, a corresponding block, and a corresponding region in the cloud infrastructure service.

5. identifying, from capability data obtained from the capability service, one or more updates to the issuing status of each of the unpublished capabilities; updating the visualization to identify previously unissued capabilities as being issued based at least in part on the one or more updates; and The method of claim 1 further comprising:

6. 6. The method of claim 5, further comprising deriving updated rankings for remaining unissued capabilities identified as unissued after receiving the one or more updates, wherein updating the visualization causes the remaining unissued capabilities to be arranged within the visualization according to the updated rankings.

7. 10. The method of claim 1 , wherein ranking the unissued capabilities is based at least in part on identifying, for each unissued capability, a set of capabilities that can be issued in response to issuing the each unissued capability.

8. 1. A cloud computing system, comprising: one or more processors; When executed by the one or more processors, the cloud infrastructure orchestration service identifying, from one or more configuration files, a collective set of capabilities individually associated with resources to be bootstrapped by said cloud infrastructure orchestration service within the data center under construction; identifying, for each respective capability in the collective set of capabilities, a first set of capabilities on which issuing the respective capability depends; Identifying a user input identifying a selected flock; identifying, for the selected flock, one or more unissued capabilities corresponding to at least one of the one or more capabilities associated with the selected flock; deriving a ranking for said unissued capabilities; and one or more memories storing computer-executable instructions for generating a visualization that identifies at least a portion of the unissued capabilities corresponding to the selected flock, wherein the unissued capabilities are identified and arranged according to the ranking.

9. 9. The cloud computing system of claim 8, wherein the ranking of the unissued capabilities corresponding to the selected flocks specifies at least a critical path that identifies an order for bootstrapping the subset of resources within the data center under construction.

10. 10. The cloud computing system of claim 8 or claim 9, wherein executing the instructions further causes the cloud infrastructure orchestration service to display the visualization at a client device, with computing resources allocated to bootstrap resources corresponding to the unpublished capabilities based at least in part on the ranking.

11. 10. The cloud computing system of claim 8 or claim 9, wherein the visualization includes a table including, for each unissued capability, an assigned rank, a corresponding block, and a corresponding region in the cloud infrastructure service.

12. Executing the instructions further includes causing the cloud infrastructure orchestration service to: identifying, from the capability data obtained from the capability service, one or more updates to the issuing status of each of the unissued capabilities; 10. The cloud computing system of claim 8 or claim 9, further comprising: updating the visualization to identify previously unpublished capabilities as being published based at least in part on the one or more updates.

13. Executing the instructions further includes causing the cloud infrastructure orchestration service to:

13. The cloud computing system of claim 12, further comprising: deriving updated rankings for remaining unpublished capabilities identified as unpublished after receiving the one or more updates; and updating the visualization so that the remaining unpublished capabilities are arranged in the visualization according to the updated rankings.

14. 10. The cloud computing system of claim 8 or claim 9, wherein ranking the unissued capabilities is based at least in part on identifying, for each unissued capability, a set of capabilities that can be issued in response to issuing the each unissued capability.

15. A program comprising computer executable instructions that, when executed by one or more processors of a cloud infrastructure orchestration service, cause the cloud infrastructure orchestration service to perform the method of any one of claims 1 to 7.

Images