Techniques for Resource Discovery

Abstract

Techniques are described for identifying resources within a region of a cloud computing environment. A resource identification service (RIS) can be configured to obtain a flock configuration file that includes resource discovery data associated with the service. The resource discovery data can indicate a set of parameters. Existing resources of the cloud computing environment are identified using the set of parameters. The RIS can perform operations to identify existing resources based at least in part on matching attributes associated with the existing resources with the set of parameters of the resource discovery data. The RIS can identify a set of import operations to perform from the flock configuration file to obtain identifiers corresponding to the existing resources. The identifiers can be provided to allow the existing resources to be used in the region construction.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Provisional Patent Application No. 63 / 308,003, entitled "Techniques for Bootstrapping a Region Build," filed February 8, 2022, U.S. Provisional Patent Application No. 63 / 312,814, entitled "Techniques for Implementing Virtual Data Centers," filed February 22, 2022, and U.S. Provisional Patent Application No. 63 / 315,043, entitled "Techniques for Resource Discovery," filed February 28, 2022, the disclosures of which are incorporated by reference in their entireties herein for all purposes. Summary of the Invention [Problem to be solved by the invention]

[0002] background Currently, cloud infrastructure services use many individual services to build datacenters (e.g., to bootstrap various resources in datacenters in a particular geographic region). In one example, a region is a logical abstraction that corresponds to a localized geographic area in which one or more datacenters are located (or will be located). Building a datacenter may include provisioning and configuring infrastructure resources and deploying code to those resources (e.g., for various services). The actions to build a datacenter may be collectively referred to as "building a region." A region may include any suitable number of datacenters, and thus building a region may include actions to build multiple datacenters. Traditional means to build a datacenter require significant manual effort. Furthermore, bootstrapping actions for one service may depend on other features and / or services in the region that may not yet be available. As the number of service teams and regions increases, the tasks to be performed to orchestrate provisioning and deployment increase significantly. Relying heavily on manual effort to bootstrap services and / or build datacenters may be time intensive, introduce risks, and may not scale well.

[0003] Quick Overview Embodiments of the present disclosure relate to a resource identification service (e.g., a "resource hunter") configured to identify existing resources (e.g., infrastructure components, artifacts, data, etc.) that can be used to bootstrap services in a region instead of creating new resources. An orchestration service (e.g., a "multi-flock orchestrator") is disclosed that can orchestrate the region construction process to bootstrap many services in a region. As part of the region construction process, the multi-flock orchestrator can use the resource identification service to identify existing resources and orchestrate the bootstrap operation such that the identified resources are used for the bootstrap operation instead of creating new resources. [Means for solving the problem]

[0004] At least one embodiment is directed to a computer-implemented method. The method may include a resource identification service of a cloud computing environment obtaining a flock configuration file including resource discovery data associated with the service. In an embodiment, the resource discovery data indicates a set of parameters. Existing resources of the cloud computing environment are identified using the set of parameters. The method may further include the resource discovery service performing an operation to identify the existing resources. In an embodiment, the existing resources may be identified based at least in part on matching attributes associated with each of the existing resources with the set of parameters of the resource discovery data. The method may further include the resource identification service identifying a set of import operations to perform from the flock configuration file to store identifiers corresponding to the identified existing resources. The method may further include transmitting identifiers corresponding to the identified existing resources based at least in part on the execution of the set of import operations.

[0005] In at least one embodiment, the set of parameters includes at least one of: a location to search for an existing resource, or values ​​corresponding to attributes of the existing resource.

[0006] In at least one embodiment, an identifier corresponding to the existing resource is sent to an orchestration service of the cloud computing environment, hi one embodiment, sending the identifier to an orchestration service of the cloud computing environment causes the data center to use the existing resource instead of generating a new resource.

[0007] In at least one embodiment, the existing resources include at least one of infrastructure components, data, or applications.

[0008] In at least one embodiment, the set of import actions is specified from a configuration file.

[0009] In at least one embodiment, the method further includes: i) the resource identification service obtaining a second set of parameters from the resource discovery data, where a second existing resource in the cloud computing environment is identified using the second set of parameters, and the method further includes: ii) the resource identification service performing additional operations to identify the second existing resource identified based at least in part on matching attributes associated with the second existing resource with the second set of parameters of the resource discovery data; iii) the resource identification service identifying a second set of import operations to perform to obtain a second identifier corresponding to the second existing resource; and iv) providing, to the computing component, the second identifier corresponding to the second existing resource identified based at least in part on performing the second set of import operations.

[0010] In at least one embodiment, the method further includes executing a set of import operations, where executing the set of import operations causes the resource identification service to identify an address corresponding to a location of the existing resource or an identifier of the existing resource.

[0011] Another embodiment is directed to a cloud computing system including one or more processors and instructions that, when executed by the one or more processors, cause a resource identification service to perform any suitable combination of the methods disclosed herein.

[0012] Yet another embodiment is directed to a non-transitory computer-readable medium storing computer-executable instructions that, when executed by one or more processors of a cloud computing system, cause a resource identification service to perform any suitable combination of the methods disclosed herein.

[0013] At least one embodiment is directed to a computer-implemented method. The method may include a multi-floc orchestrator of a cloud computing environment obtaining a plurality of floc configuration files corresponding to a plurality of services to be bootstrapped within the region during a region construction process. The method may further include the multi-floc orchestrator determining an order in which the plurality of services are bootstrapped within the region based at least in part on the plurality of floc configuration files. The method may further include the multi-floc orchestrator sending a first request to bootstrap a service of the plurality of services. The method may further include the multi-floc orchestrator receiving, at least in part on the sending of the first request, planning data indicating resources to be created to bootstrap the service. The method may further include the multi-floc orchestrator obtaining, from a resource identification service, an identifier corresponding to a previously created resource of the cloud computing environment. The method may further include the multi-floc orchestrator modifying the planning data to include an identifier corresponding to the previously created resource of the cloud computing environment. The method may further include the multi-flock orchestrator sending a second request to bootstrap the service, the second request including the planning data including an identifier corresponding to the previously created resource, and the provisioning and deployment manager using the identifier corresponding to the previously created resource to cause the service to be bootstrapped within the region using the previously created resource.

[0014] In one embodiment, the second request is sent to a provisioning and deployment service of a cloud computing environment.

[0015] In an embodiment, the method further includes validating the planning data modified to include the identifier.

[0016] In one embodiment, validating the planning data includes i) sending a third request including the planning data modified to include the identifier, ii) receiving updated planning data in response to the third request, and iii) comparing the updated planning data with the planning data modified to include the identifier, and validating the planning data is determined at least in part based on comparing the updated planning data with the modified planning data.

[0017] In an embodiment, obtaining an identifier corresponding to a previously created resource of the cloud computing environment from the resource identification service further includes the orchestration service sending a configuration file of the plurality of configuration files to the resource identification service, the configuration file being associated with a service of the plurality of services, and the resource identification service identifying the previously created resource based at least in part on data included in the configuration file.

[0018] In an embodiment, the method further includes implementing a state machine for managing transitions between the plurality of states, wherein at least one of determining an order in which the plurality of services are bootstrapped into the data center, sending the first request, obtaining an identifier corresponding to a previously created resource, modifying the planning data, or sending the second request is performed at least in part based on the state machine identifying a particular state among the plurality of states.

[0019] In one embodiment, the method further includes transitioning the state machine from a first state to a second state of the plurality of states based at least in part on one or more messages received from the capability service or the resource identification service.

[0020] Another embodiment is directed to a cloud computing system including one or more processors and instructions that, when executed by the one or more processors, cause an orchestration service to perform any suitable combination of the methods disclosed herein.

[0021] Yet another embodiment is directed to a non-transitory computer-readable medium storing computer-executable instructions that, when executed by one or more processors of a cloud computing system, cause an orchestration service to perform any suitable combination of the methods disclosed herein.

[0022] To facilitate identifying the description of any particular element or act, one or more of the most significant digits of a reference number may refer to the figure number in which that element first appears. [Brief description of the drawings]

[0023] [Figure 1] FIG. 1 is a block diagram illustrating an environment in which a Cloud Infrastructure Orchestration Service (CIOS) can operate to dynamically provide bootstrap services in a region, according to at least one embodiment. [Diagram 2] FIG. 1 is a block diagram illustrating an environment and method for building a virtual bootstrap environment (ViBE) according to at least one embodiment. [Diagram 3] 1 is a block diagram illustrating an environment and method for bootstrapping a service into a target region (eg, a target data center) using ViBE, according to at least one embodiment. [Figure 4]FIG. 1 is a block diagram illustrating an environment in which a cloud infrastructure orchestration service (CIOS) can use resource discovery to discover resources during region construction, according to at least one embodiment. [Diagram 5] FIG. 2 illustrates an example flock configuration file that includes one or more code segments related to resource detection, according to at least one embodiment. [Figure 6] FIG. 2 is a block diagram illustrating an example flow for identifying resources corresponding to a region construction, according to at least one embodiment. [Figure 7] FIG. 1 illustrates an example flow diagram illustrating several states in which execution of a bootstrap operation and / or a resource discovery operation is orchestrated in accordance with at least one embodiment. [Figure 8] FIG. 2 is a block diagram illustrating an example instance of state data in accordance with at least one embodiment. [Figure 9] FIG. 1 illustrates an example method for detecting existing resources, according to at least one embodiment. [Figure 10] FIG. 1 illustrates an example method of using existing resources to perform region construction, according to at least one embodiment. [Figure 11] FIG. 1 is a block diagram illustrating one pattern for implementing a cloud infrastructure as a service system, according to at least one embodiment. [Figure 12] FIG. 1 is a block diagram illustrating another pattern for implementing a cloud infrastructure as a service system, in accordance with at least one embodiment. [Figure 13] FIG. 1 is a block diagram illustrating another pattern for implementing a cloud infrastructure as a service system, in accordance with at least one embodiment. [Figure 14] FIG. 1 is a block diagram illustrating another pattern for implementing a cloud infrastructure as a service system, in accordance with at least one embodiment. [Figure 15]FIG. 1 is a block diagram illustrating an example computer system in accordance with at least one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0024] Detailed Description In the following description, for purposes of explanation, specific details are set forth in order to provide a thorough understanding of certain embodiments. It will be understood, however, that various embodiments can be practiced without these specific details. The figures and descriptions are not intended to be limiting. The term "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any embodiment or design described herein as "exemplary" should not necessarily be construed as preferred or advantageous over other embodiments or designs.

[0025] Example automated data center construction (region construction) infrastructure Recently, there has been an exponential increase in the adoption of cloud services. Currently, various types of cloud services are offered by various different cloud service providers (CSPs). The term cloud service is commonly used to refer to services or functionality provided on-demand (e.g., via a subscription model) by a CSP to a user or customer using systems and infrastructure (cloud infrastructure) provided by the CSP. Typically, the servers and systems that make up the CSP's infrastructure and that are used to provide cloud services to customers are separate from the customer's own on-premise services and systems. Thus, customers can utilize cloud services provided by the CSP without having to purchase separate hardware and software resources for the service. Cloud services are designed to give subscribing customers easy, scalable, on-demand access to applications and computing resources without the customer having to invest in procuring the infrastructure used to provide the service or functionality. Various different types or models of cloud services may be offered, such as Software-as-a-Service (SaaS), Platform-as-a-Service (PaaS), Infrastructure-as-a-Service (IaaS), etc. A customer may subscribe to one or more cloud services offered by a CSP. A customer may be any entity, such as an individual, an organization, or a business.

[0026] As mentioned above, a CSP is responsible for providing the infrastructure and resources used to provide cloud services to its subscribing customers. The resources provided by the CSP can include both hardware and software resources. These resources can include, for example, computational resources (e.g., virtual machines, containers, applications, processors), memory resources (e.g., databases, data stores), networking resources (e.g., routers, host machines, load balancers), identity and other resources. In one implementation, the resources provided by the CSP to provide a set of cloud services CSPs are organized as a data center. A data center can be configured to provide a particular set of cloud services. The CSP is responsible for equipping the data center with the infrastructure and resources used to provide that particular set of cloud services. A CSP can build one or more data centers.

[0027] Data centers offered by a CSP can be hosted in different regions. A region is a localized geographic area and can be identified by a region name. Regions are generally independent of each other and can be separated by great distances, such as across multiple countries or even continents. Regions are grouped into realms. Examples of CSP regions can include US West, US East, Australia East, Australia Southeast, etc.

[0028] A region may include one or more data centers, and the data centers may be located within a geographic area corresponding to the region. As an example, the data centers in a region may be located in a city within the region. For example, for a particular CSP, a data center in the US West region may be located in San Jose, California, a data center in the US East region may be located in Ashburn, Virginia, a data center in the Australia East region may be located in Sydney, Australia, a data center in the Australia Southeast region may be located in Melbourne, Australia, etc.

[0029] Data centers within a region can be organized into one or more availability domains, which are used for high availability and disaster recovery purposes. An availability domain can contain one or more data centers within a region. Availability domains within a region are isolated from each other, fault tolerant, and designed in a manner that makes it highly unlikely that data centers in more than one availability domain will fail at the same time. For example, availability domains within a region can be built in a manner that makes it highly unlikely that a failure in one availability domain within a region will affect the availability of data centers in other availability domains in the same region.

[0030] When a customer or subscriber subscribes or signs up for one or more services offered by a CSP, the CSP creates a tenancy for that customer. A tenancy is like an account created for a customer. In one implementation, a customer's tenancy exists in a single realm and has access to all regions that belong to that realm. The customer's users can then access the services to which the customer subscribes under this tenancy.

[0031] As described above, a CSP builds or deploys a data center to provide cloud services to the CSP's customers. As the CSP's customer base grows, the CSP typically builds a new data center in a new region or expands the capacity of an existing data center to accommodate the growing demand of the customers and to better serve the customers. Preferably, the data center is built in close geographic proximity to the location of the customers served by the data center. The geographical proximity of a data center to the customers served by the data center facilitates more efficient use of resources and faster and more reliable service to the customers. Thus, a CSP typically builds a new data center in a new region in a geographic area that is geographically close to the customers served by the data center. For example, for a growing customer base in Germany, the CSP may build one or more data centers in a new region in Germany.

[0032] Building a data center (or multiple data centers) in a region may also be referred to as building a region. The term "region construction" is used to refer to building one or more data centers in a region. Building a data center in a region involves provisioning or creating a new set of resources that are needed or used to provide the set of services that the data center is configured to provide. The end result of the region construction process is the creation of a data center in the region that is capable of providing the set of services targeted to that data center and includes the set of resources used to provide the set of services.

[0033] Building a new data center in a region is a highly complex task that requires coordination between various teams. Broadly speaking, it requires the execution and coordination of various tasks, such as identifying the set of services to be provided by the data center, identifying the various resources required to provide the set of services, creating, provisioning and deploying the identified resources, and properly connecting the resources so that they can be used in the intended manner. Each of these tasks has further subtasks that require coordination, which further increases the complexity. Due to this complexity, currently, building a data center in a region involves several manual initiation or manual control tasks that require careful manual coordination. As a result, the task of building a new region (i.e., building one or more data centers in a region) is very time-consuming. The time to build a data center can take, for example, several months. Moreover, this process is highly error-prone and may require several re-runs before the desired configuration of the data center is achieved, thereby further increasing the time required to build the data center. These limitations and problems significantly limit the ability of CSPs to scale in a timely manner to meet growing customer needs.

[0034] This disclosure describes techniques for reducing the time and manual effort required to build one or more data centers in a region. This is made possible by automating some of the tasks involved in building a region. The automation significantly reduces the time required to build a data center in a region and reduces the manual coordination required. Using the techniques described herein, a new data center can be built in a region in a relatively much shorter time than the weeks and months required to build a data center in a traditional region.

[0035] Disclosed herein is a cloud infrastructure orchestration service (CIOS) configured to bootstrap (e.g., provision and deploy) services to a new datacenter based on predefined configuration files that identify resources (e.g., infrastructure components and software to be deployed) for implementing a given change to the datacenter. The CIOS can use static analysis of these configuration files to identify dependencies between bootstrap tasks. The CIOS can use these dependencies to coordinate the order in which various changes are made to the new datacenter (e.g., the order in which services are bootstrapped in the region). The CIOS can detect various capabilities of the region as they become available, which allows the system to identify and implement additional changes that can now be made to the region. Using the techniques disclosed herein, the CIOS can optimize parallel processing to execute changes to the new datacenter while ensuring that tasks are not initiated until the capabilities they depend on are available in the region. In this manner, the CIOS allows region construction to be performed as a substantially automated process, thereby significantly reducing the risk of error and time required in conventional systems.

[0036] During region construction, or at any suitable time, there may be any suitable number of resources (e.g., load balancers, databases, etc.) that may have been created during a previous region construction (e.g., construction of one or more data centers of the same or different region) or at least prior to the current region construction. CIOS ensures that previously created resources (e.g., resources from another region) can be utilized and used for bootstrapping the current region being constructed. A resource identification service (also referred to as a "resource hunter") can be configured to attempt to discover resources at any suitable time. In an embodiment, the functionality provided by the resource hunter can be triggered by a multi-flock orchestrator such that previously created resources (e.g., resources of a previously constructed region or data center) can be imported and used in the region being constructed. Resources may be created out-of-band with the region construction, but to avoid creating duplicate resources, these previously created resources can be automatically imported and used for the region construction. By utilizing these previously created resources, the techniques herein ensure that processing resources are not wasted on the creation of unnecessary resources. This further allows the region construction to occur faster than would be possible if all resources were created anew.

[0037] Certain definitions A "region" is a logical abstraction that corresponds to a geographic location. A region may include any suitable number of one or more execution targets. In an embodiment, an execution target may correspond to a data center.

[0038] An "Execution Target" is the smallest unit of change for executing a release. A "Release" is an expression of intent to orchestrate a particular change to a service (e.g., deploy version 8, "add internal DNS record", etc.). For most services, an execution target represents (is) an "instance" of the service. A single service can be bootstrapped onto one or more execution targets each. An execution target can be associated with a set of devices (e.g., a data center).

[0039] "Bootstrapping" is intended to refer to the collective tasks associated with the provisioning and deployment of any suitable number of resources (e.g., infrastructure components, artifacts, etc.) that correspond to a single service.

[0040] "Service" refers to functionality provided by a set of resources. The set of resources for a service includes any suitable combination of infrastructure, platform, or software (e.g., applications) hosted by a cloud provider that can be configured to provide the functionality of the service. Services can be provided to users over the Internet.

[0041] "Artifact" refers to infrastructure components or code that is deployed to a Kubernetes engine cluster, which may include software (e.g., applications), configuration information for infrastructure components (e.g., configuration files), etc.

[0042] "Flock config" refers to a configuration file (or set of configuration files) that describes the set of all resources (e.g., infrastructure components and artifacts) associated with a single service. A Flock config can contain declarative statements that specify one or more aspects that correspond to the desired state of the service's resources.

[0043] "Service State" refers to a point-in-time snapshot of all resources (e.g., infrastructure resources, artifacts, etc.) associated with a service. The service state indicates the situation corresponding to the provisioning and / or deployment tasks associated with the service resources.

[0044] IaaS provisioning (or "provisioning") refers to obtaining a computer or virtual host for use and installing the necessary libraries or services on it. The phrase "provisioning a device" refers to making a device available for use by an end user for a specific use. A device that has undergone the provisioning process may be called a "provisioned device." Preparing a provisioned device (installing libraries and daemons) may be part of provisioning. This preparation is distinct from deploying a new application or a new version of an application to a prepared device. In most cases, deployment does not include provisioning, which may need to be done first. After preparation, the device may be called an "infrastructure component."

[0045] IaaS deployment (or "deployment") refers to the process of providing and / or installing an application or a new version of an application onto provisioned infrastructure components. After the infrastructure components are provisioned (e.g., acquired, allocated, prepared, etc.), additional software can be deployed (e.g., provisioned and installed onto the infrastructure components). After provisioning and deployment are complete, the infrastructure components can be referred to as "resources." Examples of resources can include, but are not limited to, virtual machines, databases, object storage, block storage, load balancers, etc.

[0046] A "capability" identifies a unit of functionality associated with a service. A unit may be some or all of the functionality provided by a service. For example, a capability may be exposed that indicates that a resource is available for authorization / authentication processing (e.g., a subset of the functionality provided by the resource). As another example, a capability may be exposed that indicates that the full functionality of the service is available. Capabilities may be used to identify functionality on which a resource or service depends and / or functionality of the resource or service that is available for use.

[0047] A "virtual bootstrap environment" (ViBE) refers to a virtual cloud network that is provisioned in an overlay (e.g., a "host region") of an existing region. Once provisioned, the ViBE is connected to the new region using a communication channel (e.g., an IPSec tunnel VPN). Certain important core services (or "seed" services) can be provisioned in the ViBE, such as a deployment orchestrator, public key infrastructure (PKI) services, etc. These services can provide the capabilities required to bring the hardware online, establish a chain of trust to the new region, and deploy the remaining services in the new region. The use of a virtual bootstrap environment can prevent circular dependencies between bootstrap resources by using resources in the host region. Services can be staged and texted in the ViBE before the physical region (e.g., the target region) is available.

[0048] "Cloud Infrastructure Orchestration Services" (CIOS) may refer to a system configured to manage the provisioning and deployment operations of any suitable number of services as part of a region construction.

[0049] A Multi-Flock Orchestrator (MFO) can be a computing component (e.g., a service) configured to coordinate events among components of the CIOS to automatically provision and deploy services to a target region (e.g., a new region). The MFO tracks relevant events for each service in the region build and takes action in response to those events.

[0050] "Host Region" refers to a region that hosts a Virtual Bootstrap Environment (ViBE). A host region can be used to bootstrap a ViBE.

[0051] "Target region" refers to the region being constructed. "Exposing a capability" refers to providing an indication that a particular capability is available (or unavailable), through "publishing" as used in "publisher-subscriber" computing design, or otherwise. Capabilities are "exposed" (e.g., collected by a capability service, provided to a capability service, pushed, pulled, etc.) to provide an indication that functionality of a resource / service is available. In one embodiment, capabilities can be exposed / transmitted via events, notifications, data submissions, function calls, API calls, etc. An event (or other notification / data submission, etc.) indicating availability of a particular capability can be broadcast / addressed (e.g., published) to the capability service.

[0052] A "capability service" can be a configured flock that models dependencies between different flocks. Capability services can be provided within a cloud infrastructure orchestration service and can define what capabilities, services, and features are made available in a region.

[0053] A "Real-time Regional Data Distributor" (RRDD) can be a service or system configured to manage regional data that can be populated into a flock config to dynamically create execution targets for new regions.

[0054] Described herein, in certain examples, are techniques for implementing a cloud infrastructure orchestration service (CIOS). Such techniques can be configured to manage the bootstrapping (e.g., provisioning and deployment of software to infrastructure components) in a cloud environment (e.g., region), as briefly described above. In some cases, the CIOS can include computing components (e.g., CIOS central and CIOS regional, both of which are described in more detail below) that can be configured to manage the bootstrapping tasks (provisioning and deployment) for a given service and a multi-flock orchestrator (also described in more detail below) that is configured to initiate / manage a region build (e.g., a bootstrapping operation corresponding to multiple services).

[0055] CIOS enables region construction and worldwide infrastructure provisioning and code deployment with minimal manual run-time effort from service teams (e.g., beyond initial hardware approval and / or physical transport in some cases). High-level responsibilities of CIOS include, but are not limited to, coordinating region construction in an automated manner with minimal human intervention, providing users with a view of the current state of resources managed by CIOS (e.g., of a region, across multiple regions, worldwide, etc.), and managing bootstrapping operations for bootstrapping resources within a region.

[0056] CIOS can provide view reconciliation, where a view of a desired state of a resource (e.g., a desired configuration) can be matched with the current / actual state of the resource (e.g., a current configuration). In some cases, view reconciliation can include obtaining state data to determine what resources are actually running and the current configuration and / or state of the resource. Reconciliation can occur at various granularities, such as at the service level.

[0057] The CIOS can perform plan generation, where differences between desired and current states of resources are identified. Part of plan generation can include identifying actions that will need to be performed to bring resources from their current state to the desired state. Once the user is satisfied with the plan, the plan can then be marked as approved or rejected. Thus, because the plan is generated by a machine, the user can spend less time making judgments about the plan, and the plan will be more accurate. Although the plan is too detailed for human use, the CIOS can provide this data via an advanced user interface (UI).

[0058] In one example, the CIOS can handle change control execution by automatically executing approved plans. Once an execution plan has been created and approved, engineers may not need to be involved in change control unless the CIOS initiates a rollback. The CIOS can handle rollbacks to a previous service version by automatically generating a plan to revert the service to a previous (e.g., pre-release) state (e.g., if the CIOS detects a deterioration in the service health during execution).

[0059] CIOS can measure service health by monitoring alarms and running integration tests. CIOS can help teams quickly define rollback actions in case of service degradation, which CIOS can execute automatically. CIOS can automatically generate and display plans, and can track approvals. CIOS can combine provisioning and deployment functions into a single system that coordinates these tasks across region builds. CIOS also supports automatic discovery of flocks (e.g., service resources such as flock configs corresponding to any suitable number of services), artifacts, resources, and dependencies. CIOS can discover dependencies between execution tasks at any level (e.g., resource level, execution target level, phase level, service level, etc.) by static analysis (e.g., including content parsing and processing) of one or more configuration files. Using these dependencies, CIOS can generate various data structures that can be used to drive task execution (e.g., tasks related to infrastructure resource provisioning and artifact deployment across regions) from these dependencies.

[0060] FIG. 1 is a block diagram of an environment 100 in which a cloud infrastructure orchestration (CIOS) 102 can operate to dynamically provide bootstrap services in a region, according to at least one embodiment. CIOS 102 can include, but is not limited to, the following components: a real-time regional data distributor (RRDD) 104, a multi-flock orchestrator (MFO) 106, a CIOS central 108, a CIOS regional 110, and a capability service 112. The specific functions of CIOS central 108 and CIOS regional 110 are described in more detail in U.S. patent application Ser. No. 17 / 016,754, entitled "Techniques for Deploying Infrastructure Resources with a Declarative Provisioning Tool," which is incorporated in its entirety for all purposes. In an embodiment, any suitable combination of the components of CIOS 102 can be provided as a service. In an embodiment, a portion of CIOS 102 can be deployed to a region (e.g., a data center represented by host region 103). In one embodiment, CIOS 102 may include any suitable number of cloud services (not shown in FIG. 1), as described in more detail in U.S. patent application Ser. No. 17 / 016,754 and with respect to FIGS. 2 and 3 below.

[0061] The real-time regional data distributor (RRDD) 104 can be configured to maintain and provide regional data that identifies realms, regions, execution targets, and availability domains. In some cases, the regional data can be in any suitable form (e.g., JSON format, data object / container, XML, etc.). The regional data maintained by the RRDD 104 can include any suitable number of subsets of data that can be individually referenced by a corresponding identifier. For example, an identifier "all_regions" can be associated with a data structure (e.g., a list, structure, object, etc.) that includes metadata for all defined regions. As another example, an identifier such as "realms" can be associated with a data structure that identifies metadata for a number of realms and a set of regions corresponding to each realm. In general, the regional data can maintain any suitable attributes, such as identifiers, DNS suffixes, state (e.g., region state), etc., for one or more realms, regions, availability domains (ADs), execution targets (ETs), etc. The RRDD 104 can be configured to manage regional state as part of the regional data. The regional state can include any suitable information indicative of a state of bootstrap in the region. For example, some example region states may include "initial," "under construction," "production," "suspended," or "to be decommissioned." The "initial" state may indicate a region that has not yet been bootstrapped. The "under construction" state may indicate that bootstrap of one or more flocks in the region has begun. The "production" state may indicate that bootstrap is complete and the region is ready for validation. The "suspended" state may indicate that CIOS Central 108 or CIOS Regional 110 has suspended internal interaction with the regional stack, possibly due to operational issues. The "decommissioned" state may indicate that the region has been decommissioned and is likely unavailable and / or will not be contacted again in the future.

[0062] CIOS central 108 may be configured to provide any suitable number of user interfaces through which a user (e.g., user 109) may interact with CIOS 102. For example, a user may make changes to region data through a user interface provided by CIOS central 108. CIOS central 108 may further provide various interfaces that allow a user to view changes made to flock configurations and / or artifacts, generate and view plans, approve / reject plans, and view the status of plan execution (e.g., corresponding to tasks involving infrastructure provisioning, deployment, region construction, and / or the desired state of any suitable number of resources managed by CIOS 102). CIOS central 108 may implement a control plane that may be configured to manage any suitable number of CIOS regional 110 instances. CIOS central 108 may provide one or more interfaces for presenting region data, thereby allowing user 109 to view and / or modify the region data. CIOS central 108 may be configured to invoke functions of RRDD 104 through any suitable number of interfaces. In general, CIOS central 108 can be configured to manage region data, either directly or indirectly (e.g., via RRDD 104). CIOS central 108 can be configured to edit the flock config to populate the region data as variables in the flock config.

[0063] Each instance of CIOS regional 110 may correspond to a module configured to perform bootstrapping tasks associated with a single service of the region. CIOS regional 110 may receive desired state data from CIOS central 108. In one embodiment, the desired state data may include a flock config that declares (e.g., by declarative statements) a desired state of the resources associated with the service. CIOS central 108 may maintain current state data that indicates any suitable aspect of the current state of the resources associated with the service. In one embodiment, CIOS regional 110 may identify that changes need to be made to one or more resources by comparing the desired state data to the current state data. For example, CIOS regional 110 may determine that one or more infrastructure components need to be provisioned, one or more artifacts need to be deployed, or any suitable changes need to be made to the resources of the service to match the desired state. When CIOS regional 110 performs the bootstrapping operation, it may expose data that indicates various capabilities of the resources as the capabilities become available. A "capability" identifies a unit of functionality associated with a service. A unit may be some or all of the functionality provided by a service. For example, a capability may be exposed to indicate that a resource is available for authorization / authentication processing (e.g., a subset of the functionality provided by the resource). As another example, a capability may be exposed to indicate that the full functionality of a service is available. Capabilities may be used to identify functionality on which a resource or service depends and / or functionality of the resource or service that is available for use.

[0064] The capability service 112 is configured to maintain capability data that indicates: 1) what capabilities of various services are currently available; 2) whether any resources / services are waiting for a particular capability; 3) what particular resources and / or services are waiting for a given capability; or any suitable combination of the above. The capability service 112 may provide an interface through which the capability data may be requested. The capability service 112 may provide one or more interfaces (e.g., application programming interfaces) that enable the capability service 112 to send capability data to the MFO 106 and / or the CIOS regional 110 (e.g., each instance of the CIOS regional 110). In an embodiment, any suitable component or module of the MFO 106 and / or the CIOS regional 110 may be configured to request the capability data from the capability service 112.

[0065] In one embodiment, a multi-flock orchestrator (MFO) 106 can be configured to drive region construction activities. In one embodiment, the MFO 106 can manage information describing what flocks / flock config versions and / or artifact versions to use to bootstrap a given service in a region (or to make a unit of change to a target region). In one embodiment, the MFO 106 can be configured to monitor (or otherwise be notified of) changes to the region data managed by the real-time regional data distributor 104. In one embodiment, region construction may be triggered by the MFO 106 upon receiving an indication that the region data has changed. In one embodiment, the MFO 106 may collect various flock configs and artifacts used in region construction. Some or all of the flock configs can be configured to be region agnostic. That is, the flock configs may not explicitly specify which flocks are to be bootstrapped into which regions. In one embodiment, the MFO 106 can trigger a data population process whereby the collected flock configs are recompiled (e.g., by the CIOS Central 108). During recompilation, operations may be performed (e.g., by the CIOS Central 108) to cause the region data maintained by the real-time regional data distributor 104 to be populated into the configuration files. The flock configs can reference the region data by variables / parameters, without requiring a hard-coded identity of the region data. The flock configs can use this data population to be dynamically modified at run time, without having the region data hard-coded and therefore more difficult to change.

[0066] The multi-flock orchestrator 106 may perform static flock analysis, in which the flock config is parsed to identify dependencies between resources, execution targets, phases, and flocks, specifically to identify circular dependencies that need to be removed. In an embodiment, the MFO 106 may generate any suitable number of data structures based on the identified dependencies. These data structures (e.g., directed acyclic graphs, linked lists, etc.) may be used by the cloud infrastructure orchestration service 102 to drive operations to perform region construction. For example, these data structures may collectively define the order in which services are bootstrapped within a region. Examples of such data structures are discussed in more detail below in connection with the construction dependency graph 338 of FIG. 3. If circular dependencies (e.g., service A requires service B and vice versa) exist and are identified by the static flock analysis and / or graph, the MFO may be configured to notify any appropriate service teams that changes need to be made to the corresponding flock config to correct these circular dependencies. The MFO 106 may be configured to traverse one or more data structures to manage the order in which services are bootstrapped into regions. The MFO 106 can identify capabilities available within a given region at any given time (e.g., using data obtained from the capabilities service 112). The MFO 106 can use this data to identify when the MFO 106 can bootstrap a service, when bootstrap is blocked, and / or when a bootstrap operation associated with a previously blocked service can be resumed. Based on this scanning, the MFO 106 can perform various releases in which instructions are sent by the MFO 106 to the CIOS central 108 to perform bootstrap operations corresponding to any suitable number of block configurations.In one example, the MFO 106 can be configured to identify that one or more flock configs may require multiple releases due to circular dependencies found in the graph, and as a result, the MFO 106 can send multiple sets of instructions to the CIOS central 108 for a given flock config to break the circular dependencies identified in the graph.

[0067] In one embodiment, a user may request that a new region (e.g., target region 114) be built. This may involve bootstrapping resources corresponding to various services. In one embodiment, the target region 114 may not be available for communication (and / or may not be secure) at the time the region build request is issued. Rather than deferring bootstrapping until the target region 114 is available and configured to perform the bootstrapping operation, CIOS 102 may initiate region build using a virtual bootstrap environment 116. The virtual bootstrap environment (ViBE) 116 may be an overlay network hosted by the host region 103 (an existing region that has previously been configured with a core set of services, is available for communication, and is secure). The MFO 106 may utilize the resources of the host region 103 to bootstrap resources to the ViBE 116 (commonly referred to as "building the ViBE"). For example, MFO 106 can provide instructions via CIOS central 108 to cause an instance of CIOS regional 110 in a host region (e.g., host region 103) to bootstrap another instance of CIOS regional in ViBE 116. After the CIOS regional in ViBE is available for processing, bootstrapping of services for the target region 114 can continue in ViBE 116. Services previously bootstrapped in ViBE 116 can be migrated to the target region 114 when the target region 114 is available to perform the bootstrap operation. Using these techniques, CIOS 102 can greatly increase the speed at which regions are built by greatly reducing the need for manual input and / or configuration to be provided.

[0068] 2 is a block diagram illustrating an environment 200 and method for building a virtual bootstrap environment (ViBE) 202 (an example of ViBE 116 of FIG. 1) according to at least one embodiment. ViBE 202 represents a virtual cloud network that is provisioned in an overlay of an existing region (e.g., hosted region 204, which is an example of hosted region 103 of FIG. 1 and in one embodiment is a hosted region service enclave). ViBE 202 represents an environment in which services can be staged for a target region (e.g., a region under construction, such as target region 114 of FIG. 1) before the target region is available.

[0069] To bootstrap a new region (e.g., target region 114 in FIG. 1), a core set of services can be bootstrapped. These core set of services exist in the host region 204 but do not yet exist in ViBE (nor in the target region). These basic core services provide the functionality required for provisioning devices, establishing a chain of trust to the new region, and deploying the remaining services (e.g., flocks) to the region. ViBE 202 may be a tenancy that is deployed in the host region 204. This can be thought of as a virtual region.

[0070] When the target region is available to provide the bootstrap operations, ViBE 202 can be connected to the target region such that services in the ViBE can interact with services and / or infrastructure components of the target region. This allows for the deployment of production-level services instead of a self-contained seed service as in previous systems, which would require connectivity over the Internet to the target region. Traditionally, a seed service would be deployed as part of a container collection and used to bootstrap the dependencies required to build the region. Using the existing region's infrastructure / tooling, resources can be bootstrapped (e.g., provisioned and deployed) to ViBE 202 until the target region is self-sufficient and can communicate directly, and it can connect to the service enclaves of the region (e.g., host region 204) to provision hardware and deploy services. The use of ViBE 202 allows for the maintenance of dependencies and services required to be able to provision / prepare infrastructure and deploy software while utilizing the host region's resources to break circular dependencies for core services.

[0071] A multi-flock orchestrator (MFO) 206 can be configured to perform operations to build (e.g., configure) ViBE 202. The MFO 206 can obtain appropriate flock configs corresponding to the various resources to be bootstrapped into a new region (in this case, the ViBE region, ViBE 202). For example, the MFO 206 can obtain a flock config (e.g., a "ViBE flock config") that specifies aspects of the bootstrap of capability services 208 and workers 210. As another example, the MFO 206 can obtain another flock config corresponding to the bootstrap of domain name services (DNS) 212 into ViBE 202.

[0072] In step 1, MFO 206 can instruct CIOS central 214 (e.g., an example of CIOS central 108 and CIOS central 214 of FIGS. 1 and 2, respectively). For example, MFO 206 can send a request (e.g., including a ViBE flock config) to request bootstrap of capability services 208 and workers 210 that do not yet exist in ViBE 202 at this point. In one embodiment, CIOS 214 has access to all flock configs. Thus, in one example, MFO 206 can send an identifier for a ViBE flock config rather than the file itself, and CIOS central 214 can retrieve it independently from storage (e.g., from DB 308 or Flock DB 312 of FIG. 3).

[0073] In step 2, CIOS central 214 can provide the ViBE Flock Config via a corresponding request to CIOS regional 216. CIOS regional 216 can parse the ViBE Flock Config to identify and execute specific infrastructure provisioning and deployment operations in step 3.

[0074] In one embodiment, CIOS regional 216 can use additional corresponding services for provisioning and deployment. For example, in step 4, CIOS regional 216 can instruct deployment orchestrator 218 (e.g., an example of a core service or other creation, build and deployment application software in host region 204) to execute instructions to bootstrap capability service 208 and worker 210 within ViBE 202.

[0075] In step 5, a capability may be sent to capability service 208 (from CIOS regional 216, deployment orchestrator 218 via worker 210, or otherwise) indicating that resources corresponding to the ViBE block are available. Capability service 208 may maintain this data. In one embodiment, capability service 208 adds this information to a list of available capabilities that capability service 208 maintains with ViBE. For example, the capability provided to capability service 208 in step 5 may indicate that capability service 208 and worker 210 are available for processing.

[0076] In step 6, the MFO 206 can determine that the capability service 208 and the worker 210 are available based on receiving or obtaining data (identifiers corresponding to capabilities) from the capability service 208.

[0077] In step 7, as a result of receiving / obtaining the data in step 6, MFO 206 can instruct CIOS Central 214 to bootstrap a DNS service (e.g., DNS 212) into ViBE 202. These instructions can specify or include a particular block config that corresponds to the DNS service.

[0078] In step 8, CIOS Central 214 can instruct CIOS Regional 216 to deploy DNS 212 to ViBe 202. In one embodiment, the DNS block config for DNS 212 is provided by CIOS Central 214.

[0079] In step 9, now that worker 210 is deployed to ViBE 202, worker 210 can be assigned by CIOS regional 216 to the task of deploying DNS 212. The worker can run a declarative infrastructure provisioner in the manner described above in connection with FIG. 3 to identify the set of operations that need to be performed to deploy DNS 212 (e.g., by comparing the flock config (desired state) with the current state of the resources associated with the flock (which currently do not exist)).

[0080] At step 10, deployment orchestrator 218 instructs worker 210 to deploy DNS 212 according to the actions identified at step 9. As shown, worker 210 proceeds to perform the actions to deploy DNS 212 to ViBE 202 at step 11. At step 12, worker 210 notifies capability service 208 that DNS 212 is available on ViBE 202. MFO 206 then determines that resources associated with the ViBE flock config and the DNS flock config are available, and any suitable number of additional resources can subsequently be bootstrapped into ViBE.

[0081] After steps 1 through 12 are completed, the process for building ViBE202 may be considered complete, and ViBE202 may be considered built.

[0082] FIG. 3 is a block diagram illustrating an environment 300 and method for bootstrapping a service into a target region using ViBE, according to at least one embodiment.

[0083] In step 1, a user 302 can use any suitable user interface provided by CIOS central 304 (e.g., an example of CIOS central 108 and CIOS central 214 of FIGS. 1 and 2, respectively) to modify region data. For example, a user 302 can create a new region into which several services are bootstrapped.

[0084] In step 2, CIOS central 304 may perform an operation to send the changes to RRDD 306 (e.g., an example of RRDD 104 in FIG. 1). In step 3, RRDD 306 may store the received region data in database 308, which is a data store configured to store region data including any suitable identifiers, attributes, states, etc., such as region, AD, realm, ET, etc. In one embodiment, updater 307 may be used to store the region data in database 308 or any suitable data store from which the updates may be accessible (by a service team). In one embodiment, updater 307 may be configured to notify (e.g., by any suitable electronic notification) of updates made to database 308.

[0085] In step 4, the MFO 310 (e.g., an example of the MFO 106 and MFO 206 of FIGS. 1 and 2, respectively) may detect the change in the region data. In one embodiment, the MFO 310 may be configured to poll the RRDD 306 for changes in the region data. In one embodiment, the RRDD 306 may be configured to publish or otherwise notify the MFO 310 of the region change.

[0086] In step 5, detection of a change in region data can trigger MFO 310 to retrieve a version set (e.g., a version set associated with a particular identifier, such as a "golden version set" identifier) ​​that identifies the specific version of each flock (e.g., service) to be bootstrapped into the new region and the specific version of each artifact corresponding to that flock. The version set can be retrieved from DB 312. As a flock evolves and changes, the corresponding config and artifact versions used for region construction can change. These changes can be maintained in flock DB 312 so that MFO 310 can identify which versions of flock config and artifacts to use for construction of the region (e.g., ViBE region, target region / non-ViBE region, etc.). Flock configs (e.g., all versions of flock configs) and / or artifacts (e.g., all versions of artifacts) can be stored in DB 308, DB 312, or any suitable data store accessible to CIOS Central 304 and / or MFO 310.

[0087] In step 6, MFO 310 can request CIOS Central 304 to recompile each of the flock configs associated with the version set with the current region data. In one embodiment, the request can indicate the version of each flock config and / or artifacts that correspond to those flock configs.

[0088] In step 7, CIOS Central 304 can obtain the current regional data from DB 308 (e.g., directly or via real-time regional data distributor 306) and retrieve any appropriate flock configurations and artifacts according to the version requested by MFO 310.

[0089] In step 8, CIOS central 304 can re-edit the flock config with the region data obtained in step 7 to populate the flock config with the current region data. CIOS central 304 can return the edited flock config to MFO 310. In one embodiment, CIOS central 304 can simply indicate that the edit is complete, and MFO 310 can access the re-edited flock config via RRDD 306.

[0090] In step 9, the MFO 310 may perform a static analysis of the recompiled flock config. As part of the static analysis, the MFO 310 may parse the flock config (e.g., using libraries associated with a declarative infrastructure provisioner (e.g., Terraform, etc.)) to identify dependencies between flocks. From this analysis and the identified dependencies, the MFO 310 may generate a build dependency graph 338. The build dependency graph 338 may be an acyclic directed graph that specifies the order in which flocks are bootstrapped (and / or changes indicated in the flock config are applied) into new regions. Each node in the graph may correspond to the bootstrap of any appropriate portion of a particular flock. A particular bootstrap order may be specified based at least in part on the dependencies. In an embodiment, the dependencies may be represented as attributes of the nodes and / or may be indicated by the edges of the graph connecting the nodes. The MFO 310 may traverse the graph (e.g., starting from the origin node) to drive the operation of region construction.

[0091] In an embodiment, MFO 310 can use a cycle detection algorithm to detect the presence of a cycle (e.g., service A depends on service B and vice versa). MFO 310 can identify orphan capability dependencies. For example, MFO 310 can identify orphan nodes in construction dependency graph 338 that do not connect to any other nodes. MFO 310 can identify capabilities that were published in error (e.g., when a capability was published prematurely and the corresponding functionality is not actually available yet). MFO 310 can detect from the graph that there are one or more instances of publication of the same capability. In an embodiment, any suitable number of such errors can be detected, and MFO 310 (or another suitable component, such as CIOS Central 304) can be configured to notify or otherwise present this information to a user (e.g., via an electronic notification, a user interface, etc.). In one embodiment, MFO 310 may be configured to force delete / recreate resources to break circular dependencies and may redirect CIOS Central 304 to perform bootstrap operations for those resources and / or the corresponding block configs.

[0092] The starting node may correspond to bootstrapping the ViBE flock, and the second node may correspond to bootstrapping the DNS. Steps 10-15 correspond to deployment (by deployment orchestrator 317, which is an example of deployment orchestrator 218 in FIG. 2) of the ViBE flock to ViBE 316 (e.g., an example of ViBE 116 and ViBE 202 in FIG. 1 and FIG. 2, respectively). That is, steps 10-15 of FIG. 3 generally correspond to steps 1-6 of FIG. 2. After being notified that capabilities exist corresponding to the ViBE flock to be deployed (e.g., indicating that capability service 318 and worker 320, which correspond to capability service 208 and worker 210 in FIG. 2, are available), MFO 310 resumes traversing build dependency graph 338 to identify the next operation to perform.

[0093] For example, MFO 310 may continue traversing construction dependency graph 338 to identify that a DNS block should be deployed. Steps 16-21 may be performed to deploy DNS 322 (an example of DNS 212 in FIG. 2). These operations may generally correspond to steps 7-12 in FIG. 2.

[0094] At step 21, a capability indicating that DNS 322 is available may be stored. Upon detecting this capability, MFO 310 may resume traversing build dependency graph 338. During this traversal, MFO 310 may identify that any appropriate portions of an instance of CIOS Regional (e.g., an instance of CIOS Regional 314) should be deployed to ViBE 316. In one embodiment, steps 16-21 may be substantially repeated with respect to the deployment of CIOS Regional (ViBE) 326 (an instance of CIOS Regional 314, CIOS Regional 110 in FIG. 1) and Worker 328 to ViBE 316. A capability that CIOS Regional (ViBE) 326 is available may be sent to capability service 318.

[0095] Upon detecting that CIOS Regional (ViBE) 326 is available, MFO 310 may resume traversing build dependency graph 338. During this traversal, MFO 310 may identify that a deployment orchestrator (e.g., deployment orchestrator 330, which is an example of deployment orchestrator 317) is deployed to ViBE 316. In one embodiment, steps 16-21 may be substantially repeated for the deployment of deployment orchestrator 330. Information may be sent to capability service 318 identifying capabilities indicating that deployment orchestrator 330 is available.

[0096] After the deployment orchestrator 330 is deployed, ViBE 316 can be considered available to process subsequent requests. Upon detecting that the deployment orchestrator 330 is available, MFO 310 can instruct that subsequent bootstrap requests are routed to the ViBE component without using the host region component (the component of the host region 332). Thus, MFO 310 can continue to traverse the build dependency graph 338 and at each node, instruct flock deployment to ViBE 316 via CIOS Central 304. CIOS Central 304 can request CIOS Regional (ViBE) 326 to deploy resources according to the flock configuration.

[0097] At some point during this process, the target region 334 may become available. An indication that the target region is available may be identifiable from region data for the target region 334 provided by the user 302 (e.g., as an update to the region data). The availability of the target region 334 may depend on the establishment of a network connection between the target region 334 and an external network (e.g., the Internet). The network connection may be supported over a public network (e.g., the Internet), but software security measures (e.g., IPSec) may be used to provide one or more encrypted tunnels (e.g., IPSec tunnels such as tunnel 336) from ViBE 316 to the target region 334. As used herein, "IPSec" refers to a protocol suite for authentication and encryption of network traffic over networks using the Internet Protocol (IP) and may include one or more available implementations of the protocol suite (e.g., Openswan, Libreswan, strongSwan, etc.). The network may connect ViBE 316 to a service enclave of the target region 334.

[0098] Prior to establishing the IPSec tunnel, the initial network connectivity to the target region 334 may be on a sufficient connection (e.g., an out-of-band VPN tunnel) to allow bootstrapping of networking services until an IPSec gateway can be deployed on an asset (e.g., a bare metal asset) in the target region 334. To bootstrap the network resources of the target region 334, the deployment orchestrator 330 can deploy an IPSec gateway on the asset in the target region 334. The deployment orchestrator 330 can then deploy a VPN host in the target region 334 configured to terminate the IPSec tunnel from ViBE 316. After a service in ViBE 316 (e.g., deployment orchestrator 330, service A, etc.) can establish an IPsec connection with a VPN host in the target region 334, the bootstrapping operation from ViBE 316 to the target region 334 can begin.

[0099] In an embodiment, the bootstrap operation may begin with a service in ViBE 316 provisioning resources in the target region 334 to support hosting instances of core services when deployed from ViBE 316. For example, a host provisioning service may provision a hypervisor on infrastructure (e.g., bare metal hosts) in the target region 334 to allocate computing resources to VMs. Once the host provisioning service completes the allocation of physical resources in the target region 334, the host provisioning service may publish information indicating a capability indicating that physical resources have been allocated in the target region 334. The capability may be published to the capability service 318 (e.g., by worker 328) via CIOS regional (ViBE) 326.

[0100] With the hardware allocation for the target region 334 established and notified to the capability service 318, CIOS regional (ViBE) 326 can orchestrate the deployment of instances of core services from ViBE 316 to the target region 334. This deployment may be similar to the process described above for building ViBE 316, but using ViBE components (e.g., CIOS regional (ViBE) 326, worker 328, deployment orchestrator 330) instead of host region 332 service enclave components. The deployment operations may generally correspond to steps 16-21 described above.

[0101] When a service is deployed from ViBE 316 to a target region 334, a DNS record associated with the service may correspond to an instance of the service in ViBE 316. The DNS record associated with the service can be updated later to complete the deployment of the service to the target region 334. In other words, the instance of the service in ViBE 316 can continue to receive traffic (e.g., requests) to the service until the DNS record is updated. The service can be partially deployed to the target region 334 and can publish information (e.g., to the capability service 318) indicating a capability that the service is partially deployed. For example, a service running in ViBE 316 can be deployed to the target region 334 along with corresponding compute instances, load balancers, and associated applications and other software, but may need to wait for database data to migrate to the target region 334 before being fully deployed. The DNS record (e.g., managed by DNS 322) can still be associated with the service in ViBE 316. Once the data migration for the service is complete, the DNS record can be updated to point to the operational service deployed in the target region 334. The deployed service in target region 334 can then receive traffic (eg, requests) for that service, while the instance of the service in ViBE 316 will no longer be able to receive traffic for that service.

[0102] Resource Discovery FIG. 4 is a block diagram of an environment 400 in which a cloud infrastructure orchestration service (CIOS) can use resource hunters (e.g., resource hunter 420) to discover resources (e.g., on-demand, at region build time, etc.) according to at least one embodiment. The environment 400 may be an example of the environment 100 of FIG. 1. The cloud infrastructure orchestration service (CIOS) 402 may be an example of the CIOS 102 of FIG. 1. The real-time regional data distributor (RRDD) 404 may be an example of the real-time regional data distributor 104 and / or 306 of FIGs. 1 and 3, respectively. The multi-flock orchestrator (MFO) 406 may be an example of the multi-flock orchestrator 104, 206 and / or 310 of FIGs. 1-3, respectively. The CIOS central 408 (also referred to as a "provisioning and deployment service") may be an example of the CIOS central 108, 214 and / or 304 of FIGs. 1-3, respectively. CIOS regional 410 may be an example of CIOS regional 110, 216 and / or 314 of Figures 1-3, respectively. Capability service 412 may be an example of capability service 112, 208 and / or 318 of Figures 1-3, respectively. Target region 414 may be an example of target region 114 and / or 334 of Figures 1 and 3, respectively. Virtual bootstrap environment 416 may be an example of virtual bootstrap environment 114, 202 and / or 316 of Figures 1-3, respectively. Components of CIOS 402, including RRDD 404, MFO 406, CIOS central 508, CIOS regional 410, and capability service 412, may each perform the respective functions described above in Figures 1-3 with respect to the corresponding components of Figures 1-3.

[0103] Environment 500 may include resource hunter (RH) 420 (also referred to as a "resource identification service"). RH 420 may be configured to attempt to discover resources at any suitable time. For example, any suitable number of resources in any suitable region may exist prior to region construction of that region or another region. For example, resources 422 may include any suitable number of resources (e.g., infrastructure resources, artifacts, configuration files, etc.) that exist in host region 403 (an example of host region 103, 204, and / or 332, respectively, of FIGS. 1-3). Resources 424 may exist in ViBE 416 and / or resources 426 may exist in the target region. Any suitable combination of resources 422-426 may be created at any suitable time before, during, or after the region construction process to bootstrap any suitable number of services within ViBE 416 and / or target region 414.

[0104] RH 420 can be configured to receive flock configuration information from MFO 406 (and / or any suitable components of CIOS 402 and / or any services bootstrapped within host region 403, ViBE 416 and / or target region 414). In one embodiment, MFO 406 can provide an identifier for a flock, and RH 420 can be configured to access a corresponding flock configuration file associated with that flock. Alternatively, MFO 406 can provide the flock configuration to RH 420. An example of a flock configuration is described in more detail below in connection with FIG. 5.

[0105] RH 420 may identify resource discovery data within a flock config. "Resource discovery data" refers to any suitable data (set of parameters) that may be used to identify an existing resource (e.g., one or more of resources 422-426). RH 420 may perform operations to identify any suitable number of resources 422-426 using the set of parameters of the resource discovery data. For example, RH 420 may search (within a region specified by the resource discovery data and / or globally) for a particular resource associated with attributes that match the set of parameters provided in the resource discovery data. If one or more matching resources are found, RH 420 may provide an identifier of the matching resource directly to MFO 406 and / or RH 420 may store the identifier in a record accessible to MFO 406.

[0106] The MFO 406 can include a state manager 428. The state manager 428 can be configured to implement a state machine that transitions between states to drive paths. For example, a single flock may require multiple releases (e.g., multiple submissions of instructions to the CIOS central 408, multiple releases corresponding to one or more flock configurations associated with a single service, etc.). The state manager 428 can be configured to coordinate the operations performed for a single release. This may include monitoring messages from the CIOS central 408 and / or the resource hunter 420. In an embodiment, the state manager 428 can be configured to monitor for one or more capabilities submitted by the capability service 412. The operations performed by the state manager 428 can be further described in connection with FIG. 6 and FIG. 7.

[0107] FIG. 5 illustrates an example flock configuration file 500 including one or more code segments related to resource discovery (e.g., code segment 502 and code segment 504) according to at least one embodiment. It should be appreciated that flock configuration file 500 (or other flock configuration files corresponding to the same service) may further include additional code segments describing the resources (e.g., infrastructure components, artifacts, data, etc.) of the service, several phases corresponding to bootstrapping of the service across respective sets of execution targets corresponding to each phase, several execution targets to which an instance of the service is bootstrapped, etc. In an embodiment, flock configuration file 500 (or other flock configuration files corresponding to the same service) may include information from which one or more capabilities on which the service depends may be identified. In an embodiment, flock configuration file 500 (or other flock configuration files corresponding to the same service) may indicate one or more capabilities to be posted upon bootstrapping of the resources of the flock config. As a non-limiting example, flock configuration file 500 may be one of a set of configuration files corresponding to the same flock. A flock config corresponding to a flock may declaratively describe a desired state of one or more infrastructure resources. Another flock config corresponding to that flock may declaratively describe a desired state of one or more artifacts (e.g., software applications). Another flock config (e.g., flock config 500) may specify resource discovery data. The phase and / or execution target to which the flock is applied may be specified in any appropriate flock config described above. In the example shown in FIG. 5, references to flock config 500 may be interpreted as referring to any appropriate flock config in a set of flock configs when the set of flock configs is used to describe resources for the flock.

[0108] In an embodiment, code segment 502 may include resource discovery data usable to identify one or more existing resources. For example, line 506 may indicate that data corresponding to a resource of type "dataType" and name "Resource_Name" is to be searched. The type "dataType" and name "Resource_Name" may be considered parameters of the resource discovery data defined by code segment 502. Lines 507-508 may each include additional parameters used to perform the search. For example, line 507 may include a parameter "ad" that may populate (e.g., via a locally accessible array or list) the names of each availability domain in the region as shown. Line 508 may include a parameter "compartment_id" that may populate with identifiers corresponding to all compartments in the region as shown. Line 509 may define a string value (e.g., "EXAMPLE_RESOURCE") that is usable locally within flock config 500. Users of the string values ​​defined in line 509 are discussed in more detail below.

[0109] Because lines 507 and 508 use local parameters (e.g., as indicated by the use of "local," respectively), in one embodiment, regional data values ​​corresponding to the names of all availability domains in a given region and all compartment identifiers in the region may be populated into parameters "ad" and "compartment_id," respectively. The process by which regional data values ​​are populated into local parameters is described in more detail in connection with U.S. Provisional Patent Application No. 63 / 315,024, entitled "DATA MANAGEMENT TECHNIQUES FOR CLOUD REGIONS," filed February 28, 2022 (Attorney Docket No. 088325-1282326 (306000US)), the contents of which are incorporated herein in their entirety for all purposes. In the example shown in FIG. 5, the set of parameters may include a type "dataType," a name "Resource_Name," an availability domain name for the region, and compartment identifiers for all compartments in the region.

[0110] Based at least in part on code segment 502, RH 420 may search all availability domains in all compartments of that availability domain for a resource associated with data type "dataType" and name "Resource_Name." The parameter "data," if found, may be used to access the identified resource.

[0111] Code segment 504 provides an import operation that is performed using resources identified using the parameters of code segment 502, as shown. RH 420 can be configured to perform any appropriate operation defined by a declaration (e.g., output "default imports") such as shown in line 510. In the example shown in code segment 504, at line 512, RH 420 can obtain the value of each resource identified as matching the set of parameters provided in code segment 502. The string "EXAMPLE_RESOURCE" can be used as a string to replace a portion of line 510 (e.g., "example_resource") with "EXAMPLE_RESOURCE" as shown in line 511. In this manner, a global variable (e.g., a string, an integer, etc.) can be defined in code segment 502 and then used by any appropriate portion of flock config 500 (e.g., code segment 504) as shown at 511, 513, and 515.

[0112] For each resource, the value parameter in line 512 may be set to a value corresponding to the data EXAMPLE_RESOURCE. instance. The value of one resource may be used to perform the operations of lines 514 and 516. Line 514 may be executed to obtain the address (e.g., IP address) of the resource. Line 516 may be executed to obtain an identifier for the resource. After executing lines 514 and 516 for each resource identified, RH 420 may have a list of addresses and / or identifiers of all resources found to have attributes that match the set of parameters in code segment 502. RH 420 may store these addresses / identifiers in a record accessible to the MFO of FIGS. 1-4, or RH 420 may provide this data directly to the MFO. Any appropriate data obtained by the import operation may be similarly stored and / or provided to the MFO. This example of addresses and identifiers is not intended to limit the disclosure.

[0113] FIG. 6 is a block diagram illustrating an example method 600 for identifying resources corresponding to a region build, according to at least one embodiment. MFO 602 may be an example of multi-flock orchestrator 104, 206, 310 and / or 404 of FIGS. 1-4, respectively. CIOS central 604 may be an example of CIOS central 108, 214, 304 and / or 404 of FIGS. 1-4, respectively. Resource hunter 606 may be an example of resource hunter 420 of FIG. 4. At least some of the operations performed by MFO 602 may be performed by state manager 428 of FIG. 4.

[0114] Prior to execution of method 600, operations may have been performed to select flock configs, populate those configs with real-time region data, perform static flock analysis, and generate a build dependency graph (e.g., build dependency graph 338 of FIG. 3). Referring to FIG. 3, in steps 1-4, user 302 may use any suitable interface hosted by CIOS Central 604 to update region data (e.g., region data for ViBE 316 may indicate that ViBE 316 is a ViBE region to indicate a new region to add data associated with ViBE 316 of FIG. 3). Operations described in connection with steps 1-4 of FIG. 3 may allow MFO 602 to be notified and / or detect changes in region data. As described in connection with step 5 of FIG. 3, MFO 602 may obtain a set of flock configs (e.g., a set of one or more flock configs for each service). The set of flock configs may be selected based at least in part on golden version set data maintained by MFO 602. The MFO 602 may request that the CIOS Central 604 recompile the selected flock, as described in connection with steps 6-8 of FIGURE 3. During this recompile, current regional data may be obtained from the real-time regional data distributor 306 and populated into the parameters of the flock config. The MFO 602 may then perform a static analysis of the flock config recompile to generate a build dependency graph 338, as described in connection with step 9 of FIGURE 3.

[0115] The method 600 begins at 610 where the MFO 602 (eg, state manager 428 of FIG. 4) can detect the existence of the construction dependency graph 338 and can begin state management for one pass of the construction dependency graph 338.

[0116] 7 illustrates an example flow 700 illustrating several states (e.g., collectively referred to as states 702) through which a bootstrap operation and / or a resource discovery operation is orchestrated, according to at least one embodiment. The states 702 may include any suitable number of states (e.g., a not started state 704, a waiting release state 706, a waiting plan state 708, a waiting discovery state 710, a waiting import state 712, a waiting edit state state 714, a waiting state verification 716, a waiting approval 718, a waiting completion 720, a success state 722, a discard state 724, a failure state 726, and a cancellation state 728). The specific states illustrated in FIG. 7 are not intended to limit the scope of the disclosure. More or fewer and / or different states may be used and are not necessarily limited to the specific states illustrated in FIG. 7. These specific states are described in conjunction with the example of FIG. 6.

[0117] Returning to Figure 6, at 610, the MFO 602 (e.g., state manager 428) may transition to a NOT_STARTED state (e.g., corresponding to the not started state 704 of Figure 7). While in the NOT_STARTED state, the MFO 602 may perform an operation that determines that certain predefined conditions are met. If those predefined conditions are met, the MFO 602 (e.g., state manager 428) may transition to an AWAITING_RELEASE state at 612. The AWAITING_RELEASE state may correspond to the awaiting release state 706 of Figure 7.

[0118] At 614, while in the AWAITING_RELEASE state, the MFO 602 can be configured to begin traversing a build dependency graph (e.g., build dependency graph 338 of FIG. 3). A first flock config can be selected that corresponds to a start node of the build dependency graph. The first flock config can correspond to a unit of change (unit of change in a data center) that is made to the region in the build. For example, considering that the region being built is a ViBE region (e.g., ViBE region 316), the first flock config can be associated with bootstrapping capability services 318 and workers 320 of FIG. 3 (also depicted in FIG. 2) in the ViBE region being built. The start node can be the start node because the corresponding flock config has no dependencies on other capabilities of the available ViBE region.

[0119] At 616, the MFO 602 may send a set of instructions corresponding to the release of the first flock config. In one embodiment, the MFO 602 may send a first request to the CIOS Central 604 with the first flock config including the region data populated as described above. After sending, the MFO 620 (e.g., state manager 428) may transition to state AWAITING_PLAN at 621 (corresponding to the waiting for plan state 708 of FIG. 7).

[0120] At 618, CIOS central 604 can receive the flock config and can take action to plan the release. For example, CIOS central 604 can attempt to verify the state of resources in the first flock config in ViBE region 316. CIOS central 604 can determine that an instance of CIOS regional and deployment orchestrator are required to bootstrap the resources (e.g., capability service 318 and worker 320) identified in the first flock config. Because ViBE region 316 is new, CIOS central 604 can determine that none of the resources required for the flock exist. In response, CIOS central 604 can generate planning data 620.

[0121] FIG. 8 is a block diagram illustrating an example of an instance of state data 800, according to at least one embodiment. State data 800 may be included as part of the planning data 620 of FIG. 6. As shown in FIG. 8, state data 800 may include any suitable data corresponding to any suitable number of resources to be created. In the ongoing example of FIG. 6, state data 800 may indicate that an instance of CIOS regional, an instance of deployment orchestrator, a capability service, and a worker are deployed. Each of these components may correspond to resource 1, resource 2, resource 3, and resource 4, respectively, of state data 800. Any suitable corresponding resource data (e.g., resource data 802, 804, 806, and 808 may be initially included in state data 800). Returning to FIG. 6, planning data 620 may be sent to MFO 602 at 622.

[0122] At 624, upon receiving the planning data 620, the MFO 602 sends a request to the resource hunter 606 (e.g., an example of the resource hunter 420 of FIG. 4) to identify existing resources that may be available to effect this release. At 626, the MFO 602 (e.g., the state manager 428) may transition to AWAITING_DISCOVERY (e.g., corresponding to the waiting for discovery state 710 of FIG. 7). As discussed above in connection with FIG. 5, the first flock config may include a code segment 502 that defines a set of parameters to be matched against resources.

[0123] At 628, the resource hunter 606 can extract parameters from the resource discovery data provided in the first block config (e.g., code segment 502). In an embodiment, the resource hunter 606 can take any suitable action to query in the locations specified by the resource discovery data (e.g., defined in code segment 502, each AD, each compartment, etc.) and / or using any suitable combination of the parameters provided (e.g., “dataType”, “Resource_Name” provided in code segment 502).

[0124] At 630, the resource hunter 606 may send any suitable data to the MFO 602 indicating any suitable number of resources identified as matching the resource discovery data. The MFO 602 may send another request at 632 to request that the resource hunter 606 perform an import operation.

[0125] At 634, upon receiving the data 630, the MFO 602 (eg, state manager 428) may transition to state AWAITING_IMPORT, which corresponds to the waiting for import state 714 in FIG.

[0126] At 636, the resource hunter 606 can be configured to identify a set of import operations to be performed. For example, the resource hunter 606 can identify the import operations from a first flock config in the provided resource discovery data. For example, the resource hunter 606 can identify the code segment 504 of FIG. 5 and execute the identified import operations. In the example in progress, this can obtain any suitable data regarding the matching resources, such as the address and resource identifier of each resource. In an embodiment, the first flock config for constructing ViBE 316 can include in its resource discovery data a location and parameters for identifying CIOS regional 314 and deployment orchestrator 317 in host region 332 of FIG. 3. Referring again to FIG. 8, resource X and resource Y can correspond to CIOS regional 314 and deployment orchestrator 317, respectively. Resource data X can include any suitable attribute of resource X (e.g., address and resource identifier, etc.), and resource data Y can include any suitable attribute of resource Y (e.g., address and resource identifier, etc.).

[0127] Returning to FIG. 6, at 638, the resource hunter 606 may send the obtained resource data (eg, addresses and resource identifiers of the CIOS regional 314 and deployment orchestrator 317 of the host region 332) to the MFO 602.

[0128] Upon receiving 638 the data, the MFO 602 (eg, state manager 428) may transition 640 to state AWAITING_STATE_EDIT, which corresponds to the wait to edit state 716 in FIG.

[0129] At 642, the MFO 602 may take action to update the state data of the planning data 620. Referring again to FIG. 8, the MFO 602 may modify at least a portion of the resource data 802 with any suitable portion of the resource data X (which in this example corresponds to the CIOS regional 314). In the ongoing example, the MFO 602 may modify the resource data 802 to include the obtained address and resource identifier of the CIOS regional 314 in the host region 332. Similarly, the MFO 602 may modify the resource data 804 to include the obtained address and resource identifier of the deployment orchestrator 317 in the host region 332. The state data 810 is intended to show a modified version of the state data 800 after these actions have been taken.

[0130] Returning to FIG. 6, at 644, the MFO 602 (e.g., state manager 428) may transition to the state AWAITING_STATE_VALIDATION, which corresponds to the waiting state validation state 718 in FIG.

[0131] At 646, the MFO 602 may send the revised plan data, including the status data 810 of FIG.

[0132] At 648, CIOS Central 604 can update planning data and can take action to re-plan the release using the updated planning data provided by MFO 602. As part of those actions, planning data 650 can be generated. Planning data 650 can identify resources to be created. In an ongoing embodiment, CIOS Regional 314 and Deployment Orchestrator 317 may not be included in planning data 650, or if included, planning data 650 can indicate that they are used instead of creating new resources.

[0133] At 652, the MFO 602 can receive the planning data 650 and can compare the planning data 650 to the state data that the MFO 602 sent in 646 to determine whether the planning data 650 matches the requested state data. For example, the MFO 602 can determine from the planning data 650 that the CIOS central 604 does not plan to create a new instance of a CIOS regional, but rather intends to use the CIOS regional 314 of the host region 332. Similarly, the MFO can determine from the planning data 650 that the CIOS central 604 does not plan to create a new instance of the deployment orchestrator 317, but rather intends to use the deployment orchestrator 317 of the host region 332.

[0134] At 654, if the plan data 650 matches the state data sent at 646, the MFO 602 (e.g., state manager 428) may transition to a state AWAITING_APPROVAL, which corresponds to the awaiting approval state 720 of FIG. 7. While in this state, any suitable operation may be taken to present any suitable portion of the plan data 650 to a user (e.g., via any suitable user interface hosted by CIOS central 604 to user 302 of FIG. 3). In an embodiment, if the user approves, an indication may be received at 654. Alternatively, the MFO 604 may automatically approve the plan data depending on an indication provided within the block configuration. Upon receiving an indication that the plan data 650 is approved (e.g., automatically or by user input), the MFO 602 (e.g., state manager 428) may transition to a state AWAITING_COMPLETION, which corresponds to the awaiting completion state 720 of FIG. 7. An indication may be sent at 660 that the plan data 650 has been approved.

[0135] At 662, CIOS central 604 may take any suitable action to bootstrap resources of the first flock config in the region being constructed (e.g., ViBE 316 of FIG. 3). For example, CIOS central 604 may instruct CIOS regional 314 (in light of planning data 650) to bootstrap capability service 318 of FIG. 3 and worker 320 of FIG. 3. When capability service 318 is operational, capability service 318 may be configured to expose a capability indicating this.

[0136] At 664, the MFO 602 may receive (or otherwise obtain) an indication that the corresponding capability has been published to the capability service 318. In response to determining that the published capability is specified in the flock config, the MFO 602 (e.g., state manager 428) may transition to state SUCCESSFUL at 667, which corresponds to the awaiting completion state 720 of Figure 7.

[0137] This allows one pass (also called a "release") of the flock config for the region (e.g., ViBE 316) to be completed. After reaching a successful state, MFO 602 can be configured to resume traversing the build dependency graph 338 to identify the next release / pass. This process can occur any suitable number of times, and a particular flock config can include any suitable discovery data that allows it to import and use resources in that region (or another region) using it (e.g., by a CIOS regional in that region).

[0138] A similar process can be performed to deploy instances of capability services, deployment orchestrators, CIOS regionals, and workers in the target region. After the user adds region data corresponding to the target region (e.g., target region 334), a different set of flock configs will be identified by the MFO. Any suitable portion of these flock configs can include resource discovery data at any time that can cause the MFO 602 to instruct CIOS Central that resources in ViBE be used for the bootstrap operation performed for the target region.

[0139] 7, it should be appreciated that state manager 428 can monitor for various conditions (e.g., user input, timeouts, error codes, etc.) to identify when to transition to one of states 724-728. In one embodiment, state manager 428 can transition from any of states 704-720 to any of states 724-728 at any appropriate time based at least in part on detection of a predefined condition (e.g., data that the MFO is waiting for has not been received for a predefined period of time).

[0140] FIG. 9 illustrates an example method 900 of discovering existing resources according to at least one embodiment. Method 900 may be performed by one or more components of a cloud computing environment (e.g., cloud infrastructure orchestration service 402 of FIG. 4). For example, method 900 may be performed at least in part by a resource identification service (e.g., resource hunter of FIG. 4 and / or FIG. 6). A computer-readable storage medium includes computer-readable instructions that, when executed by one or more processors of a computing device, cause a computing device to perform method 900. Method 900 may be performed in any suitable order or in parallel. It should be understood that method 900 may include more or fewer steps than those shown in FIG. 9.

[0141] Method 900 may begin at 902, where a flock configuration file (e.g., flock configuration file 500) including resource discovery data associated with a service may be obtained (e.g., by a resource identification service, such as resource hunter 420 of FIG. 4 and / or resource hunter 606 of FIG. 6). In an embodiment, the resource discovery data may indicate a set of parameters using which existing resources in a cloud computing environment are identified. For example, FIG. 5 includes code segment 502, which may include parameters such as those discussed above in connection with FIG. 5.

[0142] An operation of identifying an existing resource may be performed at 904. In an embodiment, an existing resource (e.g., CIOS regional 314 described in the example of FIG. 6) may be identified based at least in part on matching attributes associated with the existing resource with a set of parameters of the resource discovery data.

[0143] A set of import operations to perform to obtain identifiers corresponding to existing resources can be identified at 906. For example, Figure 5 includes a code segment 504 that can include example import operations such as those described above in connection with Figure 5.

[0144] At 908, identifiers corresponding to existing resources identified based at least in part on the execution of the set of import operations may be transmitted (eg, to the MFO 602 of FIG. 6).

[0145] FIG. 10 illustrates an example method 1000 of using existing resources to perform region construction according to at least one embodiment. Method 1000 may be performed by one or more components of cloud infrastructure orchestration service 102 of FIG. 1 (e.g., an orchestration service such as the multi-flock orchestrator of FIGS. 1-4 and / or 6). A computer-readable storage medium includes computer-readable instructions that, when executed by one or more processors of a computing device, cause the computing device to perform method 1000. Method 1000 may be performed in any suitable order. It should be understood that method 1000 may include more or fewer steps than shown in FIG. 10.

[0146] The method 1000 can begin at 1002 by obtaining a number of flock configuration files corresponding to a number of services. In one embodiment, the number of services are services that are bootstrapped within the region during the region construction process.

[0147] At 1004, an order in which the multiple services are bootstrapped within the region can be determined based at least in part on the multiple flock configuration files. For example, a static analysis of the multiple flock configuration files can be performed to generate a build dependency graph 338 that identifies the order in which flock configs are released to bootstrap the multiple services within the region.

[0148] At 1006, a first request for bootstrapping a service of the plurality of services may be sent. The first request may include a flock configuration file corresponding to the first flock config. These operations may correspond to step 616 of FIG. 6.

[0149] At 1008, plan data (eg, plan data 620 of FIG. 6) can be received (eg, from CIOS Central 604 of FIG. 6) that initially indicates resources to be created to bootstrap the service.

[0150] At 1010, an identifier corresponding to a previously created resource in the cloud computing environment may be obtained (e.g., from resource discovery service 606 of FIG. 6). For example, an address of CIOS regional 314 and a resource identifier may be received, as described in step 638 of FIG.

[0151] In step 1012, the planning data may be modified to include identifiers corresponding to previously created resources of the cloud computing environment. An example of modified planning data may include state data 810 of FIG.

[0152] At 1014, a second request for bootstrapping the service may be sent (e.g., to CIOS central 604). In one embodiment, the second request includes planning data that includes identifiers corresponding to the previously created resources. Sending the second request including the identifiers may cause the service to be bootstrapped within the region using the previously created resources in the manner described in connection with FIG.

[0153] Cloud Service Infrastructure Architecture Examples As mentioned above, Infrastructure as a Service (IaaS) is one particular type of cloud computing. IaaS can be configured to provide virtualized computing resources over a public network (e.g., the Internet). In the IaaS model, a cloud computing provider can host infrastructure components (e.g., servers, storage devices, network nodes (e.g., hardware), deployment software, platform virtualization (e.g., hypervisor layer), etc.). In some cases, an IaaS provider can also provide various services (e.g., billing, monitoring, logging, load balancing, clustering, etc.) that accompany those infrastructure components. Thus, these services can be policy-driven, so that an IaaS user can enforce policies to drive load balancing to maintain application availability and performance.

[0154] In some cases, IaaS customers can access resources and services over a wide area network (WAN) such as the Internet, and can use the cloud provider's services to install the remaining elements of their application stack. For example, a user can log into an IaaS platform to create virtual machines (VMs), install an operating system (OS) on each VM, deploy middleware such as databases, create storage buckets for workloads and backups, and even install enterprise software on the VMs. The customer can then use the provider's services to perform a variety of functions, including balancing network traffic, troubleshooting application issues, monitoring performance, managing disaster recovery, and more.

[0155] In most cases, the cloud computing model will require the involvement of a cloud provider, which can be, but is not necessarily, a third-party service that specializes in providing IaaS (e.g., offering, renting, selling). An entity may choose to deploy a private cloud, thereby becoming its own provider of infrastructure services.

[0156] In one example, an IaaS deployment is the process of placing a new application or a new version of an application onto a prepared application server, etc. It may also include the process of preparing the server (e.g., installing libraries, daemons, etc.). This is often managed by the cloud provider below the hypervisor layer (e.g., server, storage, network hardware, and virtualization). Thus, the customer may be responsible for handling (OS), middleware, and / or application deployment (e.g., self-service virtual machines (which can be spun up on demand, etc.)).

[0157] In one example, IaaS provisioning refers to obtaining a computer or virtual host for use and then installing the required libraries or services on it. In many cases, deployment does not include provisioning, which may need to occur first.

[0158] In some cases, there are two different challenges in IaaS provisioning. First, there is the initial challenge of provisioning an initial set of instructions before anything can be put into production. Second, there is the challenge of evolving the existing infrastructure (e.g. adding new services, modifying services, removing services, etc.) after everything has been provisioned. In some cases, these two challenges can be addressed by allowing the configuration of the infrastructure to be defined declaratively. In other words, one or more configuration files can define the infrastructure (e.g. what components are needed and how they interact). Thus, the overall topology of the infrastructure (e.g. what resources depend on which resources and how each of them work together) can be defined declaratively. In some cases, after the topology is defined, workflows can be generated that create and / or manage the different components described in the configuration files.

[0159] In one example, an infrastructure can have many interconnected elements. For example, there may be one or more virtual private clouds (VPCs) (e.g., a potentially on-demand pool of configurable and / or shared computing resources), also referred to as a core network. In one example, there may also be one or more inbound / outbound traffic group rules that are provisioned to define how the network's inbound / outbound traffic is configured and one or more virtual machines (VMs). Other infrastructure elements, such as load balancers, databases, etc., can also be provisioned. The infrastructure can evolve over time as more and more infrastructure elements are required and / or added.

[0160] In some cases, continuous deployment techniques can be employed to enable deployment of infrastructure code across various virtual computing environments. Additionally, the described techniques can enable infrastructure management within these environments. In some examples, a service team can write code that is required to be deployed to one or more, often many, different production environments (e.g., across various different geographic locations, sometimes spanning the globe). In some examples, however, the infrastructure onto which the code will be deployed must first be set up. In some cases, provisioning can be done manually, provisioning tools can be used to provision the resources, and / or deployment tools can be used to deploy the code once the infrastructure has been provisioned.

[0161] 11 is a block diagram 1100 illustrating an example platform of an IaaS architecture in accordance with at least one embodiment. A service operator 1102 can be communicatively coupled to a secure host tenancy 1104, which can include a virtual cloud network (VCN) 1106 and a secure host subnet 1108. In one example, the service operator 1102 can use one or more customer computing devices, which can be portable handheld devices (e.g., iPhones, mobile phones, iPads, computing tablets, personal digital assistants (PDAs)) or wearable devices (e.g., Google Glass head-mounted displays) running software such as Microsoft Windows Mobile and / or various mobile operating systems such as iOS, Windows Phone, Android, BlackBerry 8, Palm OS, and capable of Internet, email, short message service (SMS), Blackberry, or other communications protocols. Alternatively, the customer computing devices may be general purpose personal computers, including, for example, personal and / or laptop computers running various versions of Microsoft Windows, Apple Macintosh, and / or Linux operating systems. The customer computing devices may be workstation computers running any of the various commercially available UNIX or UNIX-like operating systems, including, but not limited to, the various GNU / Linux operating systems, such as Google Chrome OS.Alternatively or additionally, the customer computing device may be any other electronic device, such as a thin-client computer, an Internet-enabled gaming system (e.g., a Microsoft Xbox gaming console with or without a Kinect® gesture input device), and / or a personal messaging device capable of communicating via VCN 1106 and / or an Internet-accessible network.

[0162] The VCN 1106 may include a local peering gateway (LPG) 1110 that may be communicatively coupled to an SSH VCN 1112 via an LPG 1110 that is included in a secure shell (SSH) VCN 1112. The SSH VCN 1112 may include an SSH subnet 1114, which may be communicatively coupled to a control plane VCN 1116 via an LPG 1110 that is included in the control plane VCN 1116. The SSH VCN 1112 may also be communicatively coupled to a data plane VCN 1118 via the LPG 1110. The control plane VCN 1116 and the data plane VCN 1118 may be included in a service tenancy 1119 that may be owned and / or operated by the IaaS provider.

[0163] The control plane VCN 1116 may include a control plane demilitarized zone (DMZ) tier 1120 that serves as a perimeter network (e.g., a portion of an enterprise network between the enterprise intranet and an external network). DMZ-based servers may have limited duties and help thwart intrusions. Additionally, the DMZ tier 1120 may include one or more load balancer (LB) subnets 1122, a control plane application tier 1124 that may include application subnets 1126, a control plane data tier 1128 that may include a database (DB) subnet 1130 (e.g., a front-end DB subnet and / or a back-end DB subnet). The LB subnet 1122 included in the control plane DMZ tier 1120 can be communicatively coupled to an application subnet 1126 included in the control plane application tier 1124 and an Internet gateway 1134 that may be included in the control plane VCN 1116, which can be communicatively coupled to a DB subnet 1130 included in the control plane data tier 1128 and a service gateway 1136 and a network address translation (NAT) gateway 1138. The control plane VCN 1116 can include the service gateway 1136 and the NAT gateway 1138.

[0164] The control plane VCN 1116 can include a data plane mirrored application tier 1140 that can include an application subnet 1126. The application subnet 1126 included in the data plane mirrored application tier 1140 can include a virtual network interface controller (VNIC) 1142 capable of running a compute instance 1144. The compute instance 1144 can communicatively couple the application subnet 1126 of the data plane mirrored application tier 1140 to the application subnet 1126 that can be included in the data plane application tier 1146.

[0165] The data plane VCN 1118 can include a data plane application layer 1146, a data plane DMZ layer 1148, and a data plane data layer 1150. The data plane DMZ layer 1148 can include a LB subnet 1122 that can be communicatively coupled to an application subnet 1126 of the data plane application layer 1146 and an Internet gateway 1134 of the data plane VCN 1118. The application subnet 1126 can be communicatively coupled to a service gateway 1136 of the data plane VCN 1118 and a NAT gateway 1138 of the data plane VCN 1118. The data plane data layer 1150 can also include a DB subnet 1130 that can be communicatively coupled to the application subnet 1126 of the data plane application layer 1146.

[0166] The Internet gateways 1134 of the control plane VCN 1116 and the data plane VCN 1118 can be communicatively coupled to a metadata management service 1152, which can be communicatively coupled to the public Internet 1154. The public Internet 1154 can be communicatively coupled to NAT gateways 1138 of the control plane VCN 1116 and of the data plane VCN 1118. The service gateways 1136 of the control plane VCN 1116 and of the data plane VCN 1118 can be communicatively coupled to cloud services 1156.

[0167] In one example, a service gateway 1136 in the control plane VCN 1116 or in the data plane VCN 1118 can make application programming interface (API) calls to a cloud service 1156 without traversing the public Internet 1154. API calls from the service gateway 1136 to the cloud service 1156 can be one-way; that is, the service gateway 1136 can make API calls to the cloud service 1156 and the cloud service 1156 can send the requested data to the service gateway 1136. However, the cloud service 1156 cannot originate API calls to the service gateway 1136.

[0168] In one example, a secure host tenancy 1104 can be directly connected to an otherwise separable service tenancy 1119. A secure host subnet 1108 can communicate with an SSH subnet 1114 through an LPG 1110, which can enable bidirectional communication through otherwise separate systems. Connecting the secure host subnet 1108 to the SSH subnet 1114 can give the secure host subnet 1108 access to other entities in the service tenancy 1119.

[0169] The control plane VCN 1116 can enable users of the service tenancy 1119 to configure or otherwise provision desired resources. The desired resources provisioned in the control plane VCN 1116 can be deployed or otherwise used in the data plane VCN 1118. In one example, the control plane VCN 1116 can be separate from the data plane VCN 1118, and a data plane mirror application tier 1140 of the control plane VCN 1116 can communicate with a data plane application tier 1146 of the data plane VCN 1118 via a VNIC 1142, which can be included in the data plane mirror application tier 1140 and the data plane application tier 1146.

[0170] In one example, a user or customer of the system may make a request, for example, a create, read, update or delete (CRUD) operation, via the public Internet 1154, which may communicate the request to a metadata management service 1152. The metadata management service 1152 may communicate the request to the control plane VCN 1116 via an Internet gateway 1134. The request may be received by a LB subnet 1122 included in the control plane DMZ layer 1120. The LB subnet 1122 may determine that the request is valid, and in response to this determination, the LB subnet 1122 may send the request to an application subnet 1126 included in the control plane application layer 1124. If the validity of the request is verified and a call to the public Internet 1154 is required, the call to the public Internet 1154 may be sent to a NAT gateway 1138, which may make the call to the public Internet 1154. Memory that may be desired to be stored by the request may be stored in the DB subnet 1130.

[0171] In one example, a data plane mirror application layer 1140 can facilitate direct communication between the control plane VCN 1116 and the data plane VCN 1118. For example, it may be desired that configuration changes, updates, or other appropriate modifications be applied to resources included in the data plane VCN 1118. The VNIC 1142 allows the control plane VCN 1116 to communicate directly with the resources included in the data plane VCN 1118, thereby enabling it to perform configuration changes, updates, or other appropriate modifications to those resources.

[0172] In an embodiment, the control plane VCN 1116 and the data plane VCN 1118 may be included in the service tenancy 1119. In this case, a user or customer of the system may not own or operate the control plane VCN 1116 or the data plane VCN 1118. Instead, an IaaS provider may own or operate the control plane VCN 1116 and the data plane VCN 1118, both of which may be included in the service tenancy 1119. This embodiment may allow for network isolation that may prevent a user or customer from interacting with the resources of other users or other customers. This embodiment may also allow a user or customer of the system to store databases privately without having to rely on the public Internet 1154, which may not have the desired level of threat protection for storage.

[0173] In another embodiment, the LB subnet 1122 included in the control plane VCN 1116 can be configured to receive signals from the service gateway 1136. In this embodiment, the control plane VCN 1116 and the data plane VCN 1118 can be configured to be called by the IaaS provider's customers without calling the public Internet 1154. Customers of the IaaS provider may desire this embodiment because databases used by the customers can be controlled by the IaaS provider and stored on a service tenancy 1119 that is separable from the public Internet 1154.

[0174] 12 is a block diagram 1200 illustrating another example pattern of an IaaS architecture, according to at least one embodiment. A service operator 1202 (e.g., service operator 1102 of FIG. 11 ) can be communicatively coupled to a secure host tenancy 1204 (e.g., secure host tenancy 1104 of FIG. 11 ), which can include a virtual cloud network (VCN) 1206 (e.g., VCN 1106 of FIG. 11 ) and a secure host subnet 1208 (e.g., secure host subnet 1108 of FIG. 11 ). The VCN 1206 can include a local peering gateway (LPG) 1210 (e.g., LPG 1110 of FIG. 11 ) that can be communicatively coupled to an SSH VCN 1212 (e.g., SSH VCH 1112 of FIG. 11 ) via an LPG 1110 included in a secure shell (SSH) VCN 1212. SSH VCN 1212 can include an SSH subnet 1214 (e.g., SSH subnet 1114 in FIG. 11), which can be communicatively coupled to a control plane VCN 1216 (e.g., control plane VCN 1116 in FIG. 11) via an LPG 1210 included in the control plane VCN 1216. The control plane VCN 1216 can be included in a service tenancy 1219 (e.g., service tenancy 1119 in FIG. 11), and the data plane VCN 1218 (e.g., data plane VCN 1118 in FIG. 11) can be included in a customer tenancy 1221, which can be owned or operated by a user or customer of the system.

[0175] The control plane VCN 1216 may include a control plane DMZ tier 1220 (e.g., control plane DMZ tier 1120 of FIG. 11 ) which may include a LB subnet 1222 (e.g., LB subnet 1122 of FIG. 11 ), a control plane tier 1224 (e.g., control plane application tier 1124 of FIG. 11 ) which may include an application subnet 1226 (e.g., application subnet 1126 of FIG. 11 ), and a control plane data tier 1228 (e.g., control plane data tier 1128 of FIG. 11 ) which may include a database (DB) subnet 1230 (e.g., similar to DB subnet 1130 of FIG. 11 ). LB subnet 1222 included in control plane DMZ tier 1220 can be communicatively coupled to application subnet 1226 included in control plane application tier 1224 and to an Internet gateway 1234 (e.g., Internet gateway 1134 in FIG. 11 ) that may be included in control plane VCN 1216, and application subnet 1226 can be communicatively coupled to DB subnet 1230 included in control plane data tier 1228 and to a service gateway 1236 (e.g., service gateway 1136 in FIG. 11 ) and a network address translation (NAT) gateway 1238 (e.g., NAT gateway 1138 in FIG. 11 ). Control plane VCN 1216 can include service gateway 1236 and NAT gateway 1238.

[0176] The control plane VCN 1216 can include a data plane mirrored application tier 1240 (e.g., data plane mirrored application tier 1140 of FIG. 11 ), which can include an application subnet 1226. The application subnet 1226 included in the data plane mirrored application tier 1240 can include a virtual network interface controller (VNIC) 1242 (e.g., VNIC 1142) that can run a compute instance 1244 (e.g., similar to compute instance 1144 of FIG. 11 ). The compute instance 1244 can facilitate communication between the application subnet 1226 of the data plane mirrored application tier 1240 and the application subnet 1226 that can be included in the data plane application tier 1246 (e.g., data plane application tier 1146 of FIG. 11 ) via the VNIC 1242 included in the data plane mirrored application tier 1240 and the VNIC 1242 included in the data plane application tier 1246.

[0177] An Internet gateway 1234 included in the control plane VCN 1216 can be communicatively coupled to a metadata management service 1252 (e.g., metadata management service 1152 of FIG. 11 ), which can be communicatively coupled to a public Internet 1254 (e.g., public Internet 1154 of FIG. 11 ). The public Internet 1254 can be communicatively coupled to a NAT gateway 1238 included in the control plane VCN 1216. A service gateway 1236 included in the control plane VCN 1216 can be communicatively coupled to cloud services 1256 (e.g., cloud services 1156 of FIG. 11 ).

[0178] In one example, the data plane VCN 1218 may be included in the customer tenancy 1221. In this case, the IaaS provider may provide each customer with a control plane VCN 1216, and the IaaS provider may configure a unique compute instance 1244 included in the service tenancy 1219 for each customer. Each compute instance 1244 may enable communication between the control plane VCN 1216 included in the service tenancy 1219 and the data plane VCN 1218 included in the customer tenancy 1221. The compute instance 1244 may enable resources provisioned in the control plane VCN 1216 included in the service tenancy 1219 to be deployed or otherwise used in the data plane VCN 1218 included in the customer tenancy 1221.

[0179] In another example, an IaaS provider customer may have a database that resides within customer tenancy 1221. In this example, control plane VCN 1216 may include data plane mirror application tier 1240, which may include application subnet 1226. Data plane mirror application tier 1240 may reside in data plane VCN 1218, but data plane mirror application tier 1240 may not reside in data plane VCN 1218. That is, data plane mirror application tier 1240 may have access to customer tenancy 1221, but data plane mirror application tier 1240 may not reside in data plane VCN 1218 or be owned or operated by the IaaS provider customer. Data plane mirror application tier 1240 may be configured to make calls to data plane VCN 1218, but may not be configured to make calls to any entities contained in control plane VCN 1216. A customer may wish to deploy or otherwise use resources in the data plane VCN 1218 that have been provisioned in the control plane VCN 1216, and the data plane mirror application tier 1240 can facilitate the desired deployment or other use of the customer's resources.

[0180] In one embodiment, an IaaS provider's customer can filter the data plane VCN 1218. In this embodiment, the customer can determine what the data plane VCN 1218 can access, and the customer can limit access to the public internet 1254 from the data plane VCN 1218. It may not be possible for the IaaS provider to filter or otherwise control the data plane VCN 1218's access to external networks or databases. The application of filters and controls by the customer to the data plane VCN 1218 contained in the customer tenancy 1221 can facilitate isolating the data plane VCN 1218 from other customers and from the public internet 1254.

[0181] In an embodiment, cloud services 1256 can be invoked by service gateway 1236 to access services that may not exist on public internet 1254, control plane VCN 1216, or data plane VCN 1218. The connection between cloud services 1256 and control plane VCN 1216 or data plane VCN 1218 may not be live or continuous. Cloud services 1256 can exist on different networks owned or operated by the IaaS provider. Cloud services 1256 can be configured to receive calls from service gateway 1236 and not receive calls from public internet 1254. Some cloud services 1256 can be isolated from other cloud services 1256, and control plane VCN 1216 can be isolated from cloud services 1256 that may not be in the same region as control plane VCN 1216. For example, control plane VCN 1216 may be located in "Region 1" and cloud service "Deployment 11" may be located in Region 1 and Region 2. When a call is made to deployment 11 by service gateway 1236 included in control plane VCN 1216 located in region 1, the call can be sent to deployment 11 in region 1. In this example, control plane VCN 1216, or deployment 11 in region 1, cannot be communicatively coupled or otherwise communicate with deployment 11 in region 2.

[0182] 13 is a block diagram 1300 illustrating another example pattern of an IaaS architecture, according to at least one embodiment. A service operator 1302 (e.g., service operator 1102 of FIG. 11 ) can be communicatively connected to a secure host tenancy 1304 (e.g., secure host tenancy 1104 of FIG. 11 ) which can include a virtual cloud network (VCN) 1306 (e.g., VCN 1106 of FIG. 11 ) and a second secure host subnet 1308 (e.g., secure host subnet 1108 of FIG. 11 ). VCN 1306 can include an LPG 1310 (e.g., LPG 1110 of FIG. 11 ) which can be communicatively connected to an SSH VCN 1312 (e.g., SSH VCN 1112 of FIG. 11 ) via an LPG 1310 included in SSH VCN 1312. SSH VCN 1312 can include SSH subnet 1314 (e.g., SSH subnet 1114 in FIG. 11), which can be communicatively connected to a control plane VCN 1316 (e.g., control plane VCN 1116 in FIG. 11) via an LPG 1310 included in control plane VCN 1316 and to a data plane VCN 1318 (e.g., data plane 1118 in FIG. 11) via an LPG 1310 included in data plane VCN 1318. Control plane VCN 1316 and data plane VCN 1318 can be included in service tenancy 1319 (e.g., service tenancy 1119 in FIG. 11).

[0183] The control plane VCN 1316 may include a control plane DMZ tier 1320 (e.g., control plane DMZ tier 1120 of FIG. 11 ) which may include a load balancer (LB) subnet 1322 (e.g., LB subnet 1122 of FIG. 11 ), a control plane application tier 1324 (e.g., control plane application tier 1124 of FIG. 11 ) which may include an application subnet 1326 (e.g., similar to application subnet 1126 of FIG. 11 ), and a control plane data tier 1328 (e.g., control plane data tier 1128 of FIG. 11 ) which may include a DB subnet 1330. The LB subnet 1322 included in the control plane DMZ tier 1320 can be communicatively coupled to an application subnet 1326 included in the control plane application tier 1324 and to an Internet gateway 1334 (e.g., Internet gateway 1134 in FIG. 11 ) that may be included in the control plane VCN 1316, and the application subnet 1326 can be communicatively coupled to a DB subnet 1330 included in the control plane data tier 1328 and to a service gateway 1336 (e.g., service gateway 1136 in FIG. 11 ) and a network address translation (NAT) gateway 1338 (e.g., NAT gateway 1138 in FIG. 11 ). The control plane VCN 1316 can include the service gateway 1336 and the NAT gateway 1338.

[0184] The data plane VCN 1318 can include a data plane application layer 1346 (e.g., data plane application layer 1146 in FIG. 11 ), a data plane DMZ layer 1348 (e.g., data plane DMZ layer 1148 in FIG. 11 ), and a data plane data layer 1350 (e.g., data plane data layer 1150 in FIG. 11 ). The data plane DMZ layer 1348 can include a LB subnetwork 1322 that can be communicatively coupled to a trusted application subnetwork 1360 and an untrusted application subnetwork 1362 of the data plane application layer 1346 and an Internet gateway 1334 included in the data plane VCN 1318. The trusted application subnetwork 1360 can be communicatively coupled to a service gateway 1336 included in the data plane VCN 1318, a NAT gateway 1338 included in the data plane VCN 1318, and a DB subnetwork 1330 included in the data plane data layer 1350. The untrusted application subnet 1362 can be communicatively coupled to a service gateway 1336 included in the data plane VCN 1318 and a DB subnet 1330 included in the data plane data layer 1350. The data plane data layer 1350 can include a DB subnet 1330 that can be communicatively coupled to a service gateway 1336 included in the data plane VCN 1318.

[0185] The untrusted application subnet 1362 may include one or more primary VNICs 1364(1)-(N) communicatively coupled to tenant virtual machines (VMs) 1366(1)-(N). Each tenant VM 1366(1)-(N) may be communicatively coupled to a respective application subnet 1367(1)-(N) that may be included in a respective container egress VCN 1368(1)-(N) that may be included in a respective customer tenancy 1370(1)-(N). Each secondary VNIC 1372(1)-(N) may facilitate communication between the untrusted application subnet 1362 included in the data plane VCN 1318 and the application subnet included in the container egress VCN 1368(1)-(N). Each container egress VCN 1368(1)-(N) may include a NAT gateway 1338 communicatively coupled to the public Internet 1354 (e.g., public Internet 1154 of FIG. 11 ).

[0186] An Internet gateway 1334 included in the control plane VCN 1316 and included in the data plane VCN 1318 may be communicatively coupled to a metadata management service 1352 (e.g., metadata management system 1152 of FIG. 11 ), which may be communicatively coupled to the public Internet 1354. The public Internet 1354 may be communicatively coupled to a NAT gateway 1338 included in the control plane VCN 1316 and included in the data plane VCN 1318. A service gateway 1336 included in the control plane VCN 1316 and included in the data plane VCN 1318 may be communicatively coupled to cloud services 1356.

[0187] In one embodiment, data plane VCN 1318 can be integrated with customer tenancy 1370. This integration can be useful or desirable in some cases to an IaaS provider's customer, such as when support is desired when executing code. A customer can provide code for execution that may be disruptive, communicate with other customer resources, or otherwise cause undesirable effects. In response, the IaaS provider can determine whether to execute the code provided to the IaaS provider by the customer.

[0188] In one example, an IaaS provider customer may be granted temporary network access to the IaaS provider and may request to add functionality to data plane application layer 1346. The code to perform that functionality may be executed in VMs 1366(1)-(N), and the code may not be configured to run anywhere else on data plane VCN 1318. Each VM 1366(1)-(N) may be connected to one customer tenancy 1370. Each container 1371(1)-(N) contained in VM 1366(1)-(N) may be configured to execute this code. In this case, there may be double isolation (e.g., container 1371(1)-(N) may execute code, and container 1371(1)-(N) may be at least contained in VM 1366(1)-(N) contained in untrusted application subnet 1362), which may help prevent malformed or otherwise undesirable code from damaging the IaaS provider's network or from damaging a different customer's network. Containers 1371(1)-(N) can be communicatively coupled to customer tenancy 1370 and can be configured to send or receive respective data to or from customer tenancy 1370. Containers 1371(1)-(N) cannot be configured to send or receive data to or from any other entities in data plane VCN 1318. Upon completion of code execution, the IaaS provider can disable or otherwise discard containers 1371(1)-(N).

[0189] In one embodiment, trusted application subnet 1360 may execute code owned or operated by the IaaS provider. In this embodiment, trusted application subnet 1360 may be communicatively coupled to DB subnet 1330 and may be configured to perform CRUD operations on DB subnet 1330. Untrusted application subnet 1362 may be communicatively connected to DB subnet 1330, but in this embodiment, the untrusted application subnet may be configured to perform read operations on DB subnet 1330. Containers 1371(1)-(N) may be included in each customer's VMs 1366(1)-(N) and may execute code from the customer, but may not be communicatively coupled to DB subnet 1330.

[0190] In other embodiments, the control plane VCN 1316 and the data plane VCN 1318 may not be directly communicatively coupled. In this embodiment, there may be no direct communication between the control plane VCN 1316 and the data plane 1318. However, communication may be indirect by at least one method. An IaaS provider may configure an LPG 1310 that may facilitate communication between the control plane VCN 1316 and the data plane VCN 1318. In another example, the control plane VCN 1316 or the data plane VCN 1318 may make a call to a cloud service 1356 through a service gateway 1336. For example, a call from the control plane VCN 1316 to the cloud service 1356 may include a request for a service that may communicate with the data plane VCN 1318.

[0191] 14 is a block diagram illustrating another example pattern of an IaaS architecture, according to at least one embodiment. A service operator 1402 (e.g., service operator 1102 of FIG. 11) can be communicatively connected to a secure host tenancy 1404 (e.g., secure host tenancy 1104 of FIG. 11), which can include a virtual cloud network (VCN) 1406 (e.g., VCN 1106 of FIG. 11) and a secure host subnet 1408 (e.g., secure host subnet 1108 of FIG. 11). The VCN 1406 can include an LPG 1410 (e.g., LPG 1110 of FIG. 11), which can be communicatively coupled to an SSH VCN 1412 (e.g., SSH VCN 1112 of FIG. 11) via an LPG 1410 included in the SSH VCN 1412. SSH VCN 1412 can include an SSH subnet 1414 (e.g., SSH subnet 1114 in FIG. 11), which can be communicatively coupled to a control plane VCN 1416 (e.g., control plane VCN 1116 in FIG. 11) via an LPG 1410 included in control plane VCN 1416 and to a data plane VCN 1418 (e.g., data plane 1118 in FIG. 11) via an LPG 1410 included in data plane VCN 1418. Control plane VCN 1416 and data plane VCN 1418 can be included in service tenancy 1419 (e.g., service tenancy 1119 in FIG. 11).

[0192] The control plane VCN 1416 may include a control plane DMZ layer 1420 (e.g., control plane DMZ layer 1120 of FIG. 11) which may include a LB subnet 1422 (e.g., LB subnet 1122 of FIG. 11), a control plane application layer 1424 (e.g., control plane application layer 1124 of FIG. 11) which may include an application subnet 1426 (e.g., application subnet 1126 of FIG. 11), and a control plane data layer 1428 (e.g., control plane data layer 1128 of FIG. 11) which may include a DB subnet 1430 (e.g., DB subnet 1330 of FIG. 13). The LB subnet 1422 included in the control plane DMZ tier 1420 can be communicatively coupled to an application subnet 1426 included in the control plane application tier 1424 and an Internet gateway 1434 (e.g., Internet gateway 1134 in FIG. 11 ) that may be included in the control plane VCN 1416, and the application subnet 1426 can be communicatively coupled to a DB subnet 1430 included in the control plane data tier 1428 and a service gateway 1436 (e.g., service gateway in FIG. 11 ) and a network address translation (NAT) gateway 1438 (e.g., NAT gateway 1138 in FIG. 11 ). The control plane VCN 1416 can include the service gateway 1436 and the NAT gateway 1438.

[0193] The data plane VCN 1418 can include a data plane application layer 1446 (e.g., data plane application layer 1146 in FIG. 11 ), a data plane DMZ layer 1448 (e.g., data plane DMZ layer 1148 in FIG. 11 ), and a data plane data layer 1450 (e.g., data plane data layer 1150 in FIG. 11 ). The data plane DMZ layer 1448 can include a LB subnetwork 1422 that can be communicatively coupled to a trusted application subnetwork 1460 (e.g., trusted application subnetwork 1360 in FIG. 13 ) and an untrusted application subnetwork 1462 (e.g., untrusted application subnetwork 1362 in FIG. 13 ) of the data plane application layer 1446 and an Internet gateway 1434 included in the data plane VCN 1418. The trusted application subnet 1460 may be communicatively coupled to a service gateway 1436 included in the data plane VCN 1418, a NAT gateway 1438 included in the data plane VCN 1418, and a DB subnet 1430 included in the data plane data layer 1450. The untrusted application subnet 1462 may be communicatively coupled to the service gateway 1436 included in the data plane VCN 1418 and the DB subnet 1430 included in the data plane data layer 1450. The data plane data layer 1450 may include the DB subnet 1430, which may be communicatively coupled to the service gateway 1436 included in the data plane VCN 1418.

[0194] The untrusted application subnet 1462 may include primary VNICs 1464(1)-(N) that may be communicatively coupled to tenant virtual machines (VMs) 1466(1)-(N) that are in the untrusted application subnet 1462. Each tenant VM 1466(1)-(N) may execute code in a respective container 1467(1)-(N) and may be communicatively coupled to an application subnet 1426 that may be included in a data plane application tier 1446 that may be included in a container egress VCN 1468. Each secondary VNIC 1472(1)-(N) may facilitate communication between the untrusted application subnet 1462 included in the data plane VCN 1418 and the application subnet included in the container egress VCN 1468. The container egress VCN may include a NAT gateway 1438 that may be communicatively coupled to the public Internet 1454 (e.g., public Internet 1154 in FIG. 11 ).

[0195] An Internet gateway 1434 included in the control plane VCN 1416 and in the data plane VCN 1418 can be communicatively coupled to a metadata management service 1452 (e.g., metadata management system 1152 of FIG. 11 ), which can be communicatively coupled to the public Internet 1454. The public Internet 1454 can be communicatively coupled to a NAT gateway 1438 included in the control plane VCN 1416 and in the data plane VCN 1418. A service gateway 1436 included in the control plane VCN 1416 and in the data plane VCN 1418 can be communicatively coupled to cloud services 1456.

[0196] In one example, the pattern illustrated by the architecture of block diagram 1400 of FIG. 14 can be considered an exception to the pattern illustrated by the architecture of block diagram 1300 of FIG. 13 and may be desirable for customers of an IaaS provider when the IaaS provider cannot communicate directly with the customer (e.g., disconnected region). Each container 1467(1)-(N) included in a VM 1466(1)-(N) for each customer is accessible in real time by that customer. The containers 1467(1)-(N) can be configured to make calls to a respective secondary VNIC 1472(1)-(N) included in an application subnet 1426 of a data plane application layer 1446 that can be included in a container egress VCN 1468. The secondary VNIC 1472(1)-(N) can send the calls to a NAT gateway 1438 that can send the calls to the public Internet 1454. In this example, containers 1467(1)-(N) that are accessible to a customer in real time can be isolated from control plane VCN 1416 and can be isolated from other entities contained within data plane VCN 1418. Containers 1467(1)-(N) can also be isolated from resources from other customers.

[0197] In another example, a customer can use container 1467(1)-(N) to invoke cloud service 1456. In this example, the customer can execute code in container 1467(1)-(N) that requests a service from cloud service 1456. Container 1467(1)-(N) can send the request to secondary VNIC 1472(1)-(N), which can send the request to a NAT gateway, which can send the request to public Internet 1454. Public Internet 1454 can send the request via Internet gateway 1434 to LB subnet 1422 included in control plane VCN 1416. In response to determining that the request is valid, the LB subnet can send the request to application subnet 1426, which can send the request via service gateway 1436 to cloud service 1456.

[0198] It should be understood that the IaaS architectures 1100, 1200, 1300, 1400 depicted in the figures may have components other than those depicted, and the depicted embodiments are only some examples of cloud infrastructure systems that may incorporate embodiments of the present disclosure. In some other embodiments, the IaaS systems may have more or fewer components than depicted in the figures, may combine two or more components, or may have a different configuration or arrangement of components.

[0199] In one embodiment, the IaaS system described herein may include a set of application, middleware, and database service offerings that are delivered to customers in a self-service, subscription-based, elastically scalable, reliable, highly available, and secure manner. One example of such an IaaS system is Oracle Cloud Infrastructure (OCI), offered by the assignee of the present application.

[0200] 15 illustrates an exemplary computer system 1500 in which various embodiments may be implemented. The system 1500 may be used to implement any of the computer systems described above. As shown, the computer system 1500 includes a processing unit 1504 that communicates with several peripheral subsystems via a bus subsystem 1502. These peripheral subsystems may include a processing acceleration unit 1506, an I / O subsystem 1508, a storage subsystem 1518, and a communication subsystem 1524. The storage subsystem 1518 includes a tangible computer readable storage medium 1522 and a system memory 1510.

[0201] Bus subsystem 1502 provides a mechanism for allowing the various components and subsystems of computer system 1500 to communicate with each other as desired. Although bus subsystem 1502 is shown diagrammatically as a single bus, alternative embodiments of the bus subsystem may use multiple buses. Bus subsystem 1502 may be any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. For example, such architectures may include an Industry Standard Architecture (ISA) bus, a MicroChannel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus, which may be implemented as a mezzanine bus manufactured in accordance with the IEEE P1386.1 standard.

[0202] A processing unit 1504, which may be implemented as one or more integrated circuits (e.g., conventional microprocessors or microcontrollers), controls the operation of the computer system 1500. The processing unit 1504 may include one or more processors. These processors may include single-core or multi-core processors. In an embodiment, the processing unit 1504 may be implemented as one or more independent processing units 1532 and / or 1534, with each processing unit including a single-core or multi-core processor. In other embodiments, the processing unit 1504 may be implemented as a quad-core processing unit formed by integrating two dual-core processors on a single chip.

[0203] In various embodiments, the processing unit 1504 may execute various programs in response to program code and may maintain multiple parallel executing programs or processes. At any given time, some or all of the program code being executed may reside within the processor 1504 and / or within the storage subsystem 1518. With appropriate programming, the processor 1504 may provide the various functions discussed above. The computer system 1500 may further include a processing acceleration unit 1506, which may include a digital signal processor (DSP), special purpose processor, and / or the like.

[0204] The I / O subsystem 1508 can include user interface input devices and user interface output devices. User interface input devices can include a keyboard, a pointing device such as a mouse or trackball, a touchpad or touch screen integrated into a display, a scroll wheel, a click wheel, a dial, a button, a switch, a keypad, a voice input device with a voice command recognition system, a microphone, and other types of input devices. User interface input devices can include motion sensing and / or gesture recognition devices, such as, for example, a Microsoft Kinect® motion sensor that allows a user to control and interact with an input device, such as a Microsoft Xbox® 360 game controller, through a natural user interface using gestures and speech commands. User interface input devices can also include eye gesture recognition devices, such as a Google Glass® blink detector, that detects eye activity from a user (e.g., "blinking" when taking pictures and / or selecting menus) and translates the eye gestures as input to an input device (e.g., Google Glass®). Additionally, the user interface input devices may include a voice recognition sensing device that allows a user to interact with a voice recognition system (e.g., the Siri® navigator) via voice commands.

[0205] User interface input devices may include, but are not limited to, three-dimensional (3D) mice, joysticks or pointing sticks, game pads, and graphic tablets, as well as audio / visual devices such as speakers, digital cameras, digital camcorders, portable media players, webcams, image scanners, fingerprint scanners, barcode readers 3D scanners, 3D printers, laser distance meters, and eye-tracking systems. Additionally, user interface input devices may include medical imaging input devices, such as, for example, computed tomography, magnetic resonance imaging, positron emission tomography, and medical ultrasound devices. User interface input devices may also include audio input devices, such as, for example, MIDI keyboards, digital musical instruments, and the like.

[0206] User interface output devices may include a display subsystem, non-visual displays such as indicator lights or audio output devices. The display subsystem may be a flat panel device such as one using a cathode ray tube (CRT), a liquid crystal display (LCD) or a plasma display, a projection device, a touch screen, etc. In general, use of the term "output device" is intended to include any type of possible device and mechanism for outputting information from computer system 1500 to a user or to another computer. For example, user interface output devices may include a variety of display devices that visually convey text, graphics and audio / video information, such as, but not limited to, monitors, printers, speakers, headphones, automobile navigation systems, plotters, audio output devices and modems.

[0207] Computer system 1500 may include a storage subsystem 1518 that provides a tangible, non-transitory computer-readable storage medium for storing software and data structures that provide functionality of embodiments described in this disclosure. The software may include programs, code modules, instructions, scripts, etc. that, when executed by one or more cores or processors of processing unit 1504, provide the functionality described above. Storage subsystem 1518 may also provide a repository for storing data used in accordance with the present disclosure.

[0208] As shown in FIG. 15, the storage subsystem 1518 can include various components including a system memory 1510, a computer readable storage medium 1522, and a computer readable storage medium reader 1520. The system memory 1510 can store program instructions that are loadable and executable by the processing unit 1504. The system memory 1510 can also store data used during execution of the instructions and / or data generated during execution of the program instructions. A variety of different types of programs can be loaded into the system memory 1510, including, but not limited to, client applications, web browsers, mid-tier applications, relational database management systems (RDBMS), virtual machines, containers, and the like.

[0209] The system memory 1510 may also store an operating system 1516. Examples of the operating system 1516 may include Microsoft Windows, Apple Macintosh, and / or Linux operating systems, various commercially available UNIX or UNIX-like operating systems (including but not limited to the GNU / Linux operating system, Google Chrome OS, etc.), and / or mobile operating systems, such as iOS, Windows Phone, Android OS, BlackBerry OS, and Palm OS operating systems. In certain implementations in which the computer system 1500 runs one or more virtual machines, the virtual machines, along with their guest operating systems (GOS), may be loaded into the system memory 1510 and executed by one or more processors or cores of the processing unit 1504.

[0210] The system memory 1510 may be provided in different configurations depending on the type of computer system 1500. For example, the system memory 1510 may be volatile memory (such as random access memory (RAM)) and / or non-volatile memory (such as read only memory (ROM), flash memory, etc.). Different types of RAM configurations can be provided, including static random access memory (SRAM), dynamic random access memory (DRAM), etc. In some embodiments, the system memory 1510 may include a basic input / output system (BIOS), which contains the basic routines that help to transfer information between elements within the computer system 1500, such as during start-up.

[0211] Computer-readable storage medium 1522 may represent remote, local, fixed and / or removable storage devices, as well as storage media for temporarily and / or more permanently containing and storing computer-readable information for use by computer system 1500, including instructions executable by processing unit 1504 of computer system 1500.

[0212] The computer readable storage medium 1522 may include any suitable medium known or used in the art, including storage media and communication media, such as, but not limited to, volatile and non-volatile, removable and non-removable media, implemented in any method or technology for storage and / or transmission of information. This may include tangible computer readable storage media, such as RAM, ROM, Electrically Erasable Programmable ROM (EEPROM), flash memory or other memory technology, CD-ROM, Digital Versatile Disk (DVD) or other optical storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or other tangible computer readable media.

[0213] For example, the computer readable storage medium 1522 may include hard disk drives that read or write non-removable non-volatile magnetic media, magnetic disk drives that read or write removable non-volatile magnetic disks, and optical disk drives that read or write removable non-volatile optical disks such as CD ROMs, DVDs and Blu-Ray disks or other optical media. The computer readable storage medium 1522 may include, but is not limited to, Zip drives, flash memory cards, Universal Serial Bus (USB) flash drives, Secure Digital (SD) cards, DVD disks, digital video tapes, and the like. The computer readable storage medium 1522 may also include solid state drives (SSDs) based on non-volatile memory such as flash memory-based SSDs, enterprise flash drives, solid state ROMs, SSDs based on volatile memory such as solid state RAM, dynamic RAM, static RAM, DRAM-based SSDs, magnetoresistive RAM (MRAM) SSDs, and hybrid SSDs that use a combination of DRAM and flash memory-based SSDs. The disk drives and their associated computer-readable media may provide non-volatile storage of computer-readable instructions, data structures, program modules and other data for the computer system 1500.

[0214] Machine-readable instructions executable by one or more processors or cores of the processing unit 1504 may be stored on a non-transitory computer-readable storage medium. A non-transitory computer-readable storage medium may include physically tangible memory or storage devices, including volatile memory storage devices and / or non-volatile storage devices. Examples of non-transitory computer-readable storage media include magnetic storage media (e.g., disks or tapes), optical storage media (e.g., DVDs, CDs), various types of RAM, ROM or flash memory, hard drives, floppy drives, removable memory drives (e.g., USB drives), or other types of storage devices.

[0215] The communications subsystem 1524 provides an interface to other computer systems and networks. The communications subsystem 1524 serves as an interface for receiving and transmitting data to and from systems other than the computer system 1500. For example, the communications subsystem 1524 may enable the computer system 1500 to connect to one or more devices via the Internet. In some embodiments, the communications subsystem 1524 may include a radio frequency (RF) transceiver component for accessing wireless voice and / or data networks (e.g., using cellular technology, advanced data network technologies such as 3G, 4G, or EDGE (Enhanced Data Rates for Global Evolution), WiFi (IEEE 802.11 family standard, or other mobile communications technologies, or any combination thereof), a global positioning system (GPS) receiver component, and / or other components. In some embodiments, the communications subsystem 1524 may provide a wired network connection (e.g., Ethernet) in addition to or instead of a wireless interface.

[0216] In some embodiments, the communications subsystem 1524 may also receive incoming communications in the form of structured and / or unstructured data feeds 1526, event streams 1528, event updates 1530, etc., on behalf of one or more users who may use the computer system 1500.

[0217] For example, the communications subsystem 1524 can be configured to receive data feeds 1526 in real time from users of social networks and / or other communications services, such as web feeds, such as Twitter® feeds, Facebook® updates, Rich Site Summary (RSS) feeds, and / or real-time updates from one or more third party sources.

[0218] Additionally, the communications subsystem 1524 may be configured to receive data in the form of a continuous data stream, which may include an event stream 1528 of real-time events and / or event updates 1530, which may be of a continuous or unbounded nature with no apparent end. Examples of applications that generate continuous data may include, for example, sensor data applications, financial tickers, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, automobile traffic monitoring, etc.

[0219] The communications subsystem 1524 may also be configured to output structured and / or unstructured data feeds 1526, event streams 1528, event updates 1530, etc. to one or more databases in communication with one or more streaming data source computers coupled to the computer system 1500.

[0220] The computer system 1500 may be one of a variety of types, including a handheld portable device (e.g., an iPhone® mobile phone, an iPad® computing tablet, a PDA), a wearable device (e.g., a Google Glass® head mounted display), a PC, a workstation, a mainframe, a kiosk, a server rack, or any other data processing system.

[0221] Due to the ever-changing nature of computers and networks, the description of the computer system 1500 shown in the figure is intended as an example only. Many other configurations having more or fewer components than the system shown in the figure are possible. For example, customized hardware may also be used, and / or particular elements may be implemented in hardware, firmware, software (including applets), or a combination thereof. Also, connections to other computing devices, such as network input and / or output devices, may be employed. Based on this disclosure and the teachings presented herein, one of ordinary skill in the art will recognize other ways and / or methods of implementing various embodiments.

[0222] Although specific embodiments have been described, various modifications, variations, alternative constructions and equivalents are encompassed within the scope of the disclosure. The embodiments are not limited to operating in one particular data processing environment, but can freely operate in multiple data processing environments. Furthermore, while the embodiments have been described using a particular sequence of transactions and steps, it will be apparent to those skilled in the art that the scope of the disclosure is not limited to the sequence of transactions and steps described. Various features and aspects of the above-described embodiments can be used individually or in combination.

[0223] Also, while embodiments are described using a particular combination of hardware and software, it should be understood that other combinations of hardware and software are within the scope of the present disclosure. The embodiments can be implemented using only hardware, only software, or a combination thereof. The various processes described herein can be performed on the same processor or on different processors in any combination. Thus, when a component or module is described as being configured to perform an operation, such configuration can be achieved, for example, by designing an electronic circuit to perform the operation, or by programming a programmable electronic circuit (such as a microprocessor) to perform the operation, or a combination thereof. Processes can communicate using a variety of techniques, including, but not limited to, conventional techniques for inter-process communication, and different pairs of processes may use different techniques, or the same pair of processes may use different techniques at different times.

[0224] Accordingly, the specification and drawings are to be regarded in an illustrative and not a restrictive sense. However, it will be apparent that additions, substitutions, deletions and other modifications and alterations may be made thereto without departing from the broader spirit and scope of the appended claims. Thus, although certain disclosed embodiments have been described, they are not intended to be limiting. Various modifications and equivalents are also intended to be encompassed within the scope of the appended claims.

[0225] Use of the terms "a," "an," "the," and similar referents in the context of describing embodiments of the present disclosure (particularly in the context of the appended claims) should be construed to include both the singular and the plural, unless otherwise indicated herein or clearly contradicted by context. The terms "comprising," "having," "including," and "containing" should be construed as open-ended (i.e., meaning "including, but not limited to"), unless otherwise indicated. The term "connected" should be construed as being partly or wholly contained within, attached to, or joined to one another, even if there are intervening elements. The recitation of ranges of values ​​herein is merely intended to serve as a shorthand method of referring individually to each separate value falling within the range, unless otherwise indicated herein, and each separate value is incorporated herein as if it were individually set forth herein. All methods described herein can be performed in any suitable order, unless otherwise indicated herein or clearly contradicted by context. The use of any examples and illustrative language (e.g., "etc.") described herein, unless otherwise required, is intended merely to facilitate a better understanding of the embodiments and does not limit the scope of the disclosure. No language in the specification should be construed as indicating any non-claimed element as essential to the practice of the disclosure.

[0226] Disjunctive phrases such as "at least one of X, Y, or Z" are intended to be interpreted as generally used within the context to indicate that an item, term, etc., can be either X, Y, or Z, or any combination thereof (e.g., X, Y and / or Z), unless specifically stated otherwise. Thus, such disjunctive phrases are generally not intended to, and should not, imply that an embodiment requires that at least one of X, at least one of Y, or at least one of Z, respectively, be present.

[0227] Preferred embodiments of the present disclosure are described herein, including the best mode known for carrying out the present disclosure. Variations of these preferred embodiments will become apparent to those skilled in the art upon reading the above description. Those skilled in the art should be able to adopt such variations as appropriate, and the present disclosure may be carried out in a manner different from that specifically described herein. Accordingly, the present disclosure includes all modifications and equivalents of the subject matter recited in the claims appended hereto as permitted by applicable law. Also, unless otherwise indicated herein, any combination of the above-described elements is encompassed by the present disclosure in all its possible variations.

[0228] All references cited in this specification, including publications, patent applications, and patents, are hereby incorporated by reference to the same extent as if each reference was individually and specifically incorporated by reference and was set forth in its entirety herein.

[0229] Although aspects of the disclosure have been described hereinabove with reference to specific embodiments thereof, those skilled in the art will recognize that the disclosure is not limited thereto. Various features and aspects of the disclosure above can be used individually or in combination. Also, the embodiments can be used in any number of environments and applications other than those described herein without departing from the broader spirit and scope of the specification. Accordingly, the specification and drawings should be regarded as illustrative rather than restrictive.

Claims

1. A method executed by a computer, comprising: A resource identification service in a cloud computing environment obtains a configuration file including resource detection data associated with the service, the resource detection data indicates a set of parameters, and existing resources in the cloud computing environment are identified using the set of parameters, and the method comprises: The method further comprises the resource identification service performing an operation for identifying the existing resources, the existing resources being identified at least in part based on matching attributes associated with the existing resources with the set of parameters of the resource detection data, and the method comprises: The resource identification service identifying a set of import operations to perform to obtain an identifier corresponding to the existing resource; Transmitting the identifier corresponding to the existing resource identified at least in part based on execution of the set of import operations; A method executed by a computer, further comprising the above.

2. The method executed by a computer according to claim 1, wherein the set of parameters includes at least one of a location for searching for the existing resources or a value corresponding to an attribute of the existing resources.

3. The identifier corresponding to the existing resource is transmitted to an orchestration service in the cloud computing environment, and transmitting the identifier to the orchestration service in the cloud computing environment causes the existing resource to be used in the data center instead of generating a new resource. The method executed by a computer according to claim 1.

4. The method executed by a computer according to claim 1, wherein the existing resources include at least one of infrastructure components, data, or applications.

5. The method executed by a computer according to claim 1, wherein the set of import operations is identified from the configuration file.

6. The method further comprises the resource identification service obtaining a second set of parameters from the resource detection data, second existing resources in the cloud computing environment being identified using the second set of parameters, and the method comprises: The resource identification service further includes performing additional operations for identifying the second existing resource, where the second existing resource is identified at least in part based on matching attributes associated with the second existing resource with a second set of parameters of the resource detection data, and the method identifying a second set of import operations that the resource identification service performs to obtain a second identifier corresponding to the second existing resource; providing, to a computing component, the second identifier corresponding to the second existing resource identified at least in part based on execution of the second set of import operations; The method executed by a computer according to claim 1, further comprising. **Claim 7** The method executed by a computer according to claim 1, further comprising executing the set of import operations, where execution of the set of import operations causes the resource identification service to identify an address corresponding to a location of the existing resource or the identifier of the existing resource. **Claim 8** A computing device in a cloud computing environment, including one or more processors, and one or more memories storing computer-executable instructions, where when the instructions are executed by the one or more processors, the instructions cause a resource identification service in the cloud computing environment to obtain a configuration file including resource detection data associated with the service, the resource detection data indicating a set of parameters, and existing resources in the cloud computing environment are identified using the set of parameters; perform operations for identifying the existing resources, where the existing resources are identified at least in part based on matching attributes associated with the existing resources with the set of parameters of the resource detection data; identify a set of import operations performed to obtain an identifier corresponding to the existing resources; send the identifier corresponding to the existing resources identified at least in part based on execution of the set of import operations. **Claim 9** The computing device according to claim 8, wherein the set of parameters includes at least one of a location for searching for the existing resource or a value corresponding to an attribute of the existing resource.

10. The identifier corresponding to the existing resource is sent to an orchestration service of the cloud computing environment, and the sending of the identifier to the orchestration service of the cloud computing environment causes the existing resource to be used in a data center instead of generating a new resource. The computing device according to claim 8.

11. The computing device according to claim 8, wherein the existing resource includes at least one of an infrastructure component, data, or an application.

12. The computing device according to claim 8, wherein the set of import operations is specified from the configuration file.

13. The execution of the operation further causes the resource identification service to obtain a second set of parameters from the resource detection data, and a second existing resource in the cloud computing environment is identified using the second set of parameters. perform additional operations to identify the second existing resource, and the second existing resource is identified based at least in part on matching attributes associated with the second existing resource with the second set of parameters of the resource detection data. specify a second set of import operations to perform to obtain a second identifier corresponding to the second existing resource. send the second identifier corresponding to the second existing resource identified based at least in part on the execution of the second set of import operations. The computing device according to claim 8.

14. The execution of the operation further causes the resource identification service to execute the set of import operations, and the execution of the set of import operations causes the resource identification service to identify an address corresponding to the location of the existing resource or the identifier of the existing resource. The computing device according to claim 8.

15. A computer program which, when executed by one or more processors, causes a resource identification service in a cloud computing environment to execute the method according to any one of claims 1 to 7.

Images