Techniques for detecting drift in deployment orchestrator

Abstract

To provide a method, a storage medium, a system and an apparatus for implementing an infrastructure orchestration service CIOS.SOLUTION: A method of using a safety plan in a CIOS comprises receiving, by at least one of multiple computer processors, a safety plan comprising a list of resources and operations based at least in part on a deployment configuration file. Upon receiving approval of the safety plan, the method comprises preparing for performing an operation corresponding to at least one of the list of resources, and comparing the operation to the safety plan. If the operation is part of the safety plan, the method comprises performing the operation, and if the operation is not part of the safety plan, the method comprises halting the deployment and transmitting, to a user, a notification that the deployment is not in compliance with the safety plan.SELECTED DRAWING: Figure 6

Description

[Background technology]

[0001] REFERENCE TO RELATED APPLICATIONS This application is filed pursuant to 35 U.S.C. § 119(e) and is incorporated herein by reference in its entirety. U.S. application Ser. No. 62 / 963,413, filed September 21, 2020, entitled "TECHNIQUES FOR DETECTING DRIFT IN A DEPLOYMENT ORCHESTRATOR"; This application claims the benefit of and priority to U.S. Application No. 17 / 027,527, entitled "TECHNIQUES FOR MANAGING DRIFT IN A DEPLOYMENT ORCHESTRATOR," filed September 21, 2020, and U.S. Application No. 17 / 027,507, entitled "TECHNIQUES FOR MANAGING DRIFT IN A DEPLOYMENT ORCHESTRATOR," filed September 21, 2020, the entire contents of which are incorporated by reference for all purposes.

[0002] background Today, cloud infrastructure services utilize many individual services to provision and deploy code and configuration across many regions of the cloud infrastructure service. These tools require significant manual effort to use, especially considering that provisioning is largely declarative and deploying code is mandatory. Additionally, as the number of service teams and regions increases, cloud infrastructure services must continue to grow. Some strategies for deploying cloud infrastructure services to more and smaller regions involve spending per region, but these may not scale well. Summary of the Invention

[0003] A brief overview Techniques for executing infrastructure orchestration services are described. In some examples, a safety plan may be received that includes a list of resources and actions based at least in part on a deployment configuration file. Upon receiving approval of the safety plan, an action corresponding to at least one of the list of resources may be prepared to be executed. The action may be compared to the safety plan. If the action is determined to be part of the safety plan, the action may be executed. If the action is determined not to be part of the safety plan, the deployment may be stopped and a notification may be sent that the deployment does not comply with the safety plan. In some examples, if drift occurs, the action may not be part of the safety plan.

[0004] In another example, a computer-readable storage medium may include instructions that, when executed by a processor, can cause the processor to perform various operations described herein. A safety plan may be received, which may include a list of resources and operations based on a deployment configuration file. Approval of the safety plan may be received. An operation corresponding to at least one of the list of resources according to the deployment configuration file may be prepared for execution. The operation may be compared to the safety plan. If the operation is determined to be part of the safety plan, the operation may be executed. If the operation is determined not to be part of the safety plan, the deployment may be stopped and a notification may be sent that the deployment does not comply with the safety plan.

[0005] In yet another example, a system may include a processor and, when executed by the processor, and a memory capable of storing instructions capable of configuring the system to perform the operations described herein. A safety plan may be received that may include a list of resources and operations based at least in part on the deployment configuration file. Approval of the safety plan may be received. Based on the approval, an operation corresponding to at least one of the list of resources may be prepared to be executed. The operation may be compared to the safety plan. If the operation is determined to be part of the safety plan, the operation may be executed. If the operation is determined not to be part of the safety plan, the deployment may be stopped and a notification may be sent that the deployment does not comply with the safety plan.

[0006] In another example, the method may comprise at least one of the plurality of computer processors receiving a safety plan including a list of resources and respective operations based at least in part on a deployment configuration file. The method may further comprise, in accordance with receiving an indication of approval of the safety plan, at least one of the plurality of computer processors preparing to execute an operation corresponding to at least one of the list of resources according to the deployment configuration file. The method may further comprise comparing the operation with the safety plan. The method may further comprise, in accordance with a determination that the operation is part of the safety plan, at least one of the plurality of computer processors executing the operation. The method may further comprise, in accordance with a determination that the operation is not part of the safety plan, at least one of the plurality of computer processors stopping the deployment and at least one of the plurality of computer processors sending a notification that the deployment does not comply with the safety plan.

[0007] In another example, a computer-readable storage medium may include instructions that, when executed by a processor, can cause the processor to perform various operations described herein. The operations may include receiving a safety plan including a list of resources and respective operations based at least in part on a deployment configuration file. The operations may further include, in accordance with receiving an indication of approval of the safety plan, preparing to perform an operation corresponding to at least one of the list of resources according to the deployment configuration file and comparing the operation with the safety plan. The operations may further include, in accordance with determining that the operation is part of the safety plan, performing the operation. The operations may further include, in accordance with determining that the operation is not part of the safety plan, stopping the deployment and sending a notification that the deployment does not comply with the safety plan.

[0008] In another example, a system may include a processor and a memory capable of storing instructions, when executed by the processor, that configure the system to perform the operations described herein. The operations may include receiving a safety plan including a list of resources and respective operations based at least in part on a deployment configuration file. The operations may further include preparing to perform an operation corresponding to at least one of the list of resources according to the deployment configuration file in accordance with receiving an indication of approval of the safety plan. The operations may further include comparing the operation with the safety plan. The operations may further include performing the operation in accordance with a determination that the operation is part of the safety plan. The operations may further include stopping the deployment and sending a notification that the deployment does not comply with the safety plan in accordance with a determination that the operation is not part of the safety plan.

[0009] An apparatus comprising means for the steps of any of the methods described herein. To easily identify the description of any particular element or act, the most significant digit(s) of a reference number refers to the figure number in which that element is first introduced. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram illustrating an architecture for implementing at least some elements of a cloud infrastructure orchestration service, according to at least one embodiment. [Figure 2] FIG. 1 is a block diagram illustrating an architecture for implementing at least some elements of a cloud infrastructure orchestration service, according to at least one embodiment. [Figure 3] FIG. 1 is a flow diagram illustrating an example of a flocking process according to at least one embodiment. [Figure 4] FIG. 1 is a flow diagram illustrating an example of a flocking process according to at least one embodiment. [Figure 5] FIG. 1 is a block diagram illustrating a cloud infrastructure orchestration service according to at least one embodiment. [Figure 6] 1 is a flowchart illustrating a process for using a safety plan in a cloud infrastructure orchestration service, according to at least one embodiment. [Figure 7] 1 is a flowchart illustrating a process for generating a safety plan in a cloud infrastructure orchestration service, according to at least one embodiment. [Figure 8] FIG. 1 is a swim lane diagram illustrating how safety plans are generated and used, according to at least one embodiment. [Figure 9] FIG. 1 is a block diagram illustrating a disconnection region according to at least one embodiment. [Figure 10] 1 is a flowchart illustrating a process for using a safety plan in a disconnected area, according to at least one embodiment. [Figure 11] FIG. 1 is a block diagram illustrating a distributed system according to at least one embodiment. [Figure 12] A block diagram illustrating one or more components of a system environment in which services provided by one or more components of an embodiment system may be provided as cloud services, according to at least one embodiment. [Figure 13] FIG. 1 is a block diagram illustrating an example of a computer system on which various embodiments of the present disclosure may be implemented. DETAILED DESCRIPTION OF THE INVENTION

[0011] Detailed Description In some examples, Infrastructure as a Service (IaaS) is one specific type of cloud computing. IaaS can be configured to deliver virtualized computing resources over a public network (e.g., the Internet). In some examples, IaaS is one of three major categories (or subcategories) of cloud computing services. Most people consider the other major categories to be Software as a Service (SaaS) and Platform as a Service (PaaS), and SaaS is sometimes considered a broader category that encompasses both PaaS and IaaS, and some even consider IaaS to be a subcategory of PaaS.

[0012] In the IaaS model, a cloud computing provider may host infrastructure components (e.g., servers, storage devices, network nodes (e.g., hardware), deployment software, or platform virtualization (e.g., hypervisor layer)).

[0013] In some cases, IaaS providers may also provide various services (e.g., billing, monitoring, logging, security, load balancing, and clustering) to accompany these infrastructure components. Because the services can be policy-driven, IaaS users can implement policies to drive load balancing and maintain application availability and performance.

[0014] In some examples, IaaS customers can access resources and services over a wide area network (WAN), such as the Internet, and use the cloud provider's services to install the remaining elements of their application stack. For example, a user can log into an IaaS platform, create virtual machines (VMs), install an operating system (OS) on each VM, deploy middleware (such as a database), create storage buckets for workloads and backups, and even install enterprise software on the VMs. The customer can then use the provider's services to perform a variety of functions, such as network traffic balancing, troubleshooting application issues, monitoring performance, and managing disaster recovery.

[0015] In most cases, the cloud computing model requires the participation of a cloud provider. A cloud provider can be, but does not have to be, a third-party service that specializes in providing (e.g., selling) IaaS. An entity may also choose to deploy a private cloud in which it becomes its own infrastructure service provider.

[0016] In some examples, IaaS deployment is the process of placing a new application or a new version onto a prepared application server, etc. It may also include the process of preparing the server (e.g., installing libraries, daemons, etc.), which is often managed by the cloud provider below the hypervisor layer (e.g., server, storage, network hardware, and virtualization). Thus, the customer can be responsible for the (OS), middleware, and / or application deployment (e.g., on self-service virtual machines (e.g., which can be spun up on demand)).

[0017] In some instances, IaaS provisioning may even mean obtaining computers or virtual hosts for use and installing required libraries or services on them. In most cases, deployment does not include provisioning, which may need to be performed first.

[0018] In some cases, IaaS provisioning presents two distinct problems. First, there is the initial challenge of provisioning the initial set of infrastructure before anything is running. Second, there is the challenge of deploying the existing infrastructure once everything is provisioned (e.g., adding new services, modifying services, removing services, etc.). In some cases, these two challenges can be addressed by allowing the configuration of the infrastructure to be defined declaratively. In other words, the infrastructure (e.g., which components are needed and how they interact) can be defined by one or more configuration files. Thus, the entire topology of the infrastructure (e.g., which resources depend on which and how each of them works together) can be described declaratively. In some instances, once the topology is defined, workflows can be generated to create and / or manage the different components described in the configuration files.

[0019] In some examples, the infrastructure may have many interconnected elements, such as one or more virtual private clouds, also known as a core network. There may be a Virtual Private Network (VPC) (e.g., a configurable and / or shared, possibly on-demand pool of computing resources). In some examples, there may also be one or more security group rules provisioned to define how the network's security is set up, and one or more virtual machines (VMs). Other infrastructure elements, such as load balancers or databases, may also be provisioned. The infrastructure may be deployed in stages as more infrastructure elements are required and / or added.

[0020] As mentioned above, one way to provision infrastructure is to describe it declaratively. As such, a configuration file may be a declarative file that simply describes each of the infrastructure components described above and how they interact. A configuration file may describe resources and associated fields needed to create elements, and may also describe other elements that reference previously described elements. In some examples, a provisioning tool may then generate workflows for creating and managing the elements described in the configuration file.

[0021] In some examples, the provisioning tool workflow can be configured to execute various commands. One function that can be executed is view adjustment, where the provisioning tool can compare a view of the current infrastructure (e.g., the expected state of the infrastructure) with how the infrastructure is actually running. In some examples, executing the view adjustment function can include querying various resource providers or infrastructure resources to identify which resources are actually running. Another function that the provisioning tool can execute is plan generation, where the provisioning tool can compare the infrastructure components that are actually running with how the provisioning tool wants them to appear (e.g., the desired configuration). In other words, the plan generation function can determine what changes need to be made to bring the resources closer to the latest expectations. In some examples, a third function is execution (e.g., apply) function, where the provisioning tool can execute the plan generated by the plan generation function.

[0022] In general, a provisioner can be configured to receive a configuration file, parse the declarative information contained therein, and programmatically / automatically determine the order in which resources need to be provisioned to execute the plan. For example, if a VPC needs to be booted before security group rules and VMs are booted, the provisioner can make that determination and execute the boots in that order without user intervention and / or even though that information is not necessarily included in the configuration file.

[0023] In some examples, continuous deployment techniques may be employed to enable deployment of infrastructure code across various virtual computing environments. Additionally, the described techniques may enable infrastructure management within these environments. In some examples, a service team may write code that is desired to be deployed to one or more (but often many) different production environments (e.g., spanning a variety of different geographic locations, sometimes spanning the entire world). However, in some examples, the infrastructure into which the code will be deployed must first be configured. In some examples, provisioning may be done manually, and a provisioning tool may be utilized to provision resources, and / or deployment may be performed once the infrastructure is provisioned. You can deploy the code using the deployment tool.

[0024] As mentioned above, there are typically two different tools used to handle the provisioning of infrastructure resources and the deployment of code to control the infrastructure resources, and the orchestration between these two tools is performed manually. However, at scale, manual execution always leads to mismatches. Therefore, an automated tool that can both provision and deploy virtual infrastructure enables a more efficient and reliable technique for realizing virtual cloud environments.

[0025] In some examples, when two tools are used, problems can arise when a user manually makes changes to code between the provisioning and deployment phases. As described herein, techniques that use a single tool for both provisioning and deployment can mitigate that problem by automating the process so that there is no opportunity for manual code changes. A small change to the way a user codes something can cause major problems in the deployment phase. In some examples, the first time an operator performs an action in a new area (e.g., a typo in the code), the object coded with the typo may remain there forever. If an application is deployed with the typo but the application is not affected by the typo (e.g., the application still functions), it may happen that further code changes in the future are affected by the typo and cause the entire system to crash. Thus, the techniques provided herein can eliminate the gap between provisioning and deployment, which can often cause problems.

[0026] Typically, deployment modeling is declarative, allowing infrastructure resources to be declared using configuration files. For example, deployment files are typically generated using generic representable state transfer (REST) ​​concepts (e.g., REST application programming interfaces (APIs)) using create-read-update-delete (CRUD) instructions. However, deployment itself does not typically follow this concept. Furthermore, while infrastructure provisioning tools tend to be very powerful and / or expressive, tools for deployment tend to be much more restrictive in terms of the operations they can perform (e.g., they are imperative as opposed to declarative). Therefore, there has been a long-standing need for tools that can address both functional requirements (e.g., provisioning and deployment of infrastructure elements) within cloud environments.

[0027] In some examples, techniques for implementing a cloud infrastructure orchestration service (CIOS) are described herein. Such techniques, as briefly described above, can be configured to manage both the provisioning and deployment of infrastructure assets within a cloud environment. In some examples, a CIOS can include two classes of services: a central component and a regional component (e.g., a central CIOS and a regional CIOS). The following terms are used throughout the specification:

[0028] Infrastructure components: long-lived pieces of infrastructure that support running code Examples: deployment applications, load balancers, Domain Name System (DNS) entries, object storage buckets, etc.

[0029] Artifact: Code deployed to a deployment application or Kubernetes engine cluster, or configuration information (hereafter "config") applied to an infrastructure component. These can be read-only resources.

[0030] Deployment Tasks: Short-lived tasks that are often associated with deploying or testing code. Furthermore, deployment tasks are modeled as resources that do not live as long as the releases that create them.

[0031] For example, "deploy $artifact to $environment", "watch $alarm for 10 minutes", "execute $testSuite", or "wait for $manualApproval" For example, CIOS can model a deployment orchestrator's deployment as the creation of a resource that transitions to an available state when it is complete.

[0032] Because CIOS maintains the state of its associated declarative provisioners, CIOS can control the lifecycle of these short-lived resources when it is associated with a release.

[0033] Resource: A CRUD-enabled resource CIOS models each of the above constructs as a resource, and the next section explains this modeling in more detail.

[0034] Flock: The only control plane resource in CIOS. It exists primarily to represent infrastructure components by modeling their ownership.

[0035] Flock config: Describes the set of all infrastructure components, artifacts, and deployment tasks associated with a single service.

[0036] Each flock has exactly one flock config, which is checked into source control.

[0037] Flock configs are declarative: they expect CIOS to provide the domain, region, ad, and artifact version as input.

[0038] A flock is fine-grained: it consists of a single service and supporting infrastructure.

[0039] · State: A snapshot of the state of all resources in a flock at a particular point in time. Release: A tuple of a specific version of a Flock config and the specific versions of any artifacts it references. A release is considered to describe a state that may not yet exist.

[0040] Release Plan: The set of steps that CIOS will take to transition all regions from their current state to the state described by the release.

[0041] A release plan consists of a finite number of steps and clearly defined start and end times. It has the following.

[0042] Apply: This is a noun. It is a single attempt to execute a release plan. An execution changes the current state of the flock.

[0043] CIOS can be described as an orchestration layer that applies configuration to downstream systems (e.g., globally). It is designed to enable infrastructure provisioning and code deployments globally without manual effort from service teams (e.g., potentially beyond initial approval). High-level responsibilities of CIOS include, but are not limited to:

[0044] · Providing the team with a view into the current state of resources managed by the CIOS, including any ongoing change activity.

[0045] · Helping the team plan and release new changes. · Coordinate activities across various downstream systems within the region to execute approved release plans without human intervention.

[0046] ·Coordinate activities across regions / territories to execute approved release plans globally.

[0047] It handles onboarding by allowing information to be provided to CIOS. This is more work than previous implementations, as CIOS can also automate many more things. In some examples, CIOS handles pre-deployment by providing teams with the ability to automatically deploy and test code. In some examples, CIOS can handle writing change management (CM) policies by allowing teams to automatically generate plans for rolling out new artifacts (e.g., globally) as they build them. This can be done by examining the current state of each region and the current CIOS config (which can itself be an artifact). Furthermore, teams can examine these plans and iterate them by changing the CIOS config and asking CIOS to replan. Once the team is satisfied with the plan, they can create a "release" that references the plan. This plan can then be marked as approved or rejected. Teams can still write CMs, but they are merely pointers to the CIOS plan. Therefore, teams spend less time reasoning about the plan. The plans are more accurate because they are machine-generated. The plan is too detailed for humans to read, but can be displayed using a sophisticated user interface (UI).

[0048] In some examples, the CIOS can handle CM execution by automatically executing deployment plans. Once a release plan is created and approved, engineers are not involved in CM unless the CIOS initiates the rollback. In some cases, this requires teams to automate tasks that are currently manual. In some examples, the CIOS can handle change management (CM) rollback by automatically generating a plan to return a flock to its original (e.g., pre-release) state when the CIOS detects a degradation in service health during execution. In some examples, the CIOS can handle the deployment of sudden / tactical changes by receiving a release plan that is scoped to a subset of regions and / or resources managed by the CIOS, and then executing that plan.

[0049] Additionally, CIOS may support the primitives necessary to define fully automated, worldwide deployments. For example, CIOS can measure service health by monitoring alarms and running integration tests. CIOS can help teams quickly define rollback behavior in the event of service degradation, which can then be executed automatically. CIOS can automatically generate and display release plans and track approvals. In some examples, the language teams use to describe desired deployment behavior can be declarative. CIOS can combine the functionality of code deployment and infrastructure configuration (e.g., provisioning) in one system. CIOS also supports flexible ordering across regions and across components within a region. Teams can express ordering through checked-in configurations. Teams can invoke CIOS planning and release APIs programmatically.

[0050] 1 illustrates an architecture 100 for describing a technique for implementing at least a central CIOS 102. In some examples, the central CIOS 102 may be a service that handles operations at the level of a "flock." The central CIOS 102 has several responsibilities, including but not limited to:

[0051] Acting as an authentication gateway for flock metadata changes and release operations.

[0052] Storing deployment artifacts for Flock and a formal mapping of Flock metadata to the CIOS repository.

[0053] · Coordinate global releases across phases and targets. · Synchronization to enforce policies like "only one in-progress release to a flock at a time."

[0054] Detecting changes to flock configurations (configs) and artifacts and triggering release generation on such changes.

[0055] In some examples, a Source Code Version Control Management Service (SCVMS) 104 can be configured to store the official flock configuration, and an Artifact Notification Service (ANS) 106 can be subscribed to by the central CIOS 102 so that it can be notified of the construction of new artifacts. The central CIOS 102 can then map incoming changes against affected flocks and initiate release planning if desired. Additionally, in some examples, an Artifact Push Service (APS) can be invoked by the central CIOS 102 prior to a release to a target to ensure that any artifacts required for a successful release are present in the target's locale prior to the release.

[0056] In some examples, customers (e.g., engineers) 108 can call the central CIOS 102 to CRUD flocks and / or releases to see the status of ongoing CIOS activity. A Flock Management Service 110 can include one or more APIs for manipulating flocks, a View / Plan / Approve Service 112 can include CRUD APIs for creating and approving plans and viewing a central copy of the state of resources managed by all CIOSs, a Change Monitoring Service 114 can watch the SCVMS 104 for changes to flock configurations and can receive notifications from the ANS 106 about changes to other artifacts, The State Ingester Service 116 can create copies of the regional states in a central CIOS database (DB) 118 so that the Views / Plans / Approvals 112 can publish them. In some examples, the central CIOS DB 118 can be a DB of flocks, plans, and states. The flock information can be authoritative, while everything else can be an older copy of data from the regional CIOS 120.

[0057] In some examples, engineers 108 can make API calls to the flock management service 110 (e.g., via the ingress proxy fleet 122) to create a list of flocks. The protocol for making such API calls can be Hypertext Transport Protocol Secure (HTTPS), etc. The associated access control list (ACL) for this operation can include a local area network (LAN) 124 or other private connection. For example, the CIOS can manage / control network connections (e.g., dedicated, leased, and / or private connections) that replace the use of the public Internet to connect a customer's on-premises data center or network with the CIOS. Additionally, authentication and authorization (e.g., of engineers 108) can be performed by a reservation system portal that allows users to manage machine interfaces (e.g., reservation services). In some examples, the central CIOS 102 can store flock metadata, plans, and states in the central DB 118 using Java Database Connectivity (JDBC), etc. In some examples, the ANS 106 can be configured to notify the change monitoring service 114 when new artifacts are published. The ANS 106 can use HTTPS, and both authentication and authorization can be handled by Mutual Transport Layer Security Services. Additionally, in some examples, the Integrity Monitoring Service 114 can poll the SCVMS 104 for block configuration changes. This polling can be performed using Secure Shell (SSH) or other protocols. Authentication for the Integrity Monitoring Service 114 can be handled by a CIOS system account, and authorization can be handled by the SCVMS 104.

[0058] In some examples, engineers 108 can use the view / plan / approval service 112 to perform one or more of the following operations: Engineers 108 can plan and / or approve by calling the central CIOS 102 to generate and approve plans. Engineers 108 can view by calling the central CIOS 102 to see the status of ongoing CIOS activities worldwide. Additionally, engineers 108 can use the central CIOS 102 to browse replicas of the state of resources managed by the global CIOS. These API calls (or the like) can be made via HTTPS or a similar protocol. Additionally, associated ACLs can be controlled by the LAN 124, and authentication and authorization can be handled by the reservation service. In some examples, the view / plan / approval service 112 can request a plan (e.g., using HTTPS, etc.) and push approval of the plan to all regions of the regional CIOS 120. Associated ACLs can be controlled using security lists managed by the wide area network (WAN) gateway 126. Authentication can be handled by mutual transport layer security, and authorization can be handled by various identity policies. Additionally, the state ingestor service 116 can monitor the regional CIOS 120 for job status or state changes so that the CIOS can provide a central view of those statuses or state changes upon request (e.g., using HTTPS, etc.). ACLS for this can also be handled by the WAN gateway 126, and both authentication and authorization can be handled by mutual transport layer security services.

[0059] 2 is a diagram illustrating an architecture 200 for describing techniques for executing at least a regional CIOS 202. In some examples, the regional CIOS 202 is where much of the declarative provisioning and planning work, as well as approved release applications, can occur. In some examples, each instance of the regional CIOS 202 can have a regional front end that can handle operations at the level of an "execution target." It can be configured to do the following:

[0060] · Handling all CIOS authentication for incoming actions from the central CIOS 102.

[0061] · Enforcing the rule that for a given execution target, only one "execution" (plan / resource import / apply plan) can occur at a time.

[0062] Managing binary artifact storage for declarative provisioning artifacts used as inputs and outputs during declarative infrastructure provisioning runs. Examples of inputs are declarative infrastructure provisioning configuration files and input state files. A typical output is a final state file.

[0063] · Requesting work from the CIOS executor and polling for results from the CIOS executor for any given execution.

[0064] In some examples, the CIOS front end may rely on a CIOS executor, which can handle the actual execution. In some examples, the CIOS executor operates at the level of "execution", which can:

[0065] Keeping track of the pool of available worker nodes · Querying incoming job requests and assigning them to qualified workers as they become available Tracking worker status and execution updates for reporting to clients Dead nodes can be detected by the leasing protocol and tasks assigned to the dead node can be failed depending on the task status. Provide functionality to cancel / kill / pause / resume execution, and be able to map these to functionality to pass the cancel / kill / resume information to worker nodes. In some examples, a CIOS executor may rely on a CIOS worker, which may assign tasks to workers for execution and provide functions for workers to update job progress. Worker services operate at the granularity of a "task". Each worker is an agent that executes the tasks assigned to it and reports task status and output. Each worker can:

[0066] Polling the Execute Worker API for assigned work items and taking action to reconcile the assignment state with its local state Starting a container to poll for task items that do not exist locally Killing a container for locally running containers that do not have a corresponding assigned task item Reporting job status -Performing stage input / output for job container execution Launching and monitoring declarative infrastructure provisioning containers to do the actual work of releasing execution targets A CIOS worker may depend on a CIOS executor to poll for work from the CIOS executor and report results to the CIOS executor's worker endpoint. The worker may rely on the executor for all coordination. Additionally, a CIOS worker may depend on regional CIOS 202, where the worker service reads input from one or more APIs associated with the regional front-end service and writes output to these one or more APIs. Examples of inputs are configuration and start state files, and import of mappings. Examples of outputs are the declarative provisioning process, the output of the declarative provisioning state file, and the import of the result state.

[0067] In some examples, regional CIOS 202 can be a regional service for managing regional instances / deployments of CIOS. Regional CIOS 202 covers the responsibility of authoritatively storing and managing plans and states related to a particular region. Regional DB 204 can be the CIOS DB for the state and plans in that particular region. This is an authoritative copy of the regional subset of central DB 118 of FIG. 1. Scheduler 206 can be responsible for managing worker fleet capacity, assigning tasks to workers, and tracking task state progress. In some examples, task DB 208 is another CIOS DB for task state. The data in this DB is mostly for operational purposes. Additionally, workers 210 can be a fleet of Java Virtual Machines (JVMs) that manage declarative provisioning images. They receive instructions from scheduler 206 and communicate results to both scheduler 206 and regional CIOS 202. CIOS container 212 can execute declarative provisioning actions in its own private Docker 214 container. This container may not contain secrets. Additionally, in some examples, the signing proxy 216 can be configured to prevent secrets from being exfiltrated by declarative provisioning tools to avoid including secrets in the declarative provisioning image. Instead, the CIOS can perform the signing request or initiate a mutual Transport Layer Security (mTLS) service at the proxy. This facilitates the use of FIPS-compliant cryptographic libraries.

[0068] In some examples, central CIOS 102 can call regional CIOS 202 to create plans, push approvals, watch job status (service principals), and extract declarative provisioner state (service principals). Ingress proxy 218 can be configured as an ACL, and various identity policies can be used for both authentication and authorization. In some examples, regional CIOS 202 can run a declarative provisioner by asking scheduler 206 to do so. Worker 210 can ask scheduler 206 what it is supposed to be running and can report status to scheduler 206 upon completion. In some cases, mTLS may handle both authentication and authorization for regional CIOS 202 and worker 210. Furthermore, when worker 210 needs to run a declarative provisioner, it runs the declarative provisioner in a Docker container by interacting with local Docker 214. Authentication in this phase can be handled by a local unix socket. This final step may use the Docker protocol, while the previous steps may utilize HTTPS.

[0069] In some examples, the CIOS container 212 allows the declarative provisioner to interact (via an API) with the signing proxy 216, which the declarative provisioner thinks it is invoking various CIOS services. The signing proxy 216 listens on one ephemeral port per invocation of the declarative provisioner instance, known only to the declarative provisioner. The signing proxy 216 sends signing or mTLS requests. The signing proxy 216 may initiate a declarative provisioner call and pass the declarative provisioner call to other CIOS services within the service enclave. In some examples, the signing proxy 216 may also communicate with one or more public CIOS services 220. For example, the signing proxy 216 uses the internal endpoint of the public service when possible. Services that do not have an internal endpoint must use the egress proxy 222 to reach an external endpoint. This use of the signing proxy 216 may not be for cross-region communication. For example, each region's egress proxy whitelist may be only for that region's public IP ranges. In some examples, the worker 210 may then maintain state and logs from the declarative provisioners in the regional CIOS 202 so that they can be flushed to the central CIOS 102.

[0070] Using CIOS, a typical customer experience has several phases: onboarding, pre-release, worldwide release, and tactical release. For the pre-release phase, the following is an example of what happens between building a new artifact and releasing the artifact to Release 1 (e.g., R1). This should replace some or most of the current change management process. Once the relevant artifacts are built, CIOS can automatically generate releases using "all the latest versions in the flock." A release is a specific version of a flock configuration with specific inputs (e.g., artifact version, region, area, and ad). A release contains one rollforward plan per region and metadata describing the region ordering. Each region plan is a set of actions that the declarative provisioner will take to realize the flock configuration in that region. Teams with pre-release environments can use CIOS to automatically release and test software in these environments. Teams can configure CIOS to automatically test rollback plans. Teams will be able to inspect and approve releases via the CIOS UI. Teams can approve some (but not all) of the local plans within a release. If "latest version of everything" does not result in a suitable plan, the team can ask CIOS to generate plans for cherry-picked artifact versions.

[0071] For the worldwide release phase, the following is an example of how a team might execute tomorrow's version of today's "normal CM." Once a release is approved, the CIOS pushes each approved regional plan to its respective region. The CIOS operates independently within each region to apply the approved plan. The CIOS only executes the set of actions explicitly described in that region's plan. Rather than "thinking independently," it will fail. The CIOS UI shows the team the progress of the execution. The CIOS UI prompts the team if manual approval is required. If the execution fails due to an outage in the CIOS or a downstream service, the CIOS can notify the team and prompt them for next steps (e.g., abort, retry). The CIOS performs retries, but some downstream outages may exceed its willingness to retry. If the execution fails due to a degradation in service health or a test failure, the CIOS assists the team by rolling back the flock to its starting state. The CIOS notifies the team (e.g., paging) when it initiates an automatic rollback. The team must approve the rollback plan, and then the CIOS executes it.

[0072] For the Tactical Release phase, below is an example of how a team could execute tomorrow's version of "Sudden CM". When generating the plan, the team A team may ask CIOS to target plans to specific resources in any way: topologically (e.g., region, region, AD, etc.), by resource type (e.g., "metrics config only" or "deployment orchestration service deployments only"), or a combination of the above (e.g., in a disjunctive manner). Teams approve tactical releases as well as global releases. CIOS orchestrates them similarly. If a team needs to deploy a tactical release while there is an active global release, CIOS will stop the global release execution in the target region and then start the tactical release execution.

[0073] In some instances, the declarative provisioner's state (e.g., traditionally a file) is the authoritative record of the set of resources managed by the declarative provisioner. It contains a mapping between each resource's logical identifier from the configuration and the resource's actual identifier. When the declarative provisioner creates a resource, certain types of failures can occur that prevent the actual identifier from being recorded in the state. When this happens, the actual identifier is lost to the declarative provisioner. These are sometimes called "orphaned resources".

[0074] For most resources, orphans mean waste: the declarative provisioner will (for example) launch an instance it forgot about, only to launch another instance instead the next time it runs. For resources with uniqueness constraints or client-supplied identifiers, orphans prevent the declarative provisioner from moving forward. For example, if the declarative provisioner creates a user "nglass" and then fails to orphan it, the next run of the declarative provisioner will attempt to create "nglass" and fail because a user with that username already exists. In some cases, orphans are only a problem when adding new resources to the state. In some instances, the declarative provisioner's refresh behavior may naturally recover from failures to record updates and deletions.

[0075] CIOS must be robust in the event of outages of downstream services or of the CIOS itself. Since the CIOS can leverage the declarative provisioner to apply changes, this means that it must be robust in running the declarative provisioner and maintaining the declarative provisioner state. The declarative provisioner provider performs "minor" retries sufficient to avoid outages lasting several minutes. For example, a cloud provider may perform retries for a maximum of 30 minutes. If the downstream system is outage for longer than 30 minutes, the declarative provisioner fails. Upon failure, the declarative provisioner records all successful changes to its state and exits. To retry, the CIOS must re-run the declarative provisioner. Re-running the declarative provisioner allows the CIOS to perform retries in the event that the CIOS itself fails. In some examples, the CIOS may perform the following actions in a loop:

[0076] · Refresh: The declarative provisioner calls the "Get API" to retrieve a new snapshot of any resources listed in its state.

[0077] · Plan: The declarative provisioner generates a plan (a concrete set of API calls) that achieves the desired state, given the recently refreshed current state.

[0078] · Apply: The declarative provisioner executes a set of steps in the plan. CIOS may always perform all three of these steps when running a declarative provisioner. A refresh operation removes any updates or deletions that were not recorded. It also helps with recovery. The CIOS examines the results of the plan actions and compares them to the approved release plan. If the newly generated plan contains actions that were not in the approved release plan, the CIOS may fail and notify the service team.

[0079] Figure 3 illustrates a directed acyclic graph (DAG) 300 for an example flock 302. For a single flock config in CIOS, the progression of code / config from check-in to production can be described, from the initial test deployment to the final prod deployment. Internally, CIOS calls each element in the progression an execution target (ET), which spans our internal APIs but does not leak to the flock config. CIOS executes the ETs based on the DAG 200 defined in the flock config. Each ET (e.g., ET-1, ET-2, ET-3, ET-4, ET-5, ET-6, and ET-7) is roughly one copy of the service described by the flock config.

[0080] Figure 4 illustrates DAG 400 to illustrate an example flock 402. In a flock configuration, CIOS is very opinionated about how teams express this progression. Teams must model it using cloud infrastructure tenancies and regions. Teams should not model progression using regions. CIOS allows teams to use many tenancies within a region and many regions within a tenancy. However, CIOS does not allow teams to use the same region twice within a tenancy (although it is possible to use the same region twice within a region in different tenancies). DAG 400 shows a version of DAG 300 from Figure 3 expressed with tenancies and regions. This example is for an overlay service whose pre-prod ET is in the prod region. A service enclave service would have an unstable tenancy and a stable tenancy in release 1. In DAG400, IAD is the regional airport code for Dulles Airport in Washington, DC, YYZ is the regional airport code for Toronto, Ontario, PHX, LHR and FRA are the regional airport codes for Phoenix, London and Frankfurt respectively, and LUF and LFI are for two different air force bases.

[0081] In one embodiment, the CIOS and / or other technologies described herein are improvements to each of Terraform (a declarative provisioning tool), Tanden (a code generation tool), and ODO (Oracle Deployment Orchestrator). Additionally, in some examples, the CIOS and / or other technologies described herein may be implemented using at least a portion of the Terraform, Tanden, and ODO tools.

[0082] In some examples, a safety plan can be generated to safely handle drift. A configuration file can contain information that directs how the CIOS makes changes to a region (e.g., a geographic region where the cloud infrastructure is (or has been) configured). The CIOS can verify that what the client reviews as a change set (e.g., the delta between the current state of the cloud infrastructure and the expected new state after the changes) actually changes within the region. When a client interacts with the CIOS to generate a global plan, the safety plan identifies what changes are expected to be made by the CIOS based on the current world (e.g., current state) to reach the world (e.g., target state) described in the configuration file. As described herein, "world" can also refer to the expected state of the cloud infrastructure, including all settings and configurations of the cloud setup. In some cases, drift is the idea that one or more device configurations / settings may have changed between the time the plan is generated and the time the regional deployment begins (e.g., (For example, a virtual machine may have failed). That change is the drift. A CIOS can create a safety plan and send it to various regions around the world (for example, send it to various different regional CIOSs). Whenever a CIOS is about to implement a change, it can repeat the planning task. For example, it can predict the current state, compare it to the target state, and generate a diff (for example, delta, difference) between the two. That diff is the safety plan's subsection. If the safety plan is a subset of the first safety plan, drift has not occurred and the safety plan can still be implemented. Otherwise, an issue with the safety plan has been detected and changes need to be made before deployment. Additionally, in some examples, the CIOS can compare different safety plans (e.g., to handle phase changes). For example, for a phased deployment (e.g., deploying resources in the first phase before other phases), the safety plan may be reviewed and approved, and the deployment may be performed accordingly. Then, later, for a phase 2 deployment, a new safety plan may be generated and compared with the first safety plan. If the new safety plan is a subset of the first safety plan, the new safety plan may be automatically approved without sending anything to the client that generated the configuration file. It will be appreciated that this can speed up the deployment process because no delays are introduced associated with waiting for approval of the new safety plan, but also maintain the integrity of the deployment because drift has not occurred in such a situation. Alternatively, if there is something new in the new safety plan (e.g., something not present in the first safety plan), that new aspect can be presented to the user (e.g., the client) via a user interface (UI). The user can then approve or reject the new aspects (eg, allow or not allow the deployment changes to occur).

[0083] Techniques for executing infrastructure orchestration services are described. In some examples, a safety plan may be received that includes a list of resources and actions based at least in part on a deployment configuration file. Upon receiving approval of the safety plan, an action corresponding to at least one of the list of resources may be prepared to be executed. The action may be compared to the safety plan. If the action is determined to be part of the safety plan, the action may be executed. If the action is determined not to be part of the safety plan, the deployment may be stopped and a notification may be sent that the deployment is not compliant with the safety plan. In some examples, if drift occurs, the action may not be part of the safety plan.

[0084] In another example, a computer-readable storage medium may include instructions that, when executed by a processor, can cause the processor to perform various operations described herein. A safety plan may be received, which may include a list of resources and operations based on a deployment configuration file. Approval of the safety plan may be received. An operation corresponding to at least one of the list of resources according to the deployment configuration file may be prepared for execution. The operation may be compared to the safety plan. If the operation is determined to be part of the safety plan, the operation may be executed. If the operation is determined not to be part of the safety plan, the deployment may be stopped and a notification may be sent that the deployment does not comply with the safety plan.

[0085] In yet another example, a system includes a processor and, when executed by the processor, and a memory capable of storing instructions capable of configuring the system to perform the operations described herein. A safety plan may be received that may include a list of resources and operations based at least in part on the deployment configuration file. Approval of the safety plan may be received. Based on the approval, an operation corresponding to at least one of the list of resources may be arranged to be performed. The operation may be compared to the safety plan. The operation may be compared to the safety plan. If it is determined that the action is part of the safety plan, the action can be performed. If it is determined that the action is not part of the safety plan, the deployment can be stopped and a notification can be sent that the deployment is not compliant with the safety plan.

[0086] In another example, the method may include at least one of the plurality of computer processors receiving a safety plan including a list of resources and respective operations based at least in part on a deployment configuration file. The method may further include, in accordance with receiving an indication of approval of the safety plan, at least one of the plurality of computer processors preparing to execute an operation corresponding to at least one of the list of resources according to the deployment configuration file. The method may further include comparing the operation with the safety plan. The method may further include, in accordance with a determination that the operation is part of the safety plan, at least one of the plurality of computer processors executing the operation. The method may further include, in accordance with a determination that the operation is not part of the safety plan, at least one of the plurality of computer processors stopping the deployment and at least one of the plurality of computer processors sending a notification that the deployment does not comply with the safety plan.

[0087] In another example, a computer-readable storage medium may include instructions that, when executed by a processor, cause the processor to perform various operations described herein. The operations may include receiving a safety plan including a list of resources and respective operations based at least in part on a deployment configuration file. The operations may further include, in accordance with receiving an indication of approval of the safety plan, preparing to perform an operation corresponding to at least one of the list of resources according to the deployment configuration file and comparing the operation with the safety plan. The operations may further include, in accordance with determining that the operation is part of the safety plan, performing the operation. The operations may further include, in accordance with determining that the operation is not part of the safety plan, stopping the deployment and sending a notification that the deployment does not comply with the safety plan.

[0088] In another example, a system may include a processor and a memory capable of storing instructions that, when executed by the processor, configure the system to perform the operations described herein. The operations may include receiving a safety plan including a list of resources and respective operations based at least in part on a deployment configuration file. The operations may further include, in accordance with receiving an indication of approval of the safety plan, preparing to perform an operation corresponding to at least one of the list of resources according to the deployment configuration file. The operations may further include comparing the operation with the safety plan. The operations may further include, in accordance with a determination that the operation is part of the safety plan, performing the operation. The operations may further include, in accordance with a determination that the operation is not part of the safety plan, stopping the deployment and sending a notification that the deployment does not comply with the safety plan.

[0089] FIG. 5 is a block diagram 500 illustrating a CIOS capable of generating and using safety plans, according to at least one embodiment. In some examples, a safety plan may be a set of actions approved by a user and executable by the CIOS. Block diagram 500 illustrates an example architecture of a CIOS that may include a central control plane 502 and a regional data plane 504. Central control plane 502 may include a central CIOS 506 (e.g., similar to central CIOS 102 of FIG. 1 ), which may include a control plane 508, a change management module 510, a view / plan / approval module 512, and a state ingestor 514. Central control plane 508 and view / plan / approval module 512 are communicatively coupled to a local area network (LAN) gateway 516 that users 518 can use to communicate with central CIOS 506. The view / plan / approval module 512 and the state ingestor 514 may be communicatively coupled to a wide area network (WAN) 520 (e.g., the Internet), which may be communicatively coupled to a regional CIOS 522 (e.g., similar to the regional CIOS 120 of FIG. 1 or the regional CIOS 202 of FIG. 2) included in the regional data plane 504.

[0090] Regional CIOS 522 may be communicatively coupled to a scheduler node 524 and a worker node 526. Worker 526 may be communicatively coupled to scheduler 524, which may be communicatively coupled to a task database (DB) 528. Worker 526 may be communicatively coupled to a docker 530, which may include a CIOS container 532. CIOS container 532 may be communicatively coupled to a signing proxy 534, which may be communicatively coupled to a cloud service 536.

[0091] Regional CIOS 522 can receive tasks from central CIOS 506 and can send the tasks to scheduler 524. The tasks may include performing CRUD operations or any other suitable tasks for deploying infrastructure resources in the region(s) of the execution target. Scheduler 524 can record the tasks in task DB 528 and can send the tasks to workers 526, which can be included in a worker fleet. A worker fleet can include many workers 526, and scheduler 524 can select a worker 526 with the least amount of work or the most amount of available computing resources to assign the task to. Scheduler 524 can assign one task to a worker 526 at a time. Worker 526 can execute the tasks, and in executing the tasks, the worker can make calls to CIOS containers 532 included in Docker 530. CIOS containers 532 can execute tasks including infrastructure provisioning and / or deployment instructions (e.g., Terraform instructions). The instructions may direct the API call to cloud services 536, which may include locally available services that may not be available over a public network (e.g., the Internet). To make the API call to cloud services 536, CIOS container 532 may send a request to make the API call to signing proxy 534, which may determine whether the request is valid. In response to determining that the request is valid, signing proxy 534 may make the API call to cloud services 536.

[0092] In some examples, a user 518 may create a configuration file that may include operations to execute within the regional data plane 504. The user 518 may send the configuration file via a LAN gateway 516 to a central CIOS 506, which may be a computing device included in the central control plane 502. The configuration file may be received by the central CIOS 506 in the control plane 508 or in a view / plan / approval module 512. The change management module 510 may then convert the configuration file into a region-agnostic (RA) configuration file that can be used in the regional data plane 504. The view / plan / approval module 512 or state ingestor 514 may send the RA configuration file via the WAN gateway 520 to the regional CIOS 522, which may be a computing device included in the regional data plane 504.

[0093] Regional CIOS 522 can receive RA configuration files from central CIOS 506, and regional CIOS 522 can send the RA configuration files to scheduler 524, which can create tasks and send the tasks to task DB 528. A task may include compiling a set of operations to be executed on an execution target, The safety plan may include comparing the current state of resources at the execution target with the desired state of resources at the execution target. The task may be sent to a worker 526, which may perform the task. In performing the task, the worker 526 may assist in creating a safety plan, which is a set of approved actions. The actions may include deploying resources to the execution target or any other suitable actions for execution at the execution target. The worker 526 may send the safety plan via the WAN gateway 520 to a regional CIOS 522, which may send the safety plan to the central CIOS 506.

[0094] The central CIOS 506 may receive the safety plan from the regional CIOS 522 and compare the set of changes included in the safety plan with the actions at the execution target. The change management module 510 may perform this comparison and make one of two determinations: either the action is a subset of the set of changes in the safety plan, or the action is not a subset of the set of changes in the safety plan. Only these two determinations are described in detail, but other determinations may be made (e.g., what percentage difference exists between the set of changes and the safety plan, etc.). In response to determining that the action is a subset of the set of changes included in the safety plan, the action may be executed, which may include deploying resources to the execution target. In response to determining that the action is not a subset of the set of changes included in the safety plan, a notification may be sent to the user 518. The notification may be sent to the user 518 via the LAN gateway 516 and may include information alerting the user 518 that drift may be occurring or that an action that may be scheduled to be executed at the execution target may not be included in the safety plan. In response to viewing the notification, the user 518 may approve or reject the action that is not included in the safety plan. If the user 518 approves the operation, the operation may be executed and the resource may be deployed to the execution target. If the user 518 rejects the operation, the operation may not be executed, the user 518 may decide to create a new configuration file, or may decide to abandon the operation. Thus, a safety plan may be understood to be a data structure that controls the ability of the processor(s) executing the deployment process by either allowing the operation to be executed if the operation is a subset of the set of changes included in the safety plan, or by suspending the execution of the operation if the operation is not a subset of the changes included in the safety plan.

[0095] In another example, a user 518 can create a set of configuration files that may contain a desired state of resources for a set of execution targets in a set of regions. The configuration files can be sent to a central CIOS 506, which can compile the configuration files into a set of RA configuration files, one RA configuration file per region. The RA configuration files can be sent to each region via a WAN gateway 520 and received by regional CIOS 522 included in each region.

[0096] In each region, the regional CIOS 522 may receive a respective RA configuration file from the central CIOS 506, which may send the respective RA configuration file to a scheduler 524, which may create tasks and send the tasks to a task DB 528. The tasks may include compiling a set of operations to be performed on the execution targets in each region and may include comparing the current state of resources on the execution targets with the desired state of resources on the execution targets. The tasks may be sent to workers 526 that can perform the tasks. In performing the tasks, the workers 526 can help create a safety plan, which is a set of approved operations. The operations may include deploying resources to the execution targets or any other appropriate operations to execute on the execution targets. The workers 526 can send the safety plan to the regional CIOS 522, which can send the safety plan to the central CIOS 506 via the WAN gateway 520.

[0097] The central CIOS 506 can receive a set of safety plans from the regions and compile the safety plans into a master safety plan (e.g., one safety plan of multiple safety plans), also referred to as a compiled safety plan. Each safety plan included in the compiled safety plan can include similar changes. A difference in changes between the safety plans included in the compiled safety plan can indicate that drift has occurred in at least one region. The central CIOS 506 can compare the changes included in each safety plan included in the compiled safety plan with actions at the execution targets included in the respective region. The change management module 510 can perform this comparison and make one of two determinations: whether the action is a subset of the set of changes in the compiled safety plan or whether the action is not a subset of the set of changes in the compiled safety plan. In response to a determination that the action is a subset of the set of changes in the compiled safety plan, the action can be performed, which may include deploying resources to the execution targets included in the respective region. In response to a determination that the action is not a subset of the set of changes in the compiled safety plan, a notification can be sent to a user 518. The notification may be sent to the user 518 via the LAN gateway 516 and may include information alerting the user 518 that drift may have occurred in at least one region or that an operation that may be scheduled to be executed in at least one execution target may not be included in the compiled safety plan. In response to viewing the notification, the user 518 may approve or reject the operation that is not included in the safety plan. If the user 518 approves the operation, the operation may be executed and resources may be deployed to the execution targets included in the respective region. If the user 518 rejects the operation, the operation may not be executed, and the user 518 may decide to create a new configuration file or decide to abandon the operation.

[0098] 6 and 7 illustrate example flowcharts illustrating processes 600 and 700 for performing techniques of CIOS according to certain embodiments of the present disclosure. These processes are illustrated as logical flow diagrams, each operation of which may be implemented in hardware, computer instructions, or a combination thereof. In the context of computer instructions, the operations may represent computer-executable instructions stored on one or more computer-readable storage media that, when executed by one or more processors, perform the recited operations. Generally, computer-executable instructions include routines, programs, objects, components, data structures, and the like that perform particular functions or implement particular data types. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described operations may be combined in any order and / or in parallel to implement a process.

[0099] Additionally, processes 600 and 700 may be executed under the control of one or more computing devices or computer systems configured with executable instructions and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) collectively executing on one or more processors, by hardware, or by a combination thereof. As mentioned above, the code may be stored on a computer-readable storage medium, for example, in the form of a computer program including multiple instructions executable by one or more processors. In some embodiments, processes 600 and 700 may be executed in parallel by multiple processors. The computer-readable storage medium may be non-transitory.

[0100] FIG. 6 illustrates a process 600 for using safety plans in a cloud infrastructure orchestration service, according to at least one embodiment. 6 is a flowchart of a process 600. Process 600 may begin at block 602, where a central CIOS (e.g., central CIOS 506 of FIG. 5) receives a safety plan from a regional CIOS (e.g., regional CIOS 522 of FIG. 5) for a region that includes an execution target. The safety plan may include a list of resources and actions based on a deployment configuration file. The actions may include instructions for deploying resources to the execution target. In some examples, the central CIOS may receive a set of safety plans from a set of regional CIOSs that correspond to a set of regions. The safety plans may be compiled by the central CIOS into a compiled safety plan that may include resources and actions based on the deployment configuration file that corresponds to the execution targets, each of the execution targets being included in a different region.

[0101] At block 604, the central CIOS prepares to execute at least one action corresponding to at least one resource included in the deployment configuration file. In response to receiving the safety plan from the regional CIOS, the central CIOS may prepare to execute the at least one action on at least one execution target, for example, by compiling a local plan that may include the at least one action. The action may include deploying at least one resource to the execution target. In some examples, the central CIOS may prepare to execute a set of actions corresponding to a set of resources included in the set of deployment configuration files. In this example, the central CIOS may receive a set of safety plans corresponding to a set of regions from a set of regional CIOSs.

[0102] In block 606, the central CIOS compares the action to the safety plan. A change management module (e.g., change management module 510 of FIG. 5 ) that may be included in the central CIOS may perform the comparison. The comparison may include determining whether the action is a subset of the safety plan. An action may be considered a subset of the safety plan if the safety plan includes the action. In some examples, the action may not change the current state of the execution target. In this case, the central CIOS may not execute the action, but the action may be considered a subset of the safety plan.

[0103] At block 608, the action is executed. In response to the change management module determining that the action is a subset of the safety plan, the central CIOS can send a command to execute the action. The command may be sent by the central CIOS to regional CIOSs in each region and may include deploying resources included in the deployment configuration file to the execution targets. In examples where resources are desired to be deployed to multiple execution targets, multiple safety plans may be received by the central CIOS. In this case, the central CIOS may prepare to execute the action, and the change management module may compare the action with the safety plan to determine whether the action is a subset of the safety plan. In response to the change management module determining that the action is a subset of the safety plan, the central CIOS can send a command to each region to execute the action. The command may be sent from the central CIOS to regional CIOSs in each region and may include deploying resources included in the deployment configuration file to each execution target.

[0104] In block 610, the action is not performed and the deployment is stopped. In response to the change management module determining that the action is not a subset of the safety plan, the central CIOS may not send a command to perform the action; instead, the central CIOS may stop the deployment. If a resource is desired to be deployed to multiple execution targets, multiple safety plans may be received by the central CIOS. In this case, the central CIOS may prepare to perform the action, and the change management module The central CIOS may compare the actions to the safety plan to determine whether the actions are a subset of the safety plan. In response to the change management module determining that the actions are not a subset of the safety plan, the central CIOS may not send a command to execute the actions. Instead, the central CIOS may stop the deployment. In other examples, the central CIOS may stop the actions that are not a subset of the safety plan and may send a command to execute the actions that are a subset of the safety plan.

[0105] In block 612, the central CIOS sends a notification to a user (e.g., user 518 of FIG. 5 ) that the deployment does not comply with the safety plan. In response to determining that the operations are not a subset of the safety plan (as performed in block 610), the central CIOS may create and send a notification to the user that the deployment no longer complies with the safety plan. The notification may present the user with the option to choose to continue with the deployment despite the operations not being a subset of the safety plan. The user may choose to continue with the deployment or may choose to abandon the deployment. If the user chooses to abandon the deployment, the user can create a different configuration file and attempt a different deployment. If the user wants to deploy resources to multiple execution targets, the notification sent to the user may include the deployment not complying with the safety plan. The central CIOS can send commands to execute the operations determined to be a subset of the operations.

[0106] 7 is a flowchart illustrating a process 700 for generating a security plan in a cloud infrastructure orchestration service, according to at least one embodiment. At block 702, a regional CIOS (e.g., regional CIOS 522 of FIG. 5) included in a region receives a configuration file for deploying infrastructure resources from a central CIOS (e.g., central CIOS 506 of FIG. 5). The configuration file may include operations that include instructions for deploying resources to execution targets included in the region.

[0107] In block 704, the regional CIOS identifies a new current state of the cloud infrastructure in the region. The configuration file may define a desired state of the cloud infrastructure in the region, and in response to receiving the configuration file, the regional CIOS may identify a current state of resources in the region. In some examples, the current state may be similar to or identical to the desired state. In other examples, the current state may not include any resources, and the desired state may include most resources desired to be deployed to the execution target.

[0108] In block 706, the regional CIOS generates a safety plan for deploying infrastructure resources based on a comparison of the current state and the desired state. The safety plan may include actions that can be performed on the execution target. The actions may include deploying infrastructure resources to the execution target. In some instances where the current state and the desired state are similar or identical, the safety plan compiled by the regional CIOS may be empty or may not include any actions.

[0109] At block 708, the regional CIOS sends a safety plan for deploying infrastructure resources to the central CIOS. In response to creating the safety plan, the regional CIOS may send the safety plan to the central CIOS for comparison. The central CIOS may determine that the operations to be performed on the execution target are a subset of the safety plan, similar to block 608 of process 600. In response to this determination, the regional CIOS may execute the operations and deploy the resources included in the configuration file to the execution target.

[0110] FIG. 8 illustrates an example swimlane diagram illustrating a process 800 for executing techniques of CIOS according to certain embodiments of the present disclosure. This process is illustrated as a logical flow diagram, each operation of which may be implemented in hardware, computer instructions, or a combination thereof. In the context of computer instructions, the operations may represent computer-executable instructions stored on one or more computer-readable storage media that, when executed by one or more processors, perform the recited operations. Generally, computer-executable instructions include routines, programs, objects, components, data structures, and the like that perform particular functions or implement particular data types. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described operations can be combined in any order and / or in parallel to execute the process.

[0111] Furthermore, process 800 may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) executable under the control of one or more computing devices or computer systems, collectively executed on one or more processors, by hardware, or by a combination thereof. As mentioned above, the code may be stored on a computer-readable storage medium, for example, in the form of a computer program including multiple instructions executable by one or more processors. In some embodiments, process 800 may be executable in parallel by multiple processors. The computer-readable storage medium may be non-transitory.

[0112] 8 is a swim lane diagram illustrating a process 800 for explaining how a safety plan is generated and used, according to at least one embodiment. The diagram includes three lanes as shown: a user lane 802, a central lane 804, and a regional lane 806. The user lane 802 may include operations performed by a user (e.g., user 518 of FIG. 5 ) or commands given to a computing system by a user. The central lane 804 may include operations performed by a central CIOS (e.g., central CIOS 506 of FIG. 5 ). The regional lane 806 may include operations performed by a regional CIOS (e.g., regional CIOS 522 of FIG. 5 ).

[0113] Process 800 may begin at block 808 with a user creating a configuration file. A user may want to deploy infrastructure resources to an execution target included in a region. In response to this desire, the user may create a configuration file that can define a desired state of the execution target. At block 810, the user submits the configuration to a central CIOS. In response to creating the configuration file, the user may submit the configuration file to the central CIOS to initiate the deployment.

[0114] At block 812, the central CIOS receives a configuration file from the user. In response to receiving the configuration file from the user, the central CIOS may initiate the deployment. At block 814, the central CIOS compiles a region-independent (RA) configuration file. In response to initiating the deployment, the central CIOS may compile an RA configuration, which may be a configuration file that may be independent of the region to which the RA configuration file is sent. At block 816, the central CIOS sends the RA configuration file to the regional CIOS. In response to creating the RA configuration file, the central CIOS may send the RA configuration file to the regional CIOS in the region where the resources are desired to be deployed.

[0115] At block 818, the regional CIOS in the region where the resource is desired to be deployed receives the RA configuration file. The RA configuration file is used to configure the infrastructure on the execution target. The RA configuration file may include operations performed to deploy infrastructure resources. The regional CIOS may send the RA configuration file to a worker node (e.g., worker 526 of FIG. 5 ) that can perform the comparison. In block 820, the regional CIOS creates a safety plan based on the new current state. The worker may compare the new current state of the infrastructure resources in the region with the desired state of the infrastructure resources defined in the RA configuration file. The regional CIOS may create a safety plan based at least in part on the comparison performed by the worker. The safety plan may include approved changes to the resources in the execution target.

[0116] In block 822, the regional CIOS sends the safety plan to the central CIOS. In response to creating the safety plan, the regional CIOS can send the safety plan to the central CIOS to determine whether the operations can be deployed to the execution target. In block 824, the central CIOS receives the safety plan from the regional CIOS. In some examples where it is desired to deploy resources to multiple regions, the central CIOS may receive multiple safety plans. In this case, the central CIOS may compile one safety plan from the multiple safety plans, hereinafter referred to as a compiled safety plan. In block 826, the central CIOS compares the deployment operations with the safety plan. The deployment operations may include instructions to deploy infrastructure resources to the execution target. In the case of a compiled safety plan, the central CIOS may compare the operations with the compiled safety plan. In block 828, the central CIOS determines whether the operations are a subset of the safety plan. In the case of a compiled safety plan, the central CIOS may determine whether the operations are a subset of the compiled safety plan.

[0117] In block 830, in response to the central CIOS determining that the operation is a subset of the safety plan, the central CIOS sends a command to the regional CIOS to execute the operation. In this case, infrastructure resources may be deployed to the execution target, and in the case of a compiled safety plan, the resources may be deployed to multiple execution targets. In other examples, the operation may not change the state of the execution target, in which case the central CIOS may not take any action. In block 832, in response to the central CIOS determining that the operation is not a subset of the safety plan, the central CIOS stops the deployment. In this case, the central CIOS sends a notification to the user, which may inform the user that drift has occurred and that the operation does not comply with the safety plan. This notification allows the user to allow the operation to execute. If the user chooses not to allow the operation to execute, the user can abandon the deployment or create a new configuration file, submit the new configuration file to the central CIOS, and initiate a second deployment. In the case of a compiled safety plan, the central CIOS may send a notification to the user if the operation is not a subset of the compiled safety plan.

[0118] FIG. 9 is a block diagram illustrating a disconnected area 900, according to at least one embodiment. The disconnected area 900 may be communicatively coupled to a connected central CIOS 902 (e.g., central CIOS 506 of FIG. 5). However, the connection between the disconnected area 900 and the connected central CIOS 902 may not be real-time and may be delayed. The disconnected area 900 may include a central CIOS 904 (e.g., central CIOS 506 of FIG. 5), a scheduler node 906 (e.g., scheduler 524 of FIG. 5), a worker node 908 (e.g., worker 526 of FIG. 5), a docker 910 (e.g., docker 530 of FIG. 5), which may include a CIOS container 912 (e.g., CIOS container 532 of FIG. 5), and a signing proxy 914 (e.g., signing proxy 534 of FIG. 5). In some examples, the disconnected area 900 may receive information / instructions from the connected central CIOS 902. It is possible, but the connected central CIOS 902 may be disconnected with respect to communications from the disconnected region 900. In other words, the communication may be one-way, and once the connected central CIOS 902 sends information, it cannot be sure what action the disconnected region 900 will take.

[0119] The central CIOS 904 can receive tasks from the connected central CIOS 902 and can send the tasks to the scheduler 906. The tasks may include performing CRUD operations or any other suitable tasks for deploying infrastructure resources within the region of the execution target(s). The scheduler 906 can send the tasks to workers 908, which can be included in a worker fleet. The worker fleet can include many workers 908, and the scheduler 906 can select the worker 908 with the least amount of work or the most available amount of computing resources to assign the task to. The scheduler 906 can assign one task to a worker 908 at a time. The worker 908 can execute the task and, in executing the task, can make a call to a CIOS container 912 included in Docker 910. The CIOS container 912 can execute the task, including Terraform instructions. The instructions may direct the API call to cloud services 916, which may include services available to the disconnected region 900 that may not be available over a public network (e.g., the Internet). To make the API call to cloud services 916, the CIOS container 912 may send a request to make the API call to signing proxy 914, which may determine whether the request is valid. In response to determining that the request is valid, signing proxy 914 may make the API call to cloud services 916.

[0120] The connected central CIOS 902 can send a first safety plan and a configuration file to the central CIOS 904 for deployment of infrastructure resources at execution targets in the disconnected region 900. The central CIOS 904 can send the configuration file to the scheduler 906, which can assign a task to a worker 908 to create a second safety plan based on the configuration file. The first safety plan may be created from a different configuration file that is similar to or identical to the configuration file. In response to creating the second safety plan, the worker 908 can send the second safety plan to the central CIOS 904, which can compare the first safety plan with the second safety plan. In response to the comparison, the central CIOS 904 can determine that the second safety plan is a subset of the first safety plan. In this case, the second safety plan can be automatically approved, and infrastructure resources based on the configuration file can be deployed to execution targets in the disconnected region 900. The central CIOS 904 can determine that the second safety plan is not a subset of the first safety plan. In this case, the deployment can be stopped and a notification can be sent to a user (e.g., user 518 in FIG. 5 ) informing the user that drift may be occurring and that the deployment is not compliant with the second safety plan. Further, in this example, if the second safety plan is not a subset of the first safety plan, the connected central CIOS 902 can send instructions to the disconnection region 900 to stop any originally planned deployment.

[0121] FIG. 10 illustrates an example flow diagram illustrating a process 1000 for performing techniques of CIOS, according to certain embodiments of the present disclosure. This process is illustrated as a logical flow diagram, each operation of which may be implemented in hardware, computer instructions, or a combination thereof. In the context of computer instructions, the operations may be implemented in computer programs stored on one or more computer-readable storage media that, when executed by one or more processors, perform the recited operations. The references may represent computer-executable instructions. Generally, computer-executable instructions include routines, programs, objects, components, data structures, etc. that perform particular functions or implement particular data types. The order in which operations are described is not intended to be construed as a limitation, and any number of the described operations can be combined in any order and / or in parallel to implement a process.

[0122] Additionally, process 1000 may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) executable under the control of one or more computing devices or computer systems, collectively executing on one or more processors, by hardware, or a combination thereof. As noted above, the code may be stored on a computer-readable medium in the form of a computer program including multiple instructions executable by one or more processors. In some embodiments, process 1000 may be executable in parallel by multiple processors. The computer-readable storage medium may be non-transitory.

[0123] 10 is a flowchart illustrating a process 1000 for using a safety plan in a disconnected area, according to at least one embodiment. At block 1002, a configuration file for deploying infrastructure resources to a first execution target and a second execution target is received. The configuration file may include infrastructure resources that a user (e.g., user 518 of FIG. 5 ) desires to deploy to the first execution target and the second execution target. In some examples, the configuration file may include resources for deployment on the first execution target, and the second configuration file may include resources for deployment on the second execution target.

[0124] At block 1004, a first safety plan is generated based on the configuration file. The safety plan may include a list of approved changes that the user wants to execute at the first execution target. The first safety plan may be generated by a central CIOS (e.g., central CIOS 506 of FIG. 5). The first execution target may be included in a connected region. The first safety plan may be created based on a comparison of the current state of resources at the first execution target with a desired state of resources at the first execution target, which may be defined in the configuration file.

[0125] Approval of the first safety plan is received at block 1006. The user may be given the option to approve the first safety plan, or the first safety plan may be automatically approved based on operations performed on the first execution target. In response to the approval at block 1006, infrastructure resources may be deployed to the execution target based on the configuration file.

[0126] At block 1008, a second safety plan is generated for the second execution target based on the configuration file. The second safety plan may be generated by a central CIOS, which may be included in the disconnected region (e.g., disconnected region 900 of FIG. 9 ). The second execution target may also be included in the disconnected region. In some examples, the second safety plan may be created for the second execution target based on the second configuration file. In some examples, the first execution target and the second execution target may be similar or identical, and the first safety plan and the second safety plan may be similar or identical.

[0127] In block 1010, the central CIOS determines whether the second safety plan is a subset of the first safety plan. The central CIOS may include in the disconnection region: The approved changes to the second safety plan may be compared to the approved changes to the first safety plan, and the central CIOS may determine that the second safety plan is a subset of the first safety plan or that the second safety plan is not a subset of the first safety plan.

[0128] In block 1012, in accordance with the central CIOS determining that the second safety plan is a subset of the first safety plan, the second safety plan is automatically approved based on the approval of the first safety plan. In this case, the central CIOS can automatically approve the second safety plan without user input because the first safety plan has already been approved and may have been successfully executed. In some examples, the current state of the second execution target may be similar to or the same as the desired state of the second execution target defined in the configuration file. In this example, the second safety plan may not include any changes to be executed in the second execution target. In this case, the second safety plan may be automatically approved by the central CIOS because the empty second safety plan is a subset of the first safety plan.

[0129] In block 1014, in accordance with the central CIOS determining that the second safety plan is not a subset of the first safety plan, the central CIOS stops the deployment at the second execution target and notifies the user that the second safety plan is non-compliant. The central CIOS may send a notification to the user informing the user that deployment of infrastructure resources at the second execution target may be stopped, that drift may have occurred, and that the second safety plan is not a subset of the first safety plan. The notification may be presented to the user in a user interface that may present at least one difference between the first safety plan and the second safety plan. The notification may allow the user to (i) choose to deploy the resources despite the central CIOS determining that the second safety plan is non-compliant, (ii) abandon the deployment at the second execution target, (iii) submit a new configuration file to initiate a new deployment at the second execution target, or a combination thereof.

[0130] In block 1016, in response to approval of the second safety plan by the central CIOS, the central CIOS sends the second safety plan to a second execution target for deploying infrastructure resources. The central CIOS and the second execution target may be included in a disconnected region. The second safety plan may include instructions for deploying resources to the second execution target. In some examples, the second safety plan may not include instructions for deploying resources because the current state of the second execution target may be similar or identical to the desired state of the execution target defined by the configuration file. In this case, the central CIOS may not take any action.

[0131] Exemplary System 11-13 illustrate aspects of an example environment for implementing aspects of the present disclosure, according to various embodiments. FIG. 11 is a simplified diagram illustrating a distributed system 1100 for implementing one embodiment of the present disclosure. In the illustrated embodiment, the distributed system 1100 includes one or more client computing devices 1102, 1104, 1106, and 1108, which are configured to execute and operate client applications, such as a web browser, a proprietary client (e.g., Oracle Forms), etc., via one or more networks 1110. A server 1112 may be communicatively coupled to the remote client computing devices 1102, 1104, 1106, and 1108 via the network 1110.

[0132] In various embodiments, the server 1112 provides identity management services. The server 1112 may be adapted to run one or more services or software applications, such as services and applications that provide virtualized environments. In particular embodiments, the server 1112 may also provide other services or software applications, which may include non-virtualized and virtualized environments. In some embodiments, these services may be provided to users of the client computing devices 1102, 1104, 1106, and / or 1108 as web-based or cloud services or under a software-as-a-service (SaaS) model. Users operating the client computing devices 11102, 1104, 1106, and / or 1108 may then utilize one or more client applications to interact with the server 1112 to utilize the services provided by these components.

[0133] In the configuration shown in FIG. 11 , software components 1118, 1120, and 1122 of system 1100 are shown executing on server 1112. In other embodiments, one or more of the components of system 1100 and / or the services provided by these components may also be executed by one or more of client computing devices 1102, 1104, 1106, and / or 1108. Users operating these client computing devices may then utilize one or more client applications to use the services provided by these components. These components may be implemented in hardware, firmware, software, or a combination thereof. It should be understood that a variety of different system configurations are possible and may differ from distributed system 1100. The embodiment shown in FIG. 11 is therefore an example of a distributed system for implementing the system of the embodiments and is not intended to be limiting.

[0134] Client computing devices 1102, 1104, 1106, and / or 1108 may include various types of computing systems. For example, client computing devices may include portable handheld devices (e.g., iPhone®, mobile phone, iPad®, computing tablet, personal digital assistant (PDA)) or wearable devices (e.g., Google Glass® head-mounted display) running software (e.g., Microsoft Windows® Mobile) and / or various mobile operating systems (e.g., iOS, Windows Phone, Android®, BlackBerry 10, Palm OS, etc.). These client computing devices may support various applications (e.g., various Internet-related apps, email, short message service (SMS) applications, etc.) and may use various other communication protocols. Client computing devices may also include general-purpose personal computers, including, by way of example, personal and / or laptop computers running various versions of Microsoft Windows®, Apple Macintosh®, and / or Linux® operating systems. The client computing devices may be workstation computers running any of a variety of commercially available UNIX or UNIX-like operating systems, including, but not limited to, various GNU / Linux operating systems (e.g., Google Chrome OS, etc.). Client computing devices may also be thin client computers, Internet-enabled gaming systems (e.g., with a Kinect gesture input device), or other devices that can communicate over network(s) 1110. It may also include electronic devices such as a Microsoft Xbox gaming console (with or without a mobile device) and / or a personal messaging device.

[0135] 11 is shown with four client computing devices, any number of client computing devices may be supported. Other devices, such as devices with sensors, may interact with the server 1112.

[0136] The network(s) 1110 in the distributed system 1100 may be any type of network familiar to those skilled in the art capable of supporting data communications using any of a variety of available protocols, including, but not limited to, TCP / IP (Transmission Control Protocol / Internet Protocol), SNA (Systems Network Architecture), IPX (Internet Packet Exchange), AppleTalk, etc. By way of example only, the network(s) 1110 may be a local area network (LAN), an Ethernet-based network, a token ring, a wide area network, the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., a network operating under the Institute of Electrical and Electronics Engineers (IEEE) 1002.11 suite of protocols, Bluetooth and / or other wireless protocols), and / or any combination of these and / or other networks.

[0137] The servers 1112 may comprise one or more general-purpose computers, dedicated server computers (including, by way of example, PC (personal computer) servers, UNIX servers, mid-range servers, mainframe computers, rack-mounted servers, etc.), server farms, server clusters, or any other suitable arrangement and / or combination. The servers 1112 may include one or more virtual machines running a virtual operating system or other computing architectures involving virtualization. One or more flexible pools of logical storage devices may be virtualized to maintain virtual storage devices for the servers. Virtual networks may be controllable by the servers 1112 using software-defined networking. In various embodiments, the servers 1112 may be adapted to run one or more services or software applications described in the foregoing disclosure. For example, the servers 1112 may correspond to a server for performing the above-described processing according to one embodiment of the present disclosure.

[0138] Server 1112 may run an operating system, including any of those described above, as well as any commercially available server operating system. Server 1112 may also run any of a variety of other server and / or middle-tier applications, including HTTP (Hypertext Transfer Protocol) servers, FTP (File Transfer Protocol) servers, CGI (Common Gateway Interface) servers, JAVA servers, and database servers. Examples of database servers are commercially available from Oracle, Microsoft, Sybase, IBM (International Business Machines), and others. This includes, but is not limited to,

[0139] In some implementations, the server 1112 runs one or more applications for parsing and consolidating data feeds and / or event updates received from users of the client computing devices 1102, 1104, 1106, and 1108. By way of example, the data feeds and / or event updates may include, but are not limited to, Twitter feeds, Facebook updates, or real-time updates received from one or more third-party sources and continuous data streams, which may include real-time events related to sensor data applications, stock tickers, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, automobile traffic monitoring, etc. Server 1112 may also include one or more applications for displaying the data feeds and / or real-time events via one or more display devices of client computing devices 1102, 1104, 11106, and 1108.

[0140] Distributed system 1100 may also include one or more databases 1114 and 1116. These databases may provide a mechanism for storing information such as identity information and other information used by embodiments of the present disclosure. Databases 1114 and 1116 may reside in a variety of locations. As one example, one or more of databases 1114 and 1116 may reside on non-transitory storage media local to (and / or residing within) server 1112. Alternatively, databases 1114 and 1116 may be remote from server 1112 and communicate with server 1112 via a network-based or dedicated connection. In one set of embodiments, databases 1114 and 1116 may reside within a storage area network (SAN). Similarly, any files necessary to perform functions attributed to server 1112 may be stored locally and / or remotely on server 1112, as appropriate. In one set of embodiments, databases 1114 and 1116 may include relational databases, such as those provided by Oracle Corporation, adapted to store, update, and retrieve data in response to commands formatted in SQL.

[0141] 12 is a diagram illustrating an example of a computer system 1200 that may be used to implement an embodiment of the present disclosure. In some embodiments, the computer system 1200 may be used to implement any of the various servers and computer systems described above. As shown in FIG. 12, the computer system 1200 includes various subsystems, including a processing subsystem 1204 that communicates with multiple peripheral subsystems via a bus subsystem 1202. These peripheral subsystems may include a processing acceleration unit 1206, an input / output subsystem 1208, a storage subsystem 1218, and a communication subsystem 1224. The storage subsystem 1218 may include a tangible computer-readable storage medium 1622 and a system memory 1210.

[0142] Bus subsystem 1202 provides a mechanism for allowing the various components and subsystems of computer system 1200 to communicate with each other as intended. While bus subsystem 1202 is shown schematically as a single bus, alternative embodiments of the bus subsystem may utilize multiple buses. Bus subsystem 1202 may be any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. For example, such architectures may include an Industry Standard Architecture (ISA) bus, a MicroChannel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, a Peripheral Component Interconnect (PCI) bus that may be implemented as a mezzanine bus manufactured in accordance with the IEEE P1386.1 standard, and the like.

[0143] The processing subsystem 1204 controls the operation of the computer system 1200 and may include one or more processing units 1232, 1234, etc. A processing unit may include one or more processors, including single-core or multi-core processors, one or more cores of a processor, or a combination thereof. In some embodiments, the processing subsystem 1204 may include one or more special-purpose co-processors, such as a graphics processor or a digital signal processor (DSP). In some embodiments, some or all of the processing units of the processing subsystem 1204 may be implemented using customized circuitry, such as an application-specific integrated circuit (ASIC) or a field-programmable gate array (FPGA).

[0144] In some embodiments, processing units within processing subsystem 1204 can execute instructions stored in system memory 1210 or on computer-readable storage medium 1222. In various embodiments, the processing units can execute various program or code instructions and can maintain multiple simultaneously executing programs or processes. At any given time, some or all of the program code being executed can reside within system memory 1210 and / or on computer-readable storage medium 1222, including, in some cases, on one or more storage devices. With suitable programming, processing subsystem 1204 can provide the various functions described above for dynamically modifying documents (e.g., web pages) in response to usage patterns.

[0145] In particular embodiments, processing acceleration unit 1206 may be provided to accelerate the overall processing performed by computer system 1200, to perform customized processing, or to offload portions of the processing performed by processing subsystem 1204.

[0146] Input / output subsystem 1208 may include devices and mechanisms for inputting information into computer system 1200 and / or outputting information from or through computer system 1200. In general, use of the term "input device" is intended to include all possible types of devices and mechanisms for inputting information into computer system 1200. User interface input devices may include, for example, keyboards, pointing devices (such as a mouse or trackball), touchpads or touchscreens integrated into displays, scroll wheels, click wheels, dials, buttons, switches, keypads, audio input devices with voice command recognition systems, microphones, and other types of input devices. User interface input devices may also include motion-sensing and / or gesture-recognition devices (such as a Microsoft Kinect® motion sensor) that allow a user to control and interact with the input device, a Microsoft Xbox® 360 game controller, and devices that provide an interface for receiving input using gestures and verbal commands. The user interface input devices may also include an eye gesture recognition device (such as a Google Glass® blink detector) that detects eye movements from the user (e.g., "blinks" while taking a picture and / or making a menu selection) and translates the eye gestures as input to the input device (e.g., Google Glass®). Additionally, the user interface input devices may include a voice recognition sensing device that allows the user to interact with a voice recognition system (e.g., Siri® navigator) via voice commands.

[0147] Other examples of user interface input devices include a three-dimensional (3D) mouse, a joystick, User interface input devices include, but are not limited to, electronic or pointing sticks, gamepads and graphics tablets, and audio / visual devices (such as speakers, digital cameras, digital camcorders, portable media players, webcams, image scanners, fingerprint scanners, barcode readers 3D scanners, 3D printers, laser range finders, and eye-tracking devices). Additionally, user interface input devices may include medical imaging input devices, such as, for example, computed tomography, magnetic resonance imaging, position emission tomography, and medical ultrasound devices. User interface input devices may also include audio input devices, such as, for example, MIDI keyboards and digital musical instruments.

[0148] User interface output devices may include display subsystems, indicator lights, or non-visual displays (such as audio output devices). Display subsystems may be cathode ray tubes (CRTs), flat panel devices (such as those using liquid crystal displays (LCDs) or plasma displays), projection devices, touch screens, and the like. In general, use of the term "output device" is intended to include all possible types of devices and mechanisms for outputting information from computer system 1200 to a user or to another computer. For example, user interface output devices may include, but are not limited to, various display devices that visually convey text, graphics, and audio / visual information (such as monitors, printers, speakers, headphones, automobile navigation systems, plotters, audio output devices, modems, and the like).

[0149] Storage subsystem 1218 provides a repository or data store for storing information used by computer system 1200. Storage subsystem 1218 provides a tangible, non-transitory, computer-readable storage medium for storing the basic programming and data constructs that provide the functionality of some embodiments. Software (programs, code modules, instructions) that, when executed by processing subsystem 1204, provide the above-described functionality may be stored in storage subsystem 1218. This software is executable by one or more processing units of processing subsystem 1204. Storage subsystem 1218 may also provide a repository for storing data used in accordance with the present disclosure.

[0150] The storage subsystem 1218 may include one or more non-transitory memory devices, including volatile and non-volatile memory devices. As shown in FIG. 12, the storage subsystem 1218 includes a system memory 1210 and a computer-readable storage medium 1222. The system memory 1210 may include several memories, including volatile main random access memory (RAM) for storing instructions and data during program execution, and non-volatile read-only memory (ROM) or flash memory, in which fixed instructions are stored. In some implementations, a basic input / output system (BIOS), containing basic routines that help transfer information between elements within the computer system 1200, such as during start-up, may be stored in ROM. The RAM may include data and / or program modules currently being operated on and executed by the processing subsystem 1204. In some implementations, the system memory 1210 may include several different types of memory, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0151] By way of example, and not limitation, as shown in FIG. 12, system memory 1210 may store application programs 1212 (which may include client applications, web browsers, mid-tier applications, relational database management systems (RDBMS), etc.), program data 1214, and an operating system 1216. By way of example, operating system 1216 may include various versions of the Microsoft Windows, Apple Macintosh, and / or Linux operating systems, various commercially available UNIX or UNIX-like operating systems (including, but not limited to, various GNU / Linux operating systems, Google Chrome OS, etc.), and / or mobile operating systems (such as iOS, Windows Phone, Android OS, BlackBerry 10 OS, and Palm OS operating systems).

[0152] The computer-readable storage medium 1222 may store programming and data structures that provide the functionality of some embodiments. Software (programs, code modules, instructions) that, when executed by the processing subsystem 1204, cause the processor to provide the above-described functionality may be stored in the storage subsystem 1218. By way of example, the computer-readable storage medium 1222 may include non-volatile memory such as a hard disk drive, a magnetic disk drive, an optical disk drive (such as a CD-ROM, DVD, or Blu-ray disc), or other optical media. The computer-readable storage medium 1222 may include, but is not limited to, a Zip drive, a flash memory card, a Universal Serial Bus (USB) flash drive, a Secure Digital (SD) card, a DVD disc, and digital video tape. The computer-readable storage medium 1222 may also include solid-state drives (SSDs) based on non-volatile memory (such as flash memory-based SSDs, enterprise flash drives, solid-state ROM), volatile memory-based SSDs (such as solid-state RAM, dynamic RAM, static RAM), DRAM-based SSDs, magnetoresistive RAM (MRAM) SSDs, and hybrid SSDs that use a combination of DRAM-based SSDs and flash memory-based SSDs. The computer-readable medium 1222 may provide storage of computer-readable instructions, data structures, program modules, and other data for the computer system 1200.

[0153] In particular embodiments, storage subsystem 1200 may also include computer-readable storage medium reader 1220 further connectable to computer-readable storage medium 1222. Together with, and optionally in combination with, system memory 1210, computer-readable storage medium 1222 may collectively represent remote, local, fixed, and / or removable storage devices and media for storing computer-readable information.

[0154] In particular embodiments, computer system 1200 may provide support for running one or more virtual machines. Computer system 1200 may execute programs such as a hypervisor to facilitate the configuration and management of virtual machines. Each virtual machine may be assigned memory, computational (e.g., processors, cores), input / output, and networking resources. Each virtual machine may run its own operating system, which may be the same as or different from operating systems run by other virtual machines executed by computer system 1200. Thus, multiple operating systems may be run simultaneously by computer system 1200. Each virtual machine generally operates independently from other virtual machines.

[0155] The communications subsystem 1224 provides an interface to other computer systems and networks. The communications subsystem 1224 acts as an interface for sending and receiving data between other systems and the computer system 1200. For example, the communications subsystem 1224 allows the computer system 1200 to communicate with other systems and networks. The communications subsystem 1224 may enable the establishment of a communications channel to one or more client devices over the Internet for sending and receiving information to and from the client devices. Additionally, the communications subsystem 1224 may be used to convey notifications of successful login or to re-enter a password from a privileged account manager to a requesting user.

[0156] The communications subsystem 1224 may support both wired and / or wireless communications protocols. For example, in particular embodiments, the communications subsystem 1224 may include a radio frequency (RF) transceiver component for accessing wireless voice and / or data networks (e.g., using cellular telephone technology, advanced data network technology (such as 3G, 4G, or EDGE (enhanced data rates for global evolution)), WiFi (IEEE 802.11 family of standards), or other mobile communications technology, or any combination thereof), a global positioning system (GPS) receiver component, and / or other components. In some embodiments, the communications subsystem 1224 may provide a wired network connection (e.g., Ethernet) in addition to or instead of a wireless interface.

[0157] The communications subsystem 1224 can send and receive data in various formats. For example, in some embodiments, the communications subsystem 1224 can receive incoming communications in the form of structured and / or unstructured data feeds 1226, event streams 1228, and event updates 1230. For example, the communications subsystem 1224 can be configured to receive (or send) real-time data feeds 1226 from users of social media networks and / or other communications services, such as Twitter feeds, Facebook updates, web feeds (such as Rich Site Summary (RSS) feeds), and / or real-time updates from one or more third-party sources.

[0158] In particular embodiments, the communications subsystem 1224 may be configured to receive data in the form of a continuous data stream, which may include an event stream 1228 of real-time events and / or event updates 1230, which may be continuous or infinite in nature without a clear end. Examples of applications that generate continuous data may include, for example, sensor data applications, stock tickers, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, and automobile traffic monitoring.

[0159] The communications subsystem 1224 can also be configured to output structured and / or unstructured data feeds 1226, event streams 1228, event updates 1230, and the like to one or more databases that may communicate with one or more streaming data source computers coupled to the computer system 1200.

[0160] The computer system 1200 may be one of a variety of types, including a handheld portable device (e.g., an iPhone® mobile phone, an iPad® computing tablet, a PDA), a wearable device (e.g., a Google Glass® head-mounted display), a personal computer, a workstation, a mainframe, a kiosk, a server rack, or other data processing system.

[0161] The nature of computers and networks is constantly changing, so the computer The description of computer system 1200 is intended to be exemplary only. Many other configurations are possible, having more or fewer components than the system shown in Figure 12. Based on the disclosure and teachings provided herein, one of ordinary skill in the art will appreciate other ways and / or methods for implementing the various embodiments.

[0162] The systems shown in the various figures may be provided in a variety of configurations, and in some embodiments, these systems may be configured as distributed systems in which one or more components of the systems are distributed across one or more networks in one or more cloud infrastructure systems.

[0163] A cloud infrastructure system is a collection of one or more server computing devices, network devices, and / or storage devices. These resources may be divided by a cloud service provider and allocated in some manner to its customers. For example, a cloud service provider (such as Oracle Corporation of Redwood Shores, California) may offer various types of cloud services, including, but not limited to, one or more services offered under the Software as a Service (SaaS) category, services offered under the Platform as a Service (PaaS) category, services offered under the Infrastructure as a Service (IaaS) category, or services in other categories of services, including hybrid services. Examples of SaaS services include Oracle Fusion Examples of PaaS services include, but are not limited to, the ability to build and deliver a suite of on-demand applications, such as applications. SaaS services enable customers to use applications that run on cloud infrastructure systems without having to purchase the software for those applications. Examples of PaaS services include, but are not limited to, services that allow organizations (such as Oracle) to consolidate existing applications onto a shared, common architecture, and the ability to build new applications that leverage shared services offered by platforms such as Oracle Java Cloud Services (JCS) and Oracle Database Cloud Services (DBCS). IaaS services may facilitate the management and control of basic computing resources, such as storage, network, and other basic computing resources, for customers who use services offered by SaaS and PaaS platforms.

[0164] 13 is a simplified block diagram illustrating one or more components of a system environment 1300 in which services provided by one or more components of an embodiment system may be offered as cloud services, according to one embodiment of the present disclosure. In the illustrated embodiment, the system environment 1300 includes one or more client computing devices 1304, 1306, and 1308, which may be used by users to interact with a cloud infrastructure system 1302 that provides cloud services. These client computing devices may be configured to run a web browser, a proprietary client application (e.g., a client application such as Oracle Forms, or some other application), which may be used by users of the client computing devices to interact with the cloud infrastructure system 1302 and use the services offered by the cloud infrastructure system 1302.

[0165] It should be understood that the cloud infrastructure system 1302 depicted in the figure may have components other than those shown. Furthermore, the illustrated embodiment is only one example of a cloud infrastructure system that may incorporate an embodiment of the present disclosure. In some other embodiments, cloud infrastructure system 1302 may have more or fewer components than shown in the figure, may combine two or more components, or may have a different configuration or arrangement of components.

[0166] The client computing devices 1304, 1306, and 1308 may be devices similar to those described above for the client computing devices 1102, 1104, 1106, and 1108.

[0167] Although the example system environment 1300 is shown with three client computing devices, any number of client computing devices may be supported. Other devices, such as devices with sensors, may interact with the cloud infrastructure system 1302.

[0168] Network(s) 1310 may facilitate communication and exchange of data between clients 1304, 1306, and 1308 and cloud infrastructure system 1302. Each network may be any type of network familiar to those skilled in the art capable of supporting data communication using any of a variety of commercially available protocols, including those described above for network(s) 1110.

[0169] Cloud infrastructure system 1302 may comprise one or more computers and / or servers, which may include those described above for server 1112 .

[0170] In particular embodiments, the services provided by a cloud infrastructure system may include numerous services (e.g., online data storage and backup solutions, web-based email services, hosted office suites and document collaboration services, database processing, managed technical support services, etc.) made available to users of the cloud infrastructure system on demand. The services provided by the cloud infrastructure system can dynamically scale to meet the needs of its users. A specific instantiation of a service provided by the cloud infrastructure system is referred to herein as a “service instance.” In general, any service made available to users from a cloud service provider's system over a communications network such as the Internet is referred to as a “cloud service.” In a public cloud environment, the servers and systems comprising the cloud service provider's system are distinct from a customer's own on-premises servers and systems. For example, the cloud service provider's system can host applications, and users can order and use the applications on demand over a communications network such as the Internet.

[0171] In some examples, services in a computer network cloud infrastructure may include protected computer network access to storage, hosted databases, hosted web servers, software applications, or other services provided to users by a cloud vendor or as otherwise known in the art. For example, a service may include password-protected access to remote storage on the cloud over the Internet. As another example, a service may include a web-services-based hosted relational database and script for personal use by a networked developer. As another example, a service may include access to an email software application hosted on a cloud vendor's website.

[0172] In particular embodiments, cloud infrastructure system 1302 may include a suite of application, middleware, and database service offerings delivered to customers in a self-service, subscription-based, elastically scalable, reliable, highly available, and secure manner. An example of such a cloud infrastructure system is the Oracle Public Cloud (Oracle Public Cloud) offered by the present assignee. Public Cloud).

[0173] In various embodiments, cloud infrastructure system 1302 may be adapted to automatically provision, manage, and track customer subscriptions to services provided by cloud infrastructure system 1302. Cloud infrastructure system 1302 may provide cloud services through different deployment models. For example, services may be provided under a public cloud model, in which cloud infrastructure system 1302 is owned by an organization that sells cloud services (e.g., owned by Oracle Corporation) and the services are made available to the general public or to companies in different industries. As another example, services may be provided under a private cloud model, in which cloud infrastructure system 1302 is operated solely for a single organization and may provide services for one or more entities within that organization. Cloud services may also be provided under a community cloud model, in which cloud infrastructure system 1302 and the services provided by cloud infrastructure system 1302 are shared by several organizations in an associated community. Cloud services may also be provided under a hybrid cloud model, which is a combination of two or more different models.

[0174] In some embodiments, the services offered by cloud infrastructure system 1302 may include one or more services offered under the Software as a Service (SaaS) category, the Platform as a Service (PaaS) category, the Infrastructure as a Service (IaaS) category, or other service categories, including hybrid services. A customer may order one or more services offered by cloud infrastructure system 1302 via a subscription order. Cloud infrastructure system 1302 then performs processing to provide the services in the customer's subscription order.

[0175] In some embodiments, services provided by cloud infrastructure system 1302 may include, but are not limited to, application services, platform services, and infrastructure services. In some examples, application services may be provided by the cloud infrastructure system via a SaaS platform. The SaaS platform may be configured to provide cloud services that fall under the SaaS category. For example, the SaaS platform may provide the ability to build and deliver a suite of on-demand applications on an integrated development and deployment platform. The SaaS platform may manage and control the underlying software and infrastructure for delivering the SaaS services. By utilizing the services provided by the SaaS platform, customers can utilize applications that run on the cloud infrastructure system. Customers may require separate licenses and A customer may obtain application services without having to purchase software and support. A variety of different SaaS services may be offered, including, but not limited to, services that provide solutions for sales performance management, enterprise integration, and business flexibility for large organizations.

[0176] In some embodiments, platform services may be provided by a cloud infrastructure system via a PaaS platform. The PaaS platform can be configured to provide cloud services that fall into the PaaS category. Examples of platform services may include, but are not limited to, services that enable organizations (such as Oracle) to integrate existing applications on a shared, common architecture and the ability to build new applications that leverage shared services provided by the platform. The PaaS platform may manage and control the underlying software and infrastructure to provide the PaaS services. Customers can obtain the PaaS services provided by the cloud infrastructure system without having to purchase separate licenses and support. Examples of platform services include, but are not limited to, Oracle Java Cloud Services (JCS) and Oracle Database Cloud Services (DBCS).

[0177] By utilizing services provided by a PaaS platform, customers can use programming languages ​​and tools supported by the cloud infrastructure system and also control the deployed services. In some embodiments, the platform services provided by the cloud infrastructure system may include database cloud services, middleware cloud services (e.g., Oracle Fusion Middleware Services), and Java cloud services. In one embodiment, the database cloud services may support a shared service deployment model that allows organizations to pool database resources and provide database-as-a-service to customers in the form of a database cloud. In the cloud infrastructure system, the middleware cloud services may provide a platform for customers to develop and deploy various business applications, and the Java cloud services may provide a platform for customers to deploy Java applications in the cloud infrastructure system.

[0178] A variety of different infrastructure services may be provided in a cloud infrastructure system by an IaaS platform that facilitates the management and control of basic computing resources (such as storage, network, and other basic computing resources) for customers who use the services provided by the SaaS and PaaS platforms.

[0179] In particular embodiments, cloud infrastructure system 1302 may also include infrastructure resources 1330 for providing resources used to provide various services to customers of the cloud infrastructure system. In one embodiment, infrastructure resources 1330 may include a pre-integrated and optimized combination of hardware (such as servers, storage, and networking resources) for running the services provided by the PaaS and SaaS platforms.

[0180] In some embodiments, the resource in the cloud infrastructure system 1302 Resources may be shared by multiple users and dynamically reallocated per request. Furthermore, resources may be allocated to users in different time zones. For example, cloud infrastructure system 1302 may allow a first set of users in a first time zone to utilize the resources of the cloud infrastructure system for a particular time, and then allow reallocation of the same resources to another set of users located in a different time zone to maximize resource utilization.

[0181] In particular embodiments, multiple internal shared services 1332 are provided that are shared by various components or modules of cloud infrastructure system 1302 and by services provided by cloud infrastructure system 1302. These internal shared services may include, but are not limited to, security and identity services, integration services, enterprise repository services, enterprise manager services, virus scanning and whitelist services, high availability, backup and recovery services, services to enable cloud support, email services, notification services, file transfer services, and the like.

[0182] In particular embodiments, cloud infrastructure system 1302 may provide comprehensive management of cloud services (e.g., SaaS, PaaS, and IaaS services) in the cloud infrastructure system. In one embodiment, cloud management functions may include functions such as provisioning, managing, and tracking customer subscriptions received by cloud infrastructure system 1302.

[0183] In one embodiment, as shown in the figure, cloud management functionality may be provided by one or more modules, such as an order management module 1320, an order orchestration module 1322, an order provisioning module 1324, an order management and monitoring module 1326, and an identity management module 1328. These modules may include or be provided using one or more computers and / or servers, which may be general-purpose computers, dedicated server computers, server farms, server clusters, or any other suitable arrangement and / or combination.

[0184] In an example of operation 1334, a customer may interact with cloud infrastructure system 1302 using a client device, such as client device 1304, 1306, or 1308, by requesting one or more services offered by cloud infrastructure system 1302 and placing an order for subscriptions to one or more services offered by cloud infrastructure system 1302. In particular embodiments, the customer may access a cloud user interface (UI), i.e., cloud UI 1312, cloud UI 1314, and / or cloud UI 1316, and place the subscription order via these UIs. Order information received by cloud infrastructure system 1302 in response to the customer placing the order may include information identifying the customer and the one or more services offered by cloud infrastructure system 1302 to which the customer wishes to subscribe.

[0185] After an order is placed by a customer, the order information is received via the cloud UI 1312, 1314 and / or 1316.

[0186] In operation 1336, the order is stored in the order database 1318. The leader database 1318 may be one of several databases operated by the cloud infrastructure system 1302 and in coordination with other system elements.

[0187] At operation 1338, the order information is forwarded to the order management module 1320. In some examples, the order management module 1320 can be configured to perform billing and accounting functions related to the order, such as validating the order and booking the order upon validation.

[0188] At operation 1340, information about the order is communicated to the order orchestration module 1322. The order orchestration module 1322 may utilize the order information to orchestrate the provisioning of services and resources for the order placed by the customer. In some examples, the order orchestration module 1322 may use the services of the order provisioning module 1324 to orchestrate the provisioning of resources to support the subscribed services.

[0189] In particular embodiments, the order orchestration module 1322 enables management of business processes associated with each order and applies business logic to determine whether the order should proceed to provisioning. At operation 1342, upon receiving an order for a new subscription, the order orchestration module 1322 sends a request to the order provisioning module 1324 to allocate resources and configure those resources needed to fulfill the subscription order. The order provisioning module 1324 enables allocation of resources for services ordered by a customer. The order provisioning module 1324 provides a level of abstraction between the cloud services provided by the cloud infrastructure system 1302 and the physical implementation layer used to provision resources to provide the requested services. This allows the order orchestration module 1322 to be decoupled from implementation details, such as whether services and resources are actually provisioned on the fly or whether they are pre-provisioned and only allocated / allocated upon request.

[0190] At operation 1344, once the services and resources are provisioned, a notification of the provided services may be sent by the order provisioning module 1324 of the cloud infrastructure system 1302 to the customer on the client devices 1304, 1306, and / or 1308. At operation 1346, the customer's subscription orders may be managed and tracked by the order management and monitoring module 1326. In some examples, the order management and monitoring module 1326 can be configured to collect usage statistics for the services in the subscription orders, such as the amount of storage used, the amount of data transferred, the number of users, and the amount of system up and down time.

[0191] In particular embodiments, cloud infrastructure system 1302 may include identity management module 1328. Identity management module 1328 may be configured to provide identity services, such as access management and authorization services, in cloud infrastructure system 1302. In some embodiments, identity management module 1328 may control information about customers who wish to use services provided by cloud infrastructure system 1302. Such information may be used to authenticate the identities of such customers. The identity management module 1328 may also include management of descriptive information about each customer and information about how that descriptive information can be accessed and modified, and by whom.

[0192] While specific embodiments of the present disclosure have been described, various modifications, variations, alternative configurations, and equivalents are encompassed within the scope of the present disclosure. The embodiments of the present disclosure are not limited to operation in a particular data processing environment, but can freely operate in multiple data processing environments. Furthermore, while embodiments of the present disclosure have been described using a particular sequence of transactions and steps, it should be apparent to those skilled in the art that the scope of the present disclosure is not limited to the sequence of transactions and steps described. Various features and aspects of the above-described embodiments may be used individually or together.

[0193] Furthermore, while embodiments of the present disclosure have been described using a particular combination of hardware and software, it should be recognized that other combinations of hardware and software are within the scope of the present disclosure. Embodiments of the present disclosure may be implemented exclusively in hardware, exclusively in software, or using a combination thereof. The various processes described herein may be implemented on the same processor or on any combination of different processors. Thus, while components or modules are described as being configured to perform particular operations, such configuration may be achieved, for example, by designing electronic circuitry to perform the operations, by programming a programmable electronic circuit (such as a microprocessor) to perform the operations, or any combination thereof. Processes may communicate using a variety of techniques, including, but not limited to, conventional techniques for inter-process communication, and different process pairs may use different techniques, or the same process pair may use different techniques at different times.

[0194] Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense. It will be apparent, however, that additions, subtractions, deletions, and other modifications and alterations may be made without departing from the broader spirit and scope of the appended claims. Accordingly, although specific embodiments of the present disclosure have been described, these are not intended to be limiting. Various modifications and equivalents are within the scope of the following claims. These modifications include any relevant combination of the disclosed features.

[0195] Several exemplary items of the present disclosure are described below. Item 1: A method comprising: receiving, by at least one of the plurality of computer processors, a safety plan including a list of resources and respective actions based at least in part on a deployment configuration file; Following receipt of instructions for approval of the safety plan, Arranging at least one of a plurality of computer processors to perform an operation corresponding to at least one of a list of resources according to a deployment configuration file; comparing the action to a safety plan; performing, by at least one of the plurality of computer processors, the action in accordance with a determination that the action is part of the safety plan; Following a determination that the action is not part of the safety plan, At least one of the plurality of computer processors stopping the deployment; and at least one of the plurality of computer processors sending a notification that the deployment does not comply with the safety plan.

[0196] Item 2. The method of item 1, wherein the multiple processors are associated with an execution target.

[0197] Item 3. The method of item 2, wherein the execution target includes a geographic region. Item 4. The method of item 1, wherein the deployment configuration file is received from a central cloud infrastructure orchestration service.

[0198] Item 5. The method of item 4, wherein the deployment configuration file is generated by a user of a central cloud infrastructure orchestration service.

[0199] Item 6. The method of item 1, wherein each operation includes at least one of a create operation, an update operation, or a delete operation performed on or associated with at least one resource of the list of resources.

[0200] Item 7. The method of item 1, further comprising at least one of the plurality of computer processors halting the deployment in accordance with receiving disapproval of the safety plan or non-receiving approval of the safety plan.

[0201] Item 8. The method of item 1, further comprising at least one of the plurality of computer processors generating a local plan having local operations according to the deployment configuration file, and comparing the operations to the safety plan includes comparing one of the local operations to the safety plan.

[0202] Item 9. A computer-readable storage medium having written thereon instructions that, when executed by a computer processor, cause the computer processor to perform operations, the operations being: receiving a safety plan including a list of resources and their respective actions based at least in part on a deployment configuration file; Following receipt of instructions for approval of the safety plan, preparing to perform an action corresponding to at least one of a list of resources according to a deployment configuration file; Comparing the behavior to the safety plan and performing the action pursuant to a determination that the action is part of the safety plan; and Stopping the deployment pursuant to a determination that the action is not part of the safety plan; and sending a notification that the deployment is not compliant with the safety plan.

[0203] Item 10. The computer-readable storage medium of item 9, wherein the deployment configuration file is generated by a central cloud infrastructure orchestration service.

[0204] Item 11. The central cloud infrastructure orchestration service according to item 10, wherein the central cloud infrastructure orchestration service is configured to generate a list of safety plans, each safety plan in the list of safety plans corresponding to a specific execution target or a specific deployment phase. A computer-readable storage medium.

[0205] Item 12. The computer-readable storage medium of item 11, wherein the central cloud infrastructure orchestration service is further configured to contemporaneously generate the list of safety plans prior to receiving approval or disapproval of the list of safety plans.

[0206] Item 13. The computer-readable storage medium of item 12, wherein the central cloud infrastructure orchestration service is configured to generate a plurality of deployment configuration files, each deployment configuration file of the plurality of deployment configuration files being interpreted for each respective execution target on the list of safety plans.

[0207] Item 14. The computer-readable storage medium of item 10, wherein the operations further include sending the safety plan to a central cloud infrastructure orchestration service.

[0208] Item 15. Central cloud infrastructure orchestration services are Rendering the safety plan on a user interface of the user computer; Item 15. The computer-readable storage medium of item 14, configured to receive approval or disapproval of the safety plan.

[0209] Item 16. A system comprising: a processor; and a memory storing instructions that, when executed by the processor, cause the system to: receiving a safety plan including a list of resources and their respective actions based at least in part on a deployment configuration file; Following receipt of instructions for approval of the safety plan, prepares to perform an action corresponding to at least one of the list of resources according to the deployment configuration file, Compare the behavior to the safety plan, Executing the action pursuant to a determination that the action is part of the safety plan; Following a determination that the action is not part of the safety plan, Stop the deployment and A system that is configured to send notifications that a deployment is not compliant with the safety plan.

[0210] Item 17. The system of item 16, wherein the action is one of a plurality of actions corresponding to at least one of a list of resources according to a deployment configuration file, and each action of the plurality of actions is compared to the safety plan at least until the deployment is completed or a stop of the deployment occurs.

[0211] Item 18. The system of item 16, wherein the action is not part of a safety plan that is based at least in part on a behavior change that occurred in one of the resources before the action was performed.

[0212] Item 19. The system of item 16, wherein the deployment configuration file is generated by a central cloud infrastructure orchestration service.

[0213] Item 20. The system of item 19, wherein the notification is sent to a user of a central cloud infrastructure orchestration service.

[0214] Item 21. An apparatus comprising means for the steps according to any one of items 1 to 8. Item 22. A method comprising: receiving, by at least one of the plurality of computer processors, a configuration file for deployment to a first execution target and a second execution target; and generating, by at least one of the plurality of computer processors, a first safety plan for the first execution target, the first safety plan including a first list of resources and respective operations associated with deployment in the first execution target based at least in part on the configuration file, the method further comprising: receiving, by at least one of the plurality of computer processors, an acknowledgement of the first safety plan; and generating, by at least one of the plurality of computer processors, a second safety plan for the second execution target, the second safety plan including a second list of resources and respective operations associated with deployment in the second execution target based at least in part on the configuration file, the method further comprising: at least one of the plurality of computer processors determining whether the second safety plan is a subset of the first safety plan; Pursuant to a determination that the second safety plan is a subset of the first safety plan, at least one of the plurality of computer processors automatically approving the second safety plan based at least in part on the approval of the first safety plan; and at least one of the plurality of computer processors transmitting the automatically approved second safety plan to a second execution target.

[0215] Item 23. Pursuant to a determination that the second safety plan is not a subset of the first safety plan, a notification indicating that the second safety plan is not a subset of the first safety plan; Item 23. The method of item 22, further comprising providing a user interface for presenting a second list of resources and respective actions associated with the second safety plan.

[0216] Item 24. The method of item 23, further comprising determining at least one difference between the second list of resources and respective actions of the second safety plan and the first list of resources and respective actions of the first safety plan.

[0217] Item 25. The method of item 24, further comprising providing a user interface for presenting the at least one difference above all other visual representations of the second list of resources and respective actions of the second safety plan.

[0218] Item 26. The method of item 22, wherein determining whether the second safety plan is a subset of the first safety plan includes at least one of the plurality of computer processors determining whether each resource and each action in a second list of resources and respective actions of the second safety plan is listed in the first list of resources and respective actions of the first safety plan.

[0219] Item 27. The method of item 22, wherein the first execution target is a connection region of a cloud infrastructure orchestration service.

[0220] Item 28. The method according to item 27, wherein the second execution target is a disconnected region of a cloud infrastructure orchestration service. Item 29. The disconnected region includes an environment configured to receive instructions to deploy to the disconnected region, the disconnected region further configured to not transmit any data outside of the disconnected region. Item 29. The method according to Item 28,

[0221] Item 30. The method of item 28, wherein the second safety plan is for deployment on a second execution target.

[0222] Item 31. The method of item 30, wherein the second safety plan is automatically approved further pursuant to a determination that the deployment at the first execution target based at least in part on the first safety plan was successful.

[0223] Item 32. A computer-readable storage medium having written thereon instructions that, when executed by a computer processor, cause the computer processor to execute the instructions, the instructions comprising: receiving a configuration file for deployment to a first execution target and a second execution target; generating a first safety plan for the first execution target, the first safety plan including a first list of resources and respective actions associated with deployment on the first execution target based at least in part on the configuration file, and the instructions further include: receiving approval of the first safety plan; generating a second safety plan for a second execution target, the second safety plan including a second list of resources and respective actions associated with deployment at the second execution target based at least in part on the configuration file, and the instructions further include: determining whether the second safety plan is a subset of the first safety plan; Pursuant to a determination that the second safety plan is a subset of the first safety plan, automatically approving the second safety plan based at least in part on the approval of the first safety plan; and transmitting the automatically approved second safety plan to a second execution target.

[0224] Item 33. The Order further provides that, pursuant to a determination that the second safety plan is not a subset of the first safety plan: a notification indicating that the second safety plan is not a subset of the first safety plan; Item 33. The computer-readable storage medium of item 32, including providing a user interface for presenting a second list of resources and respective actions associated with the second safety plan.

[0225] Item 34. The computer-readable storage medium of item 33, wherein the instructions further include determining at least one difference between the second list of resources and respective actions of the second safety plan and the first list of resources and respective actions of the first safety plan.

[0226] Item 35. The computer-readable storage medium of item 34, wherein the instructions further include preparing a user interface for presenting the at least one difference above all other visual representations of the second list of resources and respective actions of the second safety plan.

[0227] Item 36. The computer-readable storage medium of item 32, wherein the second execution target is a disconnected region of a cloud infrastructure orchestration service, the disconnected region including an environment configured to receive instructions to deploy to the disconnected region, and the disconnected region is further configured not to transmit any data outside of the disconnected region.

[0228] Item 37. The second safety plan is for deployment on the second execution target. Item 37. The computer-readable storage medium of item 36, wherein the second safety plan is automatically approved further pursuant to a determination that deployment at the first execution target based at least in part on the first safety plan was successful.

[0229] Item 38. A system comprising: a processor; and a memory storing instructions, the instructions, when executed by the processor, receiving configuration files for deployment to the first execution target and the second execution target; and configuring the system to generate a first safety plan for a first execution target, the first safety plan including a first list of resources and respective actions associated with deployment on the first execution target based at least in part on the configuration file, the instructions further comprising: Receive approval of the first safety plan; and configuring the system to generate a second safety plan for a second execution target, the second safety plan including a second list of resources and respective actions associated with deployment at the second execution target based at least in part on the configuration file, the instructions further comprising: determining whether the second safety plan is a subset of the first safety plan; Pursuant to a determination that the second safety plan is a subset of the first safety plan, automatically approving the second safety plan based at least in part on the approval of the first safety plan; The system is configured to send the automatically approved second safety plan to a second execution target.

[0230] Item 39. The system, pursuant to a determination that the second safety plan is not a subset of the first safety plan, presenting a notice indicating that the second safety plan is not a subset of the first safety plan; Item 39. The system of item 38, configured to provide a user interface for presenting a second list of resources and respective actions associated with the second safety plan.

[0231] Item 40. The system further: determining at least one difference between the second list of resources and respective actions of the second safety plan and the first list of resources and respective actions of the first safety plan; Item 39. The system of item 39, configured to provide a user interface for presenting at least one difference over all other visual representations of the second list of resources and respective actions of the second safety plan.

[0232] Item 41. The system of item 38, wherein the second execution target is a disconnected region of a cloud infrastructure orchestration service, the disconnected region including an environment configured to receive instructions to deploy to the disconnected region, the disconnected region further configured to not transmit any data outside of the disconnected region, the second safety plan for deployment at the second execution target, the second safety plan being automatically approved further pursuant to a determination that the deployment at the first execution target is successful based at least in part on the first safety plan.

[0233] Item 42. An apparatus comprising means for the steps according to any one of Items 22 to 31.

Claims

1. 1. A method comprising: receiving, by at least one of the plurality of computer processors, a configuration file for deployment to a first execution target and a second execution target; and generating, by at least one of the plurality of computer processors, a first safety plan for the first execution target, the first safety plan including a first list of resources and respective actions associated with deployment in the first execution target based at least in part on the configuration file, the method further comprising: receiving, by at least one of the plurality of computer processors, an approval of the first safety plan; and generating, by at least one of the plurality of computer processors, a second safety plan for the second execution target, the second safety plan including a second list of resources and respective actions associated with deployment at the second execution target based at least in part on the configuration file, the method further comprising: at least one of the plurality of computer processors determining whether the second safety plan is a subset of the first safety plan; In response to determining that the second safety plan is a subset of the first safety plan, at least one of the plurality of computer processors automatically approving the second safety plan based at least in part on the approval of the first safety plan; and at least one of the plurality of computer processors transmitting the automatically approved second safety plan to the second execution target.

2. upon determining that the second safety plan is not a subset of the first safety plan, a notification indicating that the second safety plan is not a subset of the first safety plan; The method of claim 1 , further comprising: providing a user interface for presenting the second list of resources and respective actions associated with the second safety plan.

3. 3. The method of claim 2, further comprising determining at least one difference between the second list of resources and respective actions of the second safety plan and the first list of resources and respective actions of the first safety plan.

4. 4. The method of claim 3, further comprising: preparing the user interface to present the at least one difference above all other visual representations of the second list of resources and respective actions of the second safety plan.

5. 10. The method of claim 1, wherein determining whether the second safety plan is a subset of the first safety plan includes at least one of the plurality of computer processors determining whether each resource and each action in the second list of resources and each action of the second safety plan is listed in the first list of resources and each action of the first safety plan.

6. 10. The method of any one of the preceding claims, wherein the first execution target is a connection region of a cloud infrastructure orchestration service.

7. The method of claim 6 , wherein the second execution target is a disconnected region of the cloud infrastructure orchestration service.

8. 8. The method of claim 7, wherein the disconnected region includes an environment configured to receive instructions to deploy to the disconnected region, the disconnected region further configured to not transmit any data outside of the disconnected region.

9. The method of claim 7 or 8, wherein the second safety plan is for deployment on the second execution target.

10. 10. The method of claim 9, wherein the second safety plan is automatically approved further pursuant to a determination that deployment at the first execution target based at least in part on the first safety plan was successful.

11. 1. A computer-readable storage medium having instructions written thereon that, when executed by a computer processor, cause the computer processor to execute the instructions, the instructions comprising: receiving a configuration file for deployment to a first execution target and a second execution target; generating a first safety plan for the first execution target, the first safety plan including a first list of resources and respective actions associated with deployment on the first execution target based at least in part on the configuration file, the instructions further comprising: receiving approval of the first safety plan; generating a second safety plan for the second execution target, the second safety plan including a second list of resources and respective actions associated with deployment at the second execution target based at least in part on the configuration file, the instructions further comprising: determining whether the second safety plan is a subset of the first safety plan; In response to determining that the second safety plan is a subset of the first safety plan, automatically approving the second safety plan based at least in part on the approval of the first safety plan; and transmitting the automatically approved second safety plan to the second execution target.

12. The instructions further comprise, in response to a determination that the second safety plan is not a subset of the first safety plan: a notification indicating that the second safety plan is not a subset of the first safety plan; The computer-readable storage medium of claim 11 , further comprising providing a user interface for presenting the second list of resources and respective actions associated with the second safety plan.

13. 13. The computer-readable storage medium of claim 12, wherein the instructions further comprise determining at least one difference between the second list of resources and respective actions of the second safety plan and the first list of resources and respective actions of the first safety plan.

14. The instructions further include instructions for: determining whether the second safety plan resources and respective actions are associated with the second safety plan resources; 14. The computer-readable storage medium of claim 13, further comprising preparing the user interface to present the at least one difference above all other visual representations of a list.

15. 15. The computer-readable storage medium of claim 11, wherein the second execution target is a disconnected region of the cloud infrastructure orchestration service, the disconnected region including an environment configured to receive instructions to deploy to the disconnected region, and the disconnected region further configured not to transmit any data outside of the disconnected region.

16. 16. The computer-readable storage medium of claim 15, wherein the second safety plan is for the deployment at the second execution target, and the second safety plan is automatically approved further pursuant to a determination that the deployment at the first execution target based at least in part on the first safety plan is successful.

17. 1. A system comprising: a processor; a memory storing instructions that, when executed by the processor, receiving a configuration file for deployment to a first execution target and a second execution target; configuring the system to generate a first safety plan for the first execution target, the first safety plan including a first list of resources and respective actions associated with deployment at the first execution target based at least in part on the configuration file, the instructions further comprising: receiving approval of the first safety plan; and configuring the system to generate a second safety plan for the second execution target, the second safety plan including a second list of resources and respective actions associated with deployment at the second execution target based at least in part on the configuration file, the instructions further comprising: determining whether the second safety plan is a subset of the first safety plan; In response to determining that the second safety plan is a subset of the first safety plan, automatically approving the second safety plan based at least in part on the approval of the first safety plan; The system is configured to send the automatically approved second safety plan to the second execution target.

18. The system further comprises, in response to determining that the second safety plan is not a subset of the first safety plan, presenting a notification indicating that the second safety plan is not a subset of the first safety plan; 20. The system of claim 17, further configured to provide a user interface for presenting the second list of resources and respective actions associated with the second safety plan.

19. The system further comprises: determining at least one difference between the second list of resources and respective actions of the second safety plan and the first list of resources and respective actions of the first safety plan; 20. The system of claim 18, further configured to prepare the user interface for presenting the at least one difference above all other visual representations of the second list of resources and respective actions of the second safety plan.

20. 20. The system of claim 17, wherein the second execution target is a disconnected region of the cloud infrastructure orchestration service, the disconnected region including an environment configured to receive instructions to deploy to the disconnected region, the disconnected region further configured not to transmit any data outside of the disconnected region, the second safety plan is for deployment at the second execution target, and the second safety plan is automatically approved further pursuant to a determination that the deployment at the first execution target is successful based at least in part on the first safety plan.

21. Apparatus comprising means for the steps according to any one of claims 1 to 10.

Images