Technique for detecting drift in deployment orchestrater

Abstract

To provide a method, a storage medium, a system, and an apparatus for performing a cloud infrastructure orchestration service (CIOS).SOLUTION: A method for performing CIOS comprises: receiving a safety plan that includes a list of resources and actions based, at least in part, on a deployment configuration file; and upon receiving approval of the safety plan, preparing to perform a corresponding action to at least one of the list of resources, wherein the action is compared to the safety plan, and if the action is part of the safety plan, the action is executed and if the action is not part of the safety plan, the deployment is stopped; and transmitting notifications that the deployment is not compliant with the safety plan.SELECTED DRAWING: Figure 6

Description

[Background technology]

[0001] REFERENCE TO RELATED APPLICATIONS This application is a joint venture pursuant to 35 U.S.C. § 119(e) with U.S. application Ser. No. 62 / 963,413, filed on January 20, 2020, entitled "TECHNIQUES FOR DETECTING DRIFT IN A DEPLOYMENT ORCHESTRATOR," Ser. No. 17 / 027,527, filed on September 21, 2020, entitled "TECHNIQUES FOR DETECTING DRIFT IN A DEPLOYMENT ORCHESTRATOR," and Ser. No. 17 / 027,527, filed on September 21, 2020, entitled "TECHNIQUES FOR MANAGING DRIFT IN A DEPLOYMENT ORCHESTRATOR," This application claims the benefit of and priority to U.S. application Ser. No. 17 / 027,507, entitled "TECHNIQUES FOR MANAGING DRIFT IN A DEPLOYMENT ORCHESTRATOR," the entire contents of which are incorporated by reference for all purposes.

[0002] background Today, cloud infrastructure services utilize many individual services to provision and deploy code and configuration across many regions of the cloud infrastructure service. These tools require significant manual effort to use, especially considering that provisioning is largely declarative and deploying code is mandatory. Additionally, cloud infrastructure services must continue to grow as the number of service teams and regions grows. Some strategies for cloud infrastructure services to deploy to more and smaller regions include spending per region, but these may not scale well. Summary of the Invention

[0003] A quick overview Techniques for executing infrastructure orchestration services are described. In some examples, a safety plan may be received that includes a list of resources and operations based at least in part on a deployment configuration file. Upon receiving approval of the safety plan, an operation corresponding to at least one of the list of resources may be prepared to be executed. The operation may be compared to the safety plan. If the operation is determined to be part of the safety plan, the operation may be executed. If the operation is determined not to be part of the safety plan, the deployment may be stopped and a notification may be sent that the deployment is not compliant with the safety plan. In some examples, if drift occurs, the operation may not be part of the safety plan.

[0004] In another example, a computer-readable storage medium may include instructions that, when executed by a processor, may cause the processor to perform various operations described herein. A safety plan may be received, which may include a list of resources and operations based on a deployment configuration file. An approval of the safety plan may be received. An operation corresponding to at least one of the list of resources according to the deployment configuration file may be prepared to be executed. The operation may be compared to the safety plan. If the operation is determined to be part of the safety plan, the operation may be executed. If the operation is determined not to be part of the safety plan, the deployment may be stopped and a notification may be sent that the deployment is not compliant with the safety plan.

[0005] In yet another example, a system includes a processor and, when executed by the processor, and a memory capable of storing instructions capable of configuring the system to perform operations described herein. A safety plan may be received that may include a list of resources and operations based at least in part on the deployment configuration file. Approval of the safety plan may be received. Based on the approval, an operation corresponding to at least one of the list of resources may be prepared to be executed. The operation may be compared to the safety plan. If the operation is determined to be part of the safety plan, the operation may be executed. If the operation is determined to not be part of the safety plan, the deployment may be stopped and a notification may be sent that the deployment is not compliant with the safety plan.

[0006] In another example, the method may comprise at least one of the plurality of computer processors receiving a safety plan including a list of resources and respective operations based at least in part on the deployment configuration file. The method may further comprise, in accordance with receiving an indication of approval of the safety plan, at least one of the plurality of computer processors preparing to execute an operation corresponding to at least one of the list of resources according to the deployment configuration file. The method may further comprise comparing the operation to the safety plan. The method may further comprise, in accordance with a determination that the operation is part of the safety plan, at least one of the plurality of computer processors executing the operation. The method may further comprise, in accordance with a determination that the operation is not part of the safety plan, at least one of the plurality of computer processors stopping the deployment and at least one of the plurality of computer processors sending a notification that the deployment is not compliant with the safety plan.

[0007] In another example, a computer-readable storage medium may include instructions that, when executed by a processor, can cause the processor to perform various operations described herein. The operations may include receiving a safety plan including a list of resources and respective operations based at least in part on a deployment configuration file. The operations may further include preparing to perform an operation corresponding to at least one of the list of resources according to the deployment configuration file in accordance with receiving an indication of approval of the safety plan, and comparing the operation to the safety plan. The operations may further include performing the operation in accordance with a determination that the operation is part of the safety plan. The operations may further include stopping the deployment in accordance with a determination that the operation is not part of the safety plan, and sending a notification that the deployment is not compliant with the safety plan.

[0008] In another example, a system may include a processor and a memory capable of storing instructions capable of configuring the system to perform operations described herein when executed by the processor. The operations may include receiving a safety plan including a list of resources and respective operations based at least in part on a deployment configuration file. The operations may further include preparing to perform an operation corresponding to at least one of the list of resources according to the deployment configuration file in accordance with receiving an indication of approval of the safety plan. The operations may further include comparing the operation to the safety plan. The operations may further include performing the operation in accordance with a determination that the operation is part of the safety plan. The operations may further include stopping the deployment in accordance with a determination that the operation is not part of the safety plan and sending a notification that the deployment is not compliant with the safety plan.

[0009] An apparatus comprising means for the steps of any of the methods described herein. To easily identify the description of any particular element or act, the most significant digit(s) of a reference number refers to the figure number in which that element is first introduced. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram illustrating an architecture for implementing at least some elements of a cloud infrastructure orchestration service according to at least one embodiment. [Diagram 2] FIG. 1 is a block diagram illustrating an architecture for implementing at least some elements of a cloud infrastructure orchestration service according to at least one embodiment. [Diagram 3] FIG. 1 is a flow diagram illustrating an example of a flock according to at least one embodiment. [Figure 4] FIG. 1 is a flow diagram illustrating an example of a flock according to at least one embodiment. [Diagram 5] FIG. 1 is a block diagram illustrating a cloud infrastructure orchestration service according to at least one embodiment. [Figure 6] 1 is a flowchart illustrating a process for using a safety plan in a cloud infrastructure orchestration service according to at least one embodiment. [Figure 7] 1 is a flowchart illustrating a process for generating a safety plan in a cloud infrastructure orchestration service according to at least one embodiment. [Figure 8] FIG. 1 is a swim lane diagram illustrating how a safety plan is generated and used in accordance with at least one embodiment. [Figure 9] FIG. 1 is a block diagram illustrating a disconnected area according to at least one embodiment. [Figure 10] 1 is a flow chart illustrating a process for using a safety plan in a disconnected area according to at least one embodiment. [Figure 11] FIG. 1 is a block diagram illustrating a distributed system in accordance with at least one embodiment. [Figure 12] A block diagram illustrating one or more components of a system environment in which services provided by one or more components of an embodiment system may be provided as cloud services, according to at least one embodiment. [Figure 13] FIG. 1 is a block diagram illustrating an example of a computer system on which various embodiments of the present disclosure may be implemented. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Detailed Description In some examples, Infrastructure as a Service (IaaS) is one particular type of cloud computing. IaaS can be configured to provide virtualized computing resources over a public network (e.g., the Internet). In some examples, IaaS is one of three major categories (or subcategories) of cloud computing services. Most people consider the other major categories to be Software as a Service (SaaS) and Platform as a Service (PaaS), with SaaS sometimes considered a broader category that encompasses both PaaS and IaaS, and some even consider IaaS to be a subcategory of PaaS.

[0012] In the IaaS model, a cloud computing provider may host infrastructure components (e.g., servers, storage devices, network nodes (e.g., hardware), deployment software, or platform virtualization (e.g., hypervisor layer), etc.).

[0013] In some cases, IaaS providers may also provide various services (e.g., billing, monitoring, logging, security, load balancing, and clustering) to accompany these infrastructure components. Because the services can be policy-driven, IaaS users can enforce policies to drive load balancing to maintain application availability and performance.

[0014] In some examples, an IaaS customer can access resources and services over a wide area network (WAN), such as the Internet, and can use the cloud provider's services to install the remaining elements of the application stack. For example, a user can log into an IaaS platform, create virtual machines (VMs), install an operating system (OS) on each VM, deploy middleware (such as a database), create storage buckets for workloads and backups, and even install enterprise software on the VMs. The customer can then use the provider's services to perform a variety of functions, such as network traffic balancing, troubleshooting application issues, monitoring performance, and managing disaster recovery.

[0015] In most cases, the cloud computing model requires the participation of a cloud provider. A cloud provider can be, but does not have to be, a third-party service that specializes in providing (e.g., selling) IaaS. An entity may also choose to deploy a private cloud in which it becomes its own infrastructure service provider.

[0016] In some examples, IaaS deployment is the process of placing a new application or a new version onto a prepared application server, etc. It may also include the process of preparing the server (e.g., installing libraries, daemons, etc.). This is often managed by the cloud provider below the hypervisor layer (e.g., server, storage, network hardware and virtualization). Thus, the customer can be responsible for the processing (e.g., on top of self-service virtual machines (e.g., which can be spun up on demand)), (OS), middleware and / or application deployment, etc.

[0017] In some instances, IaaS provisioning may even mean obtaining computers or virtual hosts for use and installing the required libraries or services on them. In most cases, deployment does not include provisioning, which may need to be performed first.

[0018] In some cases, there are two different problems with IaaS provisioning. First, there is the initial challenge of provisioning the initial set of infrastructure before anything is running. Second, there is the challenge of deploying the existing infrastructure once everything is provisioned (e.g., adding new services, modifying services, removing services, etc.). In some cases, these two challenges can be addressed by allowing the configuration of the infrastructure to be defined declaratively. In other words, the infrastructure (e.g., which components are required and how they interact) can be defined by one or more configuration files. Thus, the entire topology of the infrastructure (e.g., which resources depend on which and how each of them works together) can be described declaratively. In some instances, once the topology is defined, workflows can be generated that create and / or manage the different components described in the configuration files.

[0019] In some examples, the infrastructure may have many interconnected elements, such as one or more virtual private clouds, also known as a core network. There may be a Virtual Private Cloud (VPC) (e.g., a configurable and / or shared, possibly on-demand pool of computing resources). In some examples, there may also be one or more security group rules provisioned to define how the network's security is set up, and one or more virtual machines (VMs). Other infrastructure elements, such as load balancers or databases, may also be provisioned. The infrastructure may evolve in stages as more infrastructure elements are required and / or added.

[0020] As mentioned above, one way to provision infrastructure is to describe it declaratively. As such, the configuration file may be a declarative file that simply describes each of the infrastructure components described above and how they interact. The configuration file may describe resources and associated fields required to create the elements, and may also describe other elements that reference previously described elements. In some examples, a provisioning tool may then generate workflows to create and manage the elements described in the configuration file.

[0021] In some examples, the provisioning tool workflow can be configured to execute various commands. One function that can be executed is a view adjustment, where the provisioning tool can compare a view of the current infrastructure (e.g., the expected state of the infrastructure) to how the infrastructure is actually running. In some examples, executing the view adjustment function can include querying various resource providers or infrastructure resources to identify which resources are actually running. Another function that the provisioning tool can execute is a plan generation, where the provisioning tool can compare the infrastructure components that are actually running to how the provisioning tool wants them to look (e.g., the desired configuration). In other words, the plan generation function can determine what changes need to be made to bring the resources closer to the latest expectations. In some examples, a third function is an execution (e.g., apply) function, where the provisioning tool can execute the plan generated by the plan generation function.

[0022] In general, a provisioner can be configured to receive a configuration file, parse the declarative information contained therein, and programmatically / automatically determine the order in which resources need to be provisioned to execute the plan. For example, if security group rules and VPCs need to be booted before VMs are booted, the provisioner can make that determination and perform the boots in that order without user intervention and / or even though that information is not necessarily included in the configuration file.

[0023] In some examples, continuous deployment techniques may be employed to enable deployment of infrastructure code across various virtual computing environments. Additionally, the described techniques may enable infrastructure management within these environments. In some examples, a service team may write code that is desired to be deployed to one or more (but often many) different production environments (e.g., spanning a variety of different geographic locations, sometimes spanning the entire world). However, in some examples, the infrastructure into which the code will be deployed must first be configured. In some examples, provisioning may be done manually, and a provisioning tool may be utilized to provision resources, and / or deployment may be performed once the infrastructure is provisioned. You can deploy the code using the deployment tool.

[0024] As mentioned above, there are typically two different tools used to handle the provisioning of infrastructure resources and the deployment of code to control the infrastructure resources, and the orchestration between these two tools is done manually. However, at scale, manual execution always leads to misalignment. Therefore, an automated tool that can both provision and deploy virtual infrastructure allows for a more efficient and reliable technique for realizing virtual cloud environments.

[0025] In some instances, when two tools are used, problems can arise when a user manually makes changes to the code between the provisioning and deployment phases. As described herein, the technique of using a single tool for both provisioning and deployment can mitigate that problem by automating the process so that there is no opportunity for manual code changes. A small change to the way a user codes something can cause major problems in the deployment phase. In some instances, the first time an operator performs an action in a new area (e.g., a typo in the code), the object coded with the typo can remain there forever. If an application is deployed with the typo but the application is not affected by the typo (e.g., the application still works), it can happen that further code changes in the future are affected by the typo and crash the entire system. Thus, the techniques provided herein can eliminate the gap between provisioning and deployment that can often cause problems.

[0026] Typically, modeling of deployments is declarative, such that infrastructure resources can be declared using configuration files. For example, typically, create-read-update-delete (CRUD) instructions are used to generate the deployment files using generic representable state transfer (REST) ​​concepts (e.g., REST application programming interfaces (APIs)). However, the deployment itself does not typically follow this concept. Furthermore, while infrastructure provisioning tools tend to be very powerful and / or expressive, tools for deployment tend to be much more restrictive in terms of the operations they can perform (e.g., they are imperative as opposed to being declarative). Thus, there has been a long-standing need for tools that can address both functional requirements (e.g., provisioning and deployment of infrastructure elements) within a cloud environment.

[0027] In some examples, techniques for implementing a cloud infrastructure orchestration service (CIOS) are described herein. Such techniques can be configured to manage both the provisioning and deployment of infrastructure assets in a cloud environment, as briefly described above. In some examples, a CIOS can include two classes of services: a central component and a regional component (e.g., a central CIOS and a regional CIOS). The following terms are used throughout the specification:

[0028] Infrastructure components: Long-lived pieces of infrastructure that support running code Examples: deployment applications, load balancers, Domain Name System (DNS) entries, object storage buckets, etc.

[0029] Artifact: Code to be deployed to a deployment application or Kubernetes engine cluster, or configuration information (hereafter "config") to be applied to an infrastructure component. These can be read-only resources.

[0030] Deployment Tasks: Short-lived tasks that are often associated with deploying or testing code. Furthermore, deployment tasks are modeled as resources that do not live as long as the releases that create them.

[0031] For example: "deploy $artifact to $environment", "watch $alarm for 10 minutes", "execute $testSuite", or "wait for $manualApproval" For example, CIOS can model a deployment orchestrator's deployment as the creation of a resource that transitions to an available state when it is complete.

[0032] Because CIOS maintains the state of its associated declarative provisioners, CIOS can control the lifecycle of these short-lived resources when it is involved in a release.

[0033] Resource: A CRUD-able resource CIOS models each of the above constructs as a resource, and the next section describes this modeling in detail.

[0034] Flock: The only control plane resource in CIOS. It exists primarily to represent infrastructure components by modeling their ownership.

[0035] Flock config: describes the set of all infrastructure components, artifacts and deployment tasks related to a single service.

[0036] Each flock has exactly one flock config, which is checked into source control.

[0037] Flock configs are declarative: they expect CIOS to provide the domain, region, ad, and artifact version as input.

[0038] A flock is fine-grained: a flock consists of a single service and supporting infrastructure.

[0039] · State: A snapshot of the state of every resource in a flock at a particular point in time. Release: A tuple of a specific version of a flock config and the specific versions of any artifacts it references. The release is considered to describe a state that may not yet exist.

[0040] Release Plan: The set of steps CIOS will take to transition all regions from their current state to the state described by the release.

[0041] A release plan has a finite number of steps and clearly defined start and end times. and

[0042] · Apply: This is a noun. It is one attempt to execute a release plan. An execution changes the current state of the flock.

[0043] CIOS can be described as an orchestration layer that applies configuration to downstream systems (e.g., globally). It is designed to enable infrastructure provisioning and code deployments globally without manual effort from service teams (e.g., potentially beyond initial approval). High-level responsibilities of CIOS include, but are not limited to:

[0044] · Providing the team with a view into the current state of resources managed by the CIOS, including any ongoing change activity.

[0045] · Helping the team plan and release new changes. · Coordinate activities across various downstream systems in the region to execute approved release plans without human intervention.

[0046] ·Coordinate activities across regions / territories to execute approved release plans globally.

[0047] It handles onboarding by allowing teams to provide information to CIOS. This is more work than previous implementations, as CIOS can also automate more things. In some examples, CIOS handles pre-deployment by providing the team with the ability to automatically deploy and test their code. In some examples, CIOS can handle writing change management (CM) policies by allowing the team to automatically generate plans for deploying new artifacts (e.g., globally) as they build them. This can be done by looking at the current state of each region and the current CIOS config (which can itself be an artifact). Additionally, the team can look at these plans and iterate on them by asking CIOS to change the CIOS config and replan. Once the team is happy with the plan, they can create a "release" that references the plan. This plan can then be marked as approved or rejected. The team can still write CMs, but they are just pointers to the CIOS plan. Thus, the team does not spend as much time reasoning about the plan. The plans are more accurate because they are machine generated. The plans are too detailed for humans to read, but can be displayed through a sophisticated user interface (UI).

[0048] In some examples, CIOS can handle CM execution by automatically executing deployment plans. Once a release plan is created and approved, engineers are not involved in CM unless CIOS initiates a rollback. In some cases, this requires teams to automate tasks that are currently manual. In some examples, CIOS can handle change management (CM) rollback by automatically generating a plan to revert a flock to its original (e.g., pre-release) state when CIOS detects degradation in service health during execution. In some examples, CIOS can handle the deployment of sudden / tactical changes by receiving a release plan that is scoped to a subset of regions and / or resources managed by CIOS, and then executing that plan.

[0049] Additionally, CIOS may support the primitives necessary to define fully automated, worldwide deployments. For example, CIOS may measure service health by monitoring alarms and running integration tests. CIOS may help teams quickly define rollback behavior in case of service degradation, which can then be executed automatically. CIOS may automatically generate and display release plans, and may track approvals. In some instances, the language that teams use to describe the desired deployment behavior may be declarative. CIOS may combine the functionality of code deployment and infrastructure configuration (e.g., provisioning) in one system. CIOS also supports flexible ordering across regions and across components within a region. Teams may express ordering through checked-in configurations. Teams may call CIOS planning and release APIs programmatically.

[0050] 1 is a diagram illustrating an architecture 100 to illustrate a technique for implementing at least a central CIOS 102. In some examples, the central CIOS 102 may be a service that handles operations at the level of a "flock." The central CIOS 102 has several responsibilities, including but not limited to:

[0051] Acting as an authentication gateway for flock metadata changes and release operations.

[0052] · Storing deployment artifacts for Flock and a formal mapping of Flock metadata to the CIOS repository.

[0053] · Coordinate global releases across phases and targets. · Synchronization to enforce policies like "only one ongoing release to a flock at a time".

[0054] Detecting changes to flock configurations and artifacts and triggering release generation upon such changes.

[0055] In some examples, a source code version control management service (SCVMS) 104 can be configured to store the authoritative flock configuration, and an artifact notification service (ANS) 106 can be subscribed to by the central CIOS 102 so that the central CIOS 102 can be informed of the construction of new artifacts. The central CIOS 102 can then map the incoming changes against the affected flocks and initiate release planning if desired. Additionally, in some examples, an artifact push service (APS) can be invoked by the central CIOS 102 prior to a release to a target to ensure that any artifacts required for a successful release are present in the target's geography prior to the release.

[0056] In some examples, customers (e.g., engineers) 108 can call the central CIOS 102 to CRUD flocks and / or releases to see the status of ongoing CIOS activity, a flock management service 110 can include one or more APIs for manipulating flocks, a view / plan / approval service 112 can include CRUD APIs for creating and approving plans and viewing a central copy of the state of resources managed by all CIOS, a change monitoring service 114 can watch the SCVMS 104 for changes to flock configurations and can receive notifications from the ANS 106 for changes to other artifacts, The State Ingester Service 116 can create copies of the regional states in a central CIOS database (DB) 118 so that the Views / Plans / Approvals 112 can expose them. In some examples, the central CIOS DB 118 can be a DB of flocks, plans and states. The flock information can be authoritative, but everything else can be stale copies of data from the regional CIOS 120.

[0057] In some examples, the engineer 108 can make an API call to the flock management service 110 (e.g., via the ingress proxy fleet 122) to create a list of flocks. The protocol for making such an API call can be Hypertext Transport Protocol Secure (HTTPS), or the like. The associated access control list (ACL) for this operation can include a local area network (LAN) 124 or other private connection. For example, the CIOS can manage / control network connections (e.g., dedicated connections, leased connections, and / or private connections) that replace the use of the public Internet to connect the customer's on-premise data center or network with the CIOS. Additionally, authentication and authorization (e.g., of the engineer 108) can be performed by a reservation system portal that allows the user to manage a machine interface (e.g., a reservation service). In some examples, the central CIOS 102 can store flock metadata, plans, and states in the central DB 118 using Java Database Connectivity (JDBC), or the like. In some examples, the ANS 106 can be configured to notify the change monitoring service 114 when new artifacts are published. The ANS 106 can use HTTPS, and both authentication and authorization can be handled by mutual transport layer security services. Additionally, in some examples, the change monitoring service 114 can poll the SCVMS 104 for block configuration changes. This polling can be performed using Secure Shell (SSH) or other protocols. Authentication of the change monitoring service 114 can be handled by a CIOS system account, and authorization can be handled by the SCVMS 104.

[0058] In some examples, the engineer 108 can use the view / plan / approval service 112 to perform one or more of the following operations: The engineer 108 can plan and / or approve by calling the central CIOS 102 to generate and approve a plan. The engineer 108 can view by calling the central CIOS 102 to see the status of ongoing CIOS activities worldwide. Additionally, the engineer 108 can use the central CIOS 102 to browse a replica of the state of resources managed by the worldwide CIOS. These API calls (or the like) can be made via HTTPS protocol or a similar protocol. Additionally, the associated ACLs can be controlled by the LAN 124, and both authentication and authorization can be handled by the reservation service. In some examples, the view / plan / approval service 112 can request a plan (e.g., using HTTPS, etc.) and push the approval of the plan to all regions of the regional CIOS 120. The associated ACLs can be controlled using security lists managed by the wide area network (WAN) gateway 126. Authentication can be handled by mutual transport layer security, and authorization can be handled by various identity policies. Additionally, the status ingestor service 116 can look to the regional CIOS 120 for job status or state changes so that the CIOS can provide a central view of those statuses or state changes upon request (e.g., also using HTTPS, etc.). ACLS for this can also be handled by the WAN gateway 126, and both authentication and authorization can be handled by mutual transport layer security services.

[0059] 2 is a diagram illustrating an architecture 200 to illustrate a technique for executing at least a regional CIOS 202. In some examples, the regional CIOS 202 is where much of the declarative provisioning and planning work, as well as approved release applications, can occur. In some examples, each instance of the regional CIOS 202 can have a regional front-end that can handle operations at the level of an "execution target." It can be configured to do the following:

[0060] · Handling all CIOS authentication for incoming actions from the central CIOS 102.

[0061] · Enforcing the rule that for a given execution target, only one "execution" (plan / resource import / apply plan) can occur at a time.

[0062] Managing binary artifact storage for declarative provisioning artifacts used as inputs and outputs during declarative infrastructure provisioning runs. Examples of inputs are declarative infrastructure provisioning configuration files and input state files. A typical output is a final state file.

[0063] · Requesting work from the CIOS executor and polling for results from the CIOS executor for any given execution.

[0064] In some examples, the CIOS front end may rely on the CIOS executor, which can handle the actual execution. In some examples, the CIOS executor operates at the level of "execution", which can:

[0065] Tracking the pool of available worker nodes · Queries incoming job requests and assigns them to qualified workers as available Tracking worker status and execution updates to report to clients - Dead nodes can be detected by the leasing protocol and, depending on the task status, tasks assigned to the dead node can be failed. Provide functionality to cancel / kill / pause / resume execution, and be able to map these to functionality for passing the cancel / kill / resume information to worker nodes. In some examples, the CIOS executor may rely on CIOS workers, which may assign tasks to workers for execution and provide functionality for workers to update job progress. Worker services operate at the granularity of a "task". Each worker is an agent that executes the tasks assigned to it and reports task status and output. Each worker can do the following:

[0066] Polling the Execute Worker API for assigned work items and taking action to make the assigned state consistent with its local state Starting a container to poll for task items that do not exist locally Killing a container for locally running containers that do not have a corresponding assigned task item Reporting the status of a job -Perform stage input and output for job container execution Launching and monitoring declarative infrastructure provisioning containers to do the actual work of releasing execution targets A CIOS worker may depend on a CIOS executor to poll for work from the CIOS executor and report results to the CIOS executor's worker endpoint. The worker may rely on the executor for all coordination. Additionally, a CIOS worker may depend on a regional CIOS 202 where the worker service reads input from and writes output to one or more APIs associated with the regional front-end services. Examples of inputs are configuration and start state files, and import of mappings. Examples of outputs are the declarative provisioning process, the output of the declarative provisioning state file, and the import of the result state.

[0067] In some examples, regional CIOS 202 can be a regional service for managing regional instances / deployments of CIOS. Regional CIOS 202 covers the responsibility of authoritatively storing and managing plans and states related to a particular region. Regional DB 204 can be a CIOS DB for state and plans in that particular region. It is an authoritative copy of the regional subset of central DB 118 of FIG. 1. Scheduler 206 can be responsible for managing worker fleet capacity, assigning tasks to workers, and tracking task state progress. In some examples, task DB 208 is another CIOS DB for task state. The data in this DB is mostly for operational purposes. Additionally, workers 210 can be a fleet of java virtual machines (JVMs) that manage declarative provisioning images. They receive instructions from scheduler 206 and communicate results to both scheduler 206 and regional CIOS 202. CIOS container 212 can execute declarative provisioning actions in its own private docker 214 container. This container may not contain any secrets. Additionally, in some examples, the signing proxy 216 can be configured to prevent exfiltration of secrets by declarative provisioning tools to avoid putting secrets in the declarative provisioning image. Instead, the CIOS can perform signing requests or initiate mutual Transport Layer Security (mTLS) services at the proxy. This facilitates the use of FIPS-compliant cryptographic libraries.

[0068] In some examples, central CIOS 102 can call regional CIOS 202 to create plans, push approvals, watch job status (service principals), and extract declarative provisioner state (service principals). Ingress proxy 218 can be configured as an ACL, and various identity policies can be used for both authentication and authorization. In some examples, regional CIOS 202 can run the declarative provisioner by asking scheduler 206 to do so. Worker 210 can ask scheduler 206 what it is supposed to be running, and can report status to scheduler 206 upon completion. In some cases, mTLS may handle both authentication and authorization for regional CIOS 202 and worker 210. Additionally, worker 210 runs the declarative provisioner in a docker container by interacting with local docker 214 if it needs to run the declarative provisioner. Authentication in this phase can be handled by a local unix socket. This final step may use the Docker protocol, while the previous steps may utilize HTTPS.

[0069] In some examples, the CIOS container 212 allows the declarative provisioner to interact (through an API) with the signing proxy 216, which the declarative provisioner thinks it is invoking various CIOS services. The signing proxy 216 listens on one ephemeral port per invocation of the declarative provisioner instance, known only to the declarative provisioner. The signing proxy 216 sends signing or mTLS requests. The signing proxy 216 may initiate a declarative provisioner call and pass the declarative provisioner call to other CIOS services in the service enclave. In some examples, the signing proxy 216 may also communicate with one or more public CIOS services 220. For example, the signing proxy 216 uses the internal endpoints of the public services when possible. Services that do not have internal endpoints must use the egress proxy 222 to reach the external endpoints. This use of the signing proxy 216 may not be for cross-regional communication. For example, each region's egress proxy whitelist may be just for the public IP ranges of that region. In some examples, the worker 210 may then maintain state and logs from the declarative provisioners in the regional CIOS 202 so that they can be spilled to the central CIOS 102.

[0070] Using CIOS, a typical customer experience has several phases: onboarding, pre-release, global release, and tactical release. For the pre-release phase, the following is an example of what happens between building a new artifact and releasing the artifact to Release 1 (e.g., R1). This should replace some or most of the current change management process. Once the relevant artifacts are built, CIOS can automatically generate a release using "all the latest versions in the flock". A release is a specific version of a flock configuration with specific inputs (e.g., artifact version, region, area, and ad). A release contains one rollforward plan per region and metadata that describes the region ordering. Each region plan is a set of actions that the declarative provisioner will take to realize the flock configuration in that region. Teams with pre-release environments can use CIOS to automatically release and test software in the above environments. Teams can configure CIOS to automatically test rollback plans. Teams will be able to inspect and approve releases via the CIOS UI. Teams can approve some (but not all) of the local plans within a release, and if "latest version of everything" does not result in a suitable plan, the team can ask CIOS to generate plans for cherry-picked artifact versions.

[0071] For a worldwide release phase, the following is an example of how a team might execute tomorrow's version of today's "normal CM". Once a release is approved, CIOS pushes each approved regional plan to its respective region. CIOS works independently within each region to apply the approved plan. CIOS only executes the set of actions explicitly described in the plan for that region. It will fail rather than "thinking independently". CIOS UI shows the team the progress of the execution. CIOS UI prompts the team if manual approval is required. If the execution fails due to an outage in CIOS or a downstream service, CIOS can notify the team and prompt the team for next steps (e.g., abort, retry). CIOS performs retries, but some outages in downstream systems may be beyond its willingness to retry. If the execution fails due to degradation of service health or test failure, CIOS assists the team by rolling back the flock to its starting state. CIOS notifies the team (e.g., paging) when it initiates an automatic rollback. The team must approve the rollback plan, and then the CIOS executes it.

[0072] For the Tactical Release phase, below is an example of how a team could execute tomorrow's version of "Sudden CM". When generating the plan, the team A team may ask CIOS to target plans to specific resources in any way: topologically (e.g., region, geography, AD, etc.), by resource type (e.g., "metrics config only" or "deployment orchestration service deployments only"), or a combination of the above (e.g., in a disjunctive manner). Teams approve tactical releases as well as global releases. CIOS orchestrates them similarly. If a team needs to deploy a tactical release while there is an active global release, CIOS will stop running the global release in the target region and then start running the tactical release.

[0073] In some examples, the declarative provisioner's state (e.g., traditionally a file) is a formal record of the set of resources managed by the declarative provisioner. It contains a mapping between each resource's logical identifier from the configuration and the resource's actual identifier. When the declarative provisioner creates a resource, some kind of failure may occur that prevents the actual identifier from being recorded in the state. When this happens, the actual identifier is lost to the declarative provisioner. These are sometimes called "orphaned resources".

[0074] For most resources, orphans mean waste: the declarative provisioner will launch an instance it forgot about (for example), and the next time it runs, launch another instance instead. For resources with uniqueness constraints or client-supplied identifiers, orphans prevent the declarative provisioner from moving forward. For example, if the declarative provisioner creates a user "nglass" and then makes it an orphan due to a failure, the next run of the declarative provisioner will fail when attempting to create "nglass" because a user with that username already exists. In some cases, orphans are only an issue when adding new resources to the state. In some instances, the declarative provisioner's refresh behavior may naturally recover from failures to record updates and deletions.

[0075] CIOS must be robust in the event of downstream service outages or CIOS itself being outaged. Since CIOS can leverage the declarative provisioner to apply changes, this means it must be robust in running the declarative provisioner and maintaining the declarative provisioner state. The declarative provisioner provider performs "small" retries that are sufficient to avoid outages that last several minutes. For example, a cloud provider may perform retries for up to 30 minutes. If the downstream system is out for longer than 30 minutes, the declarative provisioner fails. When the declarative provisioner fails, it records all successful changes in its state and exits. To retry, CIOS must rerun the declarative provisioner. Rerunning the declarative provisioner allows CIOS to perform retries in the event that CIOS itself fails. In some examples, CIOS may perform the following actions in a loop:

[0076] · Refresh: The declarative provisioner calls the "get API" to retrieve a new snapshot of any resources listed in its state.

[0077] · Plan: The declarative provisioner generates a plan (a concrete set of API calls) that will achieve the desired state, given the recently refreshed current state.

[0078] · Apply: The declarative provisioner executes a set of steps in the plan. CIOS may always perform all three of these steps when running a declarative provisioner. A refresh operation removes any updates or deletions that were not recorded. It also helps in recovery of the release plan. The CIOS looks at the results of the plan actions and compares it to the approved release plan. If the newly generated plan contains actions that were not in the approved release plan, the CIOS may fail and notify the service team.

[0079] FIG. 3 illustrates a directed acyclic graph (DAG) 300 for illustrating an example of a flock 302. For a flock config in CIOS, the progression of code / config from check-in to production can be described all the way from the first test deployment to the final prod deployment. Internally in CIOS, each element in the progression is called an execution target (ET), which spans our internal APIs but does not leak to the flock config. CIOS executes the ETs based on the DAG 200 defined in the flock config. Each ET (e.g., ET-1, ET-2, ET-3, ET-4, ET-5, ET-6, and ET-7) is roughly a copy of a service described by the flock config.

[0080] FIG. 4 illustrates a DAG 400 to illustrate an example flock 402. In the flock configuration, CIOS is very opinionated about how teams express this progress. Teams must model it using cloud infrastructure tenancies and regions. Teams should not model progress using regions. CIOS allows teams to use many tenancies in a region and many regions in a tenancy. However, CIOS does not allow teams to use the same region twice in one tenancy (although it is possible to use the same region twice in one region in different tenancies). DAG 400 shows a version of DAG 300 from FIG. 3 represented with tenancies and regions. This example is for an overlay service with a pre-prod ET in the prod region. A service enclave service will have an unstable tenancy and a stable tenancy in release 1. In DAG400, IAD is the regional airport code for Dulles Airport in Washington DC, YYZ is the regional airport code for Toronto, Ontario, PHX, LHR and FRA are the regional airport codes for Phoenix, London and Frankfurt respectively, and LUF and LFI are for two different Air Force bases.

[0081] In one embodiment, the CIOS and / or other technologies described herein are improvements to each of Terraform (a declarative provisioning tool), Tanden (a code generation tool), and ODO (Oracle Deployment Orchestrator). Further, in some examples, the CIOS and / or other technologies described herein may be implemented using at least a portion of the Terraform, Tanden, and ODO tools.

[0082] In some examples, a safety plan may be generated to safely handle drift. The configuration file may contain information that directs how the CIOS makes changes to a region (e.g., a geographic region where the cloud infrastructure is (or has been) configured). The CIOS may verify that what the client reviews as a change set (e.g., the delta between the current state of the cloud infrastructure and the expected new state after the change) actually changes in the region. When the client interacts with the CIOS to generate a global plan, the safety plan identifies what changes are expected to be made by the CIOS based on the now world (e.g., current state) to arrive at the world (e.g., target state) described in the configuration file. As described herein, "world" may also refer to the expected state of the cloud infrastructure, including all settings and configurations of the cloud setup. In some cases, drift is the idea that one or more device configurations / settings may have changed between the time the plan is generated and the time the deployment of the region begins (e.g., (e.g., a virtual machine may be failing). That change is the drift. CIOS can create a safety plan and send it to different regions around the world (e.g., send it to different CIOS in different regions). Whenever CIOS is about to implement a change, it can repeat the planning task. For example, it can predict the current state, compare it to the goal state, and generate a diff (e.g., delta, difference) between the two. That diff is the safety plan subsection. If it is a subset of the first safety plan, then drift has not occurred and the safety plan can still be implemented. Otherwise, an issue with the safety plan has been detected and changes need to be made before deployment. Additionally, in some examples, the CIOS can compare different safety plans (e.g., to handle phase changes). For example, for a deployment to a phase (e.g., deploying resources in a first phase before other phases), the safety plan may be reviewed and approved, and the deployment may be performed accordingly. Then, at a later time, for a deployment of phase 2, a new safety plan may be generated and compared to the first safety plan. If the new safety plan is a subset of the first safety plan, then the new safety plan may be automatically approved without sending anything to the client that generated the configuration file. It will be appreciated that this can speed up the deployment process since no delays are introduced associated with waiting for approval of the new safety plan, but also maintains the integrity of the deployment since no drift has occurred in such a situation. Alternatively, if there is something new in the new safety plan (e.g., something not present in the first safety plan), the new aspects can be presented to the user (e.g., the client) via a user interface (UI). The user can then approve or reject the new aspects (eg, allow or not allow a deployment change to occur).

[0083] Techniques for performing infrastructure orchestration services are described. In some examples, a safety plan may be received that includes a list of resources and actions based at least in part on a deployment configuration file. Upon receiving approval of the safety plan, an action corresponding to at least one of the list of resources may be prepared to be performed. The action may be compared to the safety plan. If the action is determined to be part of the safety plan, the action may be performed. If the action is determined to not be part of the safety plan, the deployment may be stopped and a notification may be sent that the deployment is not compliant with the safety plan. In some examples, if drift is occurring, the action may not be part of the safety plan.

[0084] In another example, a computer-readable storage medium may include instructions that, when executed by a processor, may cause the processor to perform various operations described herein. A safety plan may be received that may include a list of resources and operations based on a deployment configuration file. An approval of the safety plan may be received. An operation corresponding to at least one of the list of resources according to the deployment configuration file may be prepared to be executed. The operation may be compared to the safety plan. If the operation is determined to be part of the safety plan, the operation may be executed. If the operation is determined not to be part of the safety plan, the deployment may be stopped and a notification may be sent that the deployment is not compliant with the safety plan.

[0085] In yet another example, a system includes a processor and, when executed by the processor, and a memory capable of storing instructions capable of configuring the system to perform the operations described herein. A safety plan may be received that may include a list of resources and operations based at least in part on the deployment configuration file. Approval of the safety plan may be received. Based on the approval, an operation corresponding to at least one of the list of resources may be arranged to be performed. The operation may be compared to the safety plan. The operation may be compared to the safety plan. If it is determined that the action is part of the safety plan, the action may be executed. If it is determined that the action is not part of the safety plan, the deployment may be stopped and a notification may be sent that the deployment is not compliant with the safety plan.

[0086] In another example, the method may include at least one of the plurality of computer processors receiving a safety plan of a list including resources and respective operations based at least in part on a deployment configuration file. The method may further comprise, in accordance with receiving an indication of approval of the safety plan, at least one of the plurality of computer processors preparing to execute an operation corresponding to at least one of the list of resources according to the deployment configuration file. The method may further comprise comparing the operation to the safety plan. The method may further comprise, in accordance with a determination that the operation is part of the safety plan, at least one of the plurality of computer processors executing the operation. The method may further comprise, in accordance with a determination that the operation is not part of the safety plan, at least one of the plurality of computer processors stopping the deployment and at least one of the plurality of computer processors sending a notification that the deployment is not compliant with the safety plan.

[0087] In another example, a computer-readable storage medium may include instructions that, when executed by a processor, can cause the processor to perform various operations described herein. The operations may include receiving a safety plan including a list of resources and respective operations based at least in part on a deployment configuration file. The operations may further include preparing to perform an operation corresponding to at least one of the list of resources according to the deployment configuration file in accordance with receiving an indication of approval of the safety plan, and comparing the operation with the safety plan. The operations may further include performing the operation in accordance with a determination that the operation is part of the safety plan. The operations may further include stopping the deployment in accordance with a determination that the operation is not part of the safety plan, and sending a notification that the deployment is not compliant with the safety plan.

[0088] In another example, a system may include a processor and a memory capable of storing instructions that, when executed by the processor, configure the system to perform operations described herein. The operations may include receiving a safety plan including a list of resources and respective operations based at least in part on a deployment configuration file. The operations may further include preparing to perform an operation corresponding to at least one of the list of resources according to the deployment configuration file in accordance with receiving an indication of approval of the safety plan. The operations may further include comparing the operation to the safety plan. The operations may further include performing the operation in accordance with a determination that the operation is part of the safety plan. The operations may further include stopping the deployment in accordance with a determination that the operation is not part of the safety plan and sending a notification that the deployment is not compliant with the safety plan.

[0089] FIG. 5 is a block diagram 500 illustrating a CIOS that can generate and use safety plans, according to at least one embodiment. In some examples, a safety plan can be a set of actions approved by a user and executable by the CIOS. Block diagram 500 illustrates an example architecture of a CIOS that can include a central control plane 502 and a regional data plane 504. Central control plane 502 can include a central CIOS 506 (e.g., similar to central CIOS 102 of FIG. 1 ), which can include a control plane 508, a change management module 510, a view / plan / approval module 512, and a state ingestor 514. Central control plane 508 and view / plan / approval module 512 are communicatively coupled to a local area network (LAN) gateway 516 that users 518 can use to communicate with central CIOS 506. The view / plan / approval module 512 and the state ingestor 514 may be communicatively coupled to a wide area network (WAN) 520 (e.g., the Internet), which may be communicatively coupled to a regional CIOS 522 (e.g., similar to the regional CIOS 120 of FIG. 1 or the regional CIOS 202 of FIG. 2) included in the regional data plane 504.

[0090] Regional CIOS 522 may be communicatively coupled to a scheduler node 524 and a worker node 526. Worker 526 may be communicatively coupled to scheduler 524, which may be communicatively coupled to a task database (DB) 528. Worker 526 may be communicatively coupled to a docker 530, which may include a CIOS container 532. CIOS container 532 may be communicatively coupled to a signing proxy 534, which may be communicatively coupled to a cloud service 536.

[0091] Regional CIOS 522 can receive tasks from central CIOS 506 and can send tasks to scheduler 524. The tasks can include performing CRUD operations or any other suitable tasks for deploying infrastructure resources to the region of the execution target(s). Scheduler 524 can record the tasks in task DB 528 and can send the tasks to workers 526, which can be included in a worker fleet. A worker fleet can include many workers 526, and scheduler 524 can select a worker 526 with a minimum amount of work or a maximum amount of available computing resources to assign the task to. Scheduler 524 can assign one task at a time to a worker 526. Worker 526 can execute the task, and in executing the task, the worker can make a call to a CIOS container 532 included in docker 530. CIOS container 532 can execute the task including infrastructure provisioning and / or deployment instructions (e.g., Terraform instructions). The instructions can direct the API call to cloud services 536, which may include locally available services that may not be available over a public network (e.g., the Internet). To make an API call to cloud services 536, the CIOS container 532 can send a request to make the API call to signing proxy 534, which can determine whether the request is valid. In response to determining that the request is valid, signing proxy 534 can make the API call to cloud services 536.

[0092] In some examples, a user 518 may create a configuration file that may include operations to execute in the regional data plane 504. The user 518 may send the configuration file via a LAN gateway 516 to a central CIOS 506, which may be a computing device included in the central control plane 502. The configuration file may be received by the central CIOS 506 in the control plane 508 or in a view / plan / approval module 512. The change management module 510 may then convert the configuration file into a region-agnostic (RA) configuration file that may be used in the regional data plane 504. The view / plan / approval module 512 or state ingestor 514 may send the RA configuration file via the WAN gateway 520 to the regional CIOS 522, which may be a computing device included in the regional data plane 504.

[0093] Regional CIOS 522 can receive RA configuration files from central CIOS 506, which can send the RA configuration files to scheduler 524, which can create tasks and send the tasks to task DB 528. A task may include compiling a set of operations to be executed on an execution target, and the execution The execution target may include comparing a current state of resources at the row target to a desired state of resources at the execution target. The tasks may be sent to a worker 526 that may execute the tasks. In executing the tasks, the worker 526 may assist in creating a safety plan, which is a set of approved actions. The actions may include deploying resources to the execution target or any other suitable actions for execution at the execution target. The worker 526 may send the safety plan via a WAN gateway 520 to a regional CIOS 522, which may send the safety plan to the central CIOS 506.

[0094] The central CIOS 506 may receive the safety plan from the regional CIOS 522 and compare the set of changes included in the safety plan with the actions at the execution target. The change management module 510 may perform this comparison and make one of two determinations: the action is a subset of the set of changes in the safety plan, or the action is not a subset of the set of changes in the safety plan. Only these two determinations are described in detail, but other determinations may be made (e.g., what percentage difference is there between the set of changes and the safety plan, etc.). In response to determining that the action is a subset of the set of changes included in the safety plan, the action may be executed, which may include deploying resources to the execution target. In response to determining that the action is not a subset of the set of changes included in the safety plan, a notification may be sent to the user 518. The notification may be sent to the user 518 via the LAN gateway 516 and may include information alerting the user 518 that drift may be occurring or that an action that may be scheduled to be executed at the execution target may not be included in the safety plan. In response to viewing the notification, the user 518 may approve or reject the action that is not included in the safety plan. If the user 518 approves the operation, the operation may be executed and the resource may be deployed to the execution target. If the user 518 rejects the operation, the operation may not be executed, the user 518 may decide to create a new configuration file, or may decide to abandon the operation. Thus, a safety plan may be understood to be a data structure that controls the ability of the processor(s) executing the deployment process by either allowing the operation to be executed if it is a subset of the set of changes included in the safety plan, or aborting the execution of the operation if it is not a subset of the changes included in the safety plan.

[0095] In another example, a user 518 can create a set of configuration files that can contain a desired state of resources for a set of execution targets in a set of regions. The configuration files can be sent to a central CIOS 506, which can compile the configuration files into a set of RA configuration files, including one RA configuration file per region. The RA configuration files can be sent to each region via a WAN gateway 520 and received by a regional CIOS 522 included in each region.

[0096] In each region, the regional CIOS 522 may receive a respective RA configuration file from the central CIOS 506, which may send the respective RA configuration file to a scheduler 524 that may create tasks and send the tasks to a task DB 528. The tasks may include compiling a set of operations to be performed on the execution targets in each region and may include comparing the current state of the resources at the execution targets with the desired state of the resources at the execution targets. The tasks may be sent to a worker 526 that can execute the tasks. In executing the tasks, the worker 526 may help create a safety plan, which is a set of approved operations. The operations may include deploying resources to the execution targets or any other suitable operations to execute at the execution targets. The worker 526 may send the safety plan to the regional CIOS 522, which may send the safety plan to the central CIOS 506 via the WAN gateway 520.

[0097] The central CIOS 506 can receive a set of safety plans from the regions and compile the safety plans into a master safety plan (e.g., one safety plan of a plurality of safety plans), also described as a compiled safety plan. Each safety plan included in the compiled safety plan can include similar changes. A difference in changes between the safety plans included in the compiled safety plan may indicate that drift has occurred in at least one region. The central CIOS 506 can compare the changes included in each safety plan included in the compiled safety plan with actions at the execution targets included in the respective regions. The change management module 510 can perform this comparison and make one of two determinations: a determination that the actions are a subset of the set of changes included in the compiled safety plan or a determination that the actions are not a subset of the set of changes included in the compiled safety plan. In response to a determination that the actions are a subset of the set of changes included in the compiled safety plan, an action can be performed, which may include deploying resources to the execution targets included in the respective regions. In response to a determination that the actions are not a subset of the set of changes included in the compiled safety plan, a notification can be sent to a user 518. The notification may be sent to the user 518 via the LAN gateway 516 and may include information alerting the user 518 that drift may have occurred in at least one region or that an operation that may be scheduled to be executed in at least one execution target may not be included in the compiled safety plan. In response to viewing the notification, the user 518 may approve or reject the operation that is not included in the safety plan. If the user 518 approves the operation, the operation may be executed and resources may be deployed to the execution targets included in the respective region. If the user 518 rejects the operation, the operation may not be executed and the user 518 may decide to create a new configuration file or may decide to abandon the operation.

[0098] 6 and 7 show example flowcharts illustrating processes 600 and 700 for performing techniques of CIOS according to certain embodiments of the present disclosure. These processes are illustrated as logical flow diagrams, each operation of which may be implemented in hardware, computer instructions, or a combination thereof. In the context of computer instructions, the operations may represent computer-executable instructions stored on one or more computer-readable storage media that, when executed by one or more processors, perform the recited operations. Generally, computer-executable instructions include routines, programs, objects, components, data structures, and the like that perform particular functions or implement particular data types. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described operations may be combined in any order and / or in parallel to implement a process.

[0099] Additionally, processes 600 and 700 may be executed under the control of one or more computing devices or computer systems configured with executable instructions and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) that collectively execute on one or more processors, by hardware, or by a combination thereof. As mentioned above, the code may be stored in a computer-readable storage medium, for example, in the form of a computer program that includes a plurality of instructions executable by one or more processors. In some embodiments, processes 600 and 700 may be executed in parallel by multiple processors. The computer-readable storage medium may be non-transitory.

[0100] FIG. 6 illustrates a process 600 for using a safety plan in a cloud infrastructure orchestration service, according to at least one embodiment. 6 is a flowchart of a process 600 for deploying a security plan to an execution target in a central CIOS (e.g., central CIOS 506 of FIG. 5 ). Process 600 may begin at block 602, where a central CIOS (e.g., central CIOS 506 of FIG. 5 ) receives a safety plan from a regional CIOS (e.g., regional CIOS 522 of FIG. 5 ) for a region that includes an execution target. The safety plan may include a list of resources and operations based on a deployment configuration file. The operations may include instructions to deploy resources to the execution target. In some examples, the central CIOS may receive a set of safety plans from a set of regional CIOS corresponding to the set of regions. The safety plans may be compiled by the central CIOS into a compiled safety plan that may include resources and operations based on a deployment configuration file that corresponds to the execution targets, each of the execution targets included in a different region.

[0101] At block 604, the central CIOS prepares to execute at least one operation corresponding to the at least one resource included in the deployment configuration file. In response to receiving the safety plan from the regional CIOS, the central CIOS may prepare to execute at least one operation on the at least one execution target, for example, by compiling a local plan that may include the at least one operation. The operation may include deploying the at least one resource to the execution target. In some examples, the central CIOS may prepare to execute a set of operations that correspond to a set of resources included in the set of deployment configuration files. In this example, the central CIOS may receive a set of safety plans corresponding to a set of regions from a set of regional CIOS.

[0102] In block 606, the central CIOS compares the action to the safety plan. A change management module (e.g., change management module 510 of FIG. 5), which may be included in the central CIOS, may perform the comparison. The comparison may include determining whether the action is a subset of the safety plan. The action may be considered a subset of the safety plan if the safety plan includes the action. In some examples, the action may not change the current state of the execution target. In this case, the central CIOS may not execute the action, but the action may be considered a subset of the safety plan.

[0103] At block 608, the action is executed. In response to the change management module determining that the action is a subset of the safety plan, the central CIOS may send a command to execute the action. The command may be sent by the central CIOS to the regional CIOS in each region and may include deploying resources included in the deployment configuration file to the execution targets. In an example where resources are desired to be deployed to multiple execution targets, multiple safety plans may be received by the central CIOS. In this case, the central CIOS may prepare to execute the action, and the change management module may compare the action to the safety plan to determine whether the action is a subset of the safety plan. In response to the change management module determining that the action is a subset of the safety plan, the central CIOS may send a command to each region to execute the action. The command may be sent from the central CIOS to the regional CIOS in each region and may include deploying resources included in the deployment configuration file to each execution target.

[0104] In block 610, the action is not performed and the deployment is stopped. In response to the change management module determining that the action is not a subset of the safety plan, the central CIOS may not send a command to perform the action, and instead the central CIOS may stop the deployment. If a resource is desired to be deployed to multiple execution targets, multiple safety plans may be received by the central CIOS. In this case, the central CIOS may prepare to perform the action and the change management module may , the action may be compared to the safety plan to determine whether the action is a subset of the safety plan. In response to the change management module determining that the action is not a subset of the safety plan, the central CIOS may not send a command to execute the action. Instead, the central CIOS may stop the deployment. In other examples, the central CIOS may stop the action that is not a subset of the safety plan and may send a command to execute the action that is a subset of the safety plan.

[0105] In block 612, the central CIOS sends a notification to a user (e.g., user 518 of FIG. 5) that the deployment does not comply with the safety plan. In response to determining that the operations are not a subset of the safety plan (as occurs in block 610), the central CIOS may create and send a notification to the user that the deployment no longer complies with the safety plan. The notification may present the user with an option to choose to continue with the deployment despite the operations not being a subset of the safety plan. The user may choose to continue with the deployment or may choose to abandon the deployment. If the user chooses to abandon the deployment, the user may create another configuration file and attempt a different deployment. If it is desired to deploy resources to multiple execution targets, the notification sent to the user may include the deployment not complying with the safety plan. The central CIOS may send commands to execute the operations determined to be a subset of the operations.

[0106] 7 is a flow chart illustrating a process 700 for generating a safety plan in a cloud infrastructure orchestration service according to at least one embodiment. At block 702, a regional CIOS (e.g., regional CIOS 522 of FIG. 5) included in a region receives a configuration file for deploying infrastructure resources from a central CIOS (e.g., central CIOS 506 of FIG. 5). The configuration file may include operations that include instructions for deploying resources to execution targets included in the region.

[0107] In block 704, the regional CIOS identifies a new current state of the cloud infrastructure in the region. The configuration file may define a desired state of the cloud infrastructure in the region, and in response to receiving the configuration file, the regional CIOS may identify a current state of resources in the region. In some examples, the current state may be similar or identical to the desired state. In other examples, the current state may not include any resources, and the desired state may include most of the resources desired to be deployed to the execution target.

[0108] In block 706, the regional CIOS generates a safety plan for deploying infrastructure resources based on a comparison of the current state and the desired state. The safety plan may include actions executable on the execution target. The actions may include deploying infrastructure resources to the execution target. In some instances where the current state and the desired state are similar or identical, the safety plan compiled by the regional CIOS may be empty or may not include any actions.

[0109] At block 708, the regional CIOS sends the safety plan for deploying the infrastructure resources to the central CIOS. In response to creating the safety plan, the regional CIOS may send the safety plan to the central CIOS for performing a comparison. The central CIOS may determine that the operations to be performed on the execution target are a subset of the safety plan, similar to block 608 of process 600. In response to this determination, the regional CIOS may execute the operations to deploy the resources included in the configuration file to the execution target.

[0110] FIG. 8 illustrates an example swim lane diagram illustrating a process 800 for performing techniques of CIOS, according to certain embodiments of the present disclosure. The process is illustrated as a logical flow diagram, each operation of which may be implemented in hardware, computer instructions, or a combination thereof. In the context of computer instructions, the operations may represent computer-executable instructions stored on one or more computer-readable storage media that, when executed by one or more processors, perform the recited operations. Generally, computer-executable instructions include routines, programs, objects, components, data structures, and the like that perform particular functions or implement particular data types. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described operations may be combined in any order and / or in parallel to perform the process.

[0111] Further, process 800 may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) executable under the control of one or more computing devices or computer systems and collectively executed on one or more processors, by hardware, or by a combination thereof. As mentioned above, the code may be stored in a computer-readable storage medium, for example, in the form of a computer program including a plurality of instructions executable by one or more processors. In some embodiments, process 800 may be executable in parallel by multiple processors. The computer-readable storage medium may be non-transitory.

[0112] 8 is a swim lane diagram illustrating a process 800 for illustrating how a safety plan is generated and used, according to at least one embodiment. The diagram includes three lanes as shown: a user lane 802, a center lane 804, and a regional lane 806. The user lane 802 may include operations performed by a user (e.g., user 518 of FIG. 5) or commands given to a computing system by a user. The center lane 804 may include operations performed by a central CIOS (e.g., central CIOS 506 of FIG. 5). The regional lane 806 may include operations performed by a regional CIOS (e.g., regional CIOS 522 of FIG. 5).

[0113] Process 800 may begin at block 808 with a user creating a configuration file. A user may wish to deploy infrastructure resources to an execution target contained in a region. In response to this desire, the user may create a configuration file that may define a desired state of the execution target. At block 810, the user sends the configuration to a central CIOS. In response to creating the configuration file, the user may send the configuration file to the central CIOS to initiate the deployment.

[0114] At block 812, the central CIOS receives a configuration file from the user. In response to receiving the configuration file from the user, the central CIOS may initiate the deployment. At block 814, the central CIOS compiles a region-independent (RA) configuration file. In response to initiating the deployment, the central CIOS may compile an RA configuration, which may be a configuration file that may be independent of the region to which the RA configuration file is sent. At block 816, the central CIOS sends the RA configuration file to the regional CIOS. In response to creating the RA configuration file, the central CIOS may send the RA configuration file to the regional CIOS of the region in which the resource is desired to be deployed.

[0115] At block 818, the regional CIOS in the region in which the resource is desired to be deployed receives the RA configuration file. The RA configuration file is used to configure the infrastructure on the execution target. The regional CIOS may include operations performed to deploy the infrastructure resources. The regional CIOS may send the RA configuration file to a worker node (e.g., worker 526 of FIG. 5) that may perform the comparison. In block 820, the regional CIOS creates a safety plan based on the new current state. The worker may compare the new current state of the infrastructure resources in the region to the desired state of the infrastructure resources defined in the RA configuration file. The regional CIOS may create a safety plan based at least in part on the comparison performed by the worker. The safety plan may include approved changes to the resources at the execution target.

[0116] At block 822, the regional CIOS sends the safety plan to the central CIOS. In response to creating the safety plan, the regional CIOS may send the safety plan to the central CIOS to determine whether the operations may be deployed to the execution target. At block 824, the central CIOS receives the safety plan from the regional CIOS. In some instances where it is desired that resources be deployed to multiple regions, the central CIOS may receive multiple safety plans. In this case, the central CIOS may compile one safety plan of the multiple safety plans, hereinafter referred to as a compiled safety plan. At block 826, the central CIOS compares the deployment operations to the safety plan. The deployment operations may include instructions to deploy infrastructure resources to the execution target. In the case of a compiled safety plan, the central CIOS may compare the operations to the compiled safety plan. At block 828, the central CIOS determines whether the operations are a subset of the safety plan. In the case of a compiled safety plan, the central CIOS may determine whether the operations are a subset of the compiled safety plan.

[0117] In block 830, in response to the central CIOS determining that the operation is a subset of the safety plan, the central CIOS sends a command to the regional CIOS to execute the operation. In this case, infrastructure resources may be deployed to the execution target, and in the case of a compiled safety plan, resources may be deployed to multiple execution targets. In other examples, the operation may not change the state of the execution target, in which case the central CIOS may take no action. In block 832, in response to the central CIOS determining that the operation is not a subset of the safety plan, the central CIOS stops the deployment. In this case, the central CIOS sends a notification to the user that may inform the user that drift has occurred and that the operation does not comply with the safety plan. This notification may allow the user to allow the operation to be executed. If the user chooses not to allow the operation to be executed, the user may abandon the deployment or create a new configuration file and send the new configuration file to the central CIOS to initiate a second deployment. In the case of a compiled safety plan, the central CIOS may send a notification to the user if the operation is not a subset of the compiled safety plan.

[0118] FIG. 9 is a block diagram illustrating a disconnected area 900, according to at least one embodiment. The disconnected area 900 may be communicatively coupled to a connected central CIOS 902 (e.g., central CIOS 506 of FIG. 5). However, the connection between the disconnected area 900 and the connected central CIOS 902 may not be real-time and may be delayed. The disconnected area 900 may include a central CIOS 904 (e.g., central CIOS 506 of FIG. 5), a scheduler node 906 (e.g., scheduler 524 of FIG. 5), a worker node 908 (e.g., worker 526 of FIG. 5), a docker 910 (e.g., docker 530 of FIG. 5), which may include a CIOS container 912 (e.g., CIOS container 532 of FIG. 5), and a signing proxy 914 (e.g., signing proxy 534 of FIG. 5). In some examples, the disconnected area 900 may be capable of receiving information / instructions from the connected central CIOS 902. It is possible, but the connected central CIOS 902 may be disconnected with respect to communications from the disconnected area 900. In other words, the communication may be one-way and when the connected central CIOS 902 sends information, it cannot be sure what action the disconnected area 900 will take.

[0119] The central CIOS 904 can receive tasks from the connected central CIOS 902 and can send the tasks to the scheduler 906. The tasks can include performing CRUD operations or any other suitable tasks for deploying infrastructure resources in the region of the execution target(s). The scheduler 906 can send the tasks to workers 908, which can be included in a worker fleet. The worker fleet can include many workers 908, and the scheduler 906 can select the worker 908 with the least amount of work or the most available amount of computing resources to assign the task. The scheduler 906 can assign one task at a time to the worker 908. The worker 908 can execute the task and, in executing the task, can make a call to a CIOS container 912 included in a docker 910. The CIOS container 912 can execute the task including Terraform instructions. The instructions may direct the API call to cloud services 916, which may include services available to the disconnected region 900 that may not be available over a public network (e.g., the Internet). To make an API call to cloud services 916, the CIOS container 912 may send a request to make the API call to the signing proxy 914, which may determine whether the request is valid. In response to determining that the request is valid, the signing proxy 914 may make the API call to the cloud services 916.

[0120] The connected central CIOS 902 can send a first safety plan and a configuration file to the central CIOS 904 for deployment of infrastructure resources at the execution targets in the disconnected region 900. The central CIOS 904 can send the configuration file to the scheduler 906, which can assign a task to a worker 908 to create a second safety plan based on the configuration file. The first safety plan may be created from a different configuration file that is similar or identical to the configuration file. In response to creating the second safety plan, the worker 908 can send the second safety plan to the central CIOS 904, which can compare the first safety plan with the second safety plan. In response to the comparison, the central CIOS 904 can determine that the second safety plan is a subset of the first safety plan. In this case, the second safety plan can be automatically approved and infrastructure resources based on the configuration file can be deployed to the execution targets in the disconnected region 900. The central CIOS 904 can determine that the second safety plan is not a subset of the first safety plan. In this case, the deployment can be stopped and a notification can be sent to the user (e.g., user 518 of FIG. 5 ) informing the user that drift may be occurring and that the deployment is not compliant with the second safety plan. Further, in this example, the connected central CIOS 902 can send instructions to the disconnected region 900 to stop any originally planned deployment if the second safety plan is not a subset of the first safety plan.

[0121] FIG. 10 illustrates an example flow diagram illustrating a process 1000 for performing techniques of CIOS, according to certain embodiments of the present disclosure. This process is illustrated as a logical flow diagram, the operations of which may be implemented in hardware, computer instructions, or a combination thereof. In the context of computer instructions, the operations may be implemented in computer programs stored on one or more computer-readable storage media that, when executed by one or more processors, perform the recited operations. In the embodiments described herein, the term "computer-executable instructions" may represent computer-executable instructions. Generally, computer-executable instructions include routines, programs, objects, components, data structures, and the like that perform particular functions or implement particular data types. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described operations can be combined in any order and / or in parallel to implement a process.

[0122] Additionally, process 1000 may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) executable under the control of one or more computing devices or computer systems that collectively execute on one or more processors, by hardware, or by a combination thereof. As mentioned above, the code may be stored on a computer-readable medium in the form of a computer program that includes a plurality of instructions executable by one or more processors. In some embodiments, process 1000 is executable in parallel by multiple processors. The computer-readable storage medium may be non-transitory.

[0123] 10 is a flow chart illustrating a process 1000 for using a safety plan in a disconnected area, according to at least one embodiment. At block 1002, a configuration file is received for deploying infrastructure resources to a first execution target and a second execution target. The configuration file may include infrastructure resources that a user (e.g., user 518 of FIG. 5) desires to deploy to the first execution target and the second execution target. In some examples, the configuration file may include resources for deployment on the first execution target and the second configuration file may include resources for deployment on the second execution target.

[0124] At block 1004, a first safety plan is generated based on the configuration file. The safety plan may include a list of approved changes that the user wants to execute at the first execution target. The first safety plan may be generated by a central CIOS (e.g., central CIOS 506 of FIG. 5). The first execution target may be included in a connected region. The first safety plan may be created based on a comparison of a current state of resources at the first execution target and a desired state of resources at the first execution target, which may be defined in the configuration file.

[0125] Approval of the first safety plan is received at block 1006. The user may be given the option to approve the first safety plan, or the first safety plan may be automatically approved based on the operations performed on the first execution target. In response to the approval at block 1006, infrastructure resources may be deployed to the execution target based on the configuration file.

[0126] At block 1008, a second safety plan is generated for the second execution target based on the configuration file. The second safety plan may be generated by a central CIOS that may be included in the disconnected region (e.g., disconnected region 900 of FIG. 9). The second execution target may also be included in the disconnected region. In some examples, the second safety plan may be created for the second execution target based on the second configuration file. In some examples, the first execution target and the second execution target may be similar or identical, and the first safety plan and the second safety plan may be similar or identical.

[0127] In block 1010, the central CIOS determines whether the second safety plan is a subset of the first safety plan. The central CIOS may include in the disconnection region, The approved changes to the second safety plan may be compared to the approved changes to the first safety plan, and the central CIOS may determine that the second safety plan is a subset of the first safety plan or that the second safety plan is not a subset of the first safety plan.

[0128] In accordance with the central CIOS determining in block 1012 that the second safety plan is a subset of the first safety plan, the second safety plan is automatically approved based on the approval of the first safety plan. The central CIOS may automatically approve the second safety plan without input from a user in this case because the first safety plan has already been approved and the first safety plan may have been successfully executed. In some examples, the current state of the second execution target may be similar or the same as the desired state of the second execution target defined in the configuration file. In this example, the second safety plan may not include any changes to be executed in the second execution target. The second safety plan in this case may be automatically approved by the central CIOS because the empty second safety plan is a subset of the first safety plan.

[0129] In accordance with the central CIOS determining that the second safety plan is not a subset of the first safety plan at block 1014, the central CIOS stops the deployment at the second execution target and notifies the user that the second safety plan is non-compliant. The central CIOS may send a notification to the user informing the user that the deployment of infrastructure resources at the second execution target may be stopped, that drift may have occurred, and that the second safety plan is not a subset of the first safety plan. The notification may be presented to the user in a user interface that may present at least one difference between the first safety plan and the second safety plan. The notification may enable the user to (i) choose to deploy the resources despite the central CIOS determination that the second safety plan is non-compliant, (ii) abandon the deployment at the second execution target, (iii) submit a new configuration file to initiate a new deployment at the second execution target, or a combination thereof.

[0130] In block 1016, in response to approval of the second safety plan by the central CIOS, the central CIOS sends the second safety plan to the second execution target for deploying infrastructure resources. The central CIOS and the second execution target may be included in a disconnected region. The second safety plan may include instructions to deploy resources to the second execution target. In some examples, the second safety plan may not include instructions to deploy resources because the current state of the second execution target may be similar or identical to the desired state of the execution target defined by the configuration file. In this case, the central CIOS may not take any action.

[0131] Exemplary System 11-13 are diagrams illustrating aspects of an example environment for implementing aspects of the present disclosure, according to various embodiments. FIG. 11 is a simplified diagram illustrating a distributed system 1100 for implementing one embodiment of the present disclosure. In the illustrated embodiment, the distributed system 1100 includes one or more client computing devices 1102, 1104, 1106, and 1108 that are configured to execute and operate client applications, such as a web browser, a proprietary client (e.g., Oracle Forms), etc., via one or more networks 1110. A server 1112 can be communicatively coupled to the remote client computing devices 1102, 1104, 1106, and 1108 via the network 1110.

[0132] In various embodiments, the server 1112 provides identity management services. The server 1112 may be adapted to run one or more services or software applications, such as services and applications that provide virtualized environments. In certain embodiments, the server 1112 may also provide other services or software applications, which may include non-virtualized and virtualized environments. In some embodiments, these services may be provided to users of the client computing devices 1102, 1104, 1106, and / or 1108 as web-based or cloud services, or under a software-as-a-service (SaaS) model. Users operating the client computing devices 11102, 1104, 1106, and / or 1108 may then utilize one or more client applications to interact with the server 1112 to utilize the services provided by these components.

[0133] In the configuration shown in FIG. 11, the software components 1118, 1120, and 1122 of the system 1100 are shown as executing on the server 1112. In other embodiments, one or more of the components of the system 1100 and / or the services provided by these components can also be executed by one or more of the client computing devices 1102, 1104, 1106, and / or 1108. Users operating these client computing devices can then utilize one or more client applications to use the services provided by these components. These components may be implemented in hardware, firmware, software, or a combination thereof. It should be understood that a variety of different system configurations are possible and may differ from the distributed system 1100. The embodiment shown in FIG. 11 is therefore an example of a distributed system for implementing the system of the embodiments and is not intended to be limiting.

[0134] The client computing devices 1102, 1104, 1106, and / or 1108 may include various types of computing systems. For example, the client computing devices may include portable handheld devices (e.g., iPhones, mobile phones, iPads, computing tablets, personal digital assistants (PDAs)) or wearable devices (e.g., Google Glasses head-mounted displays) running software (e.g., Microsoft Windows Mobile) and / or various mobile operating systems (e.g., iOS, Windows Phone, Android, Blackberry 10, Palm OS, etc.). These client computing devices may support a variety of applications (e.g., various Internet-related apps, e-mail, short message service (SMS) applications, etc.) and may use a variety of other communication protocols. The client computing devices may also include general-purpose personal computers, including, by way of example, personal and / or laptop computers running various versions of Microsoft Windows, Apple Macintosh, and / or Linux operating systems. The client computing devices may be workstation computers running any of a variety of commercially available UNIX or UNIX-like operating systems, including, but not limited to, various GNU / Linux operating systems (e.g., Google Chrome OS, etc.). Client computing devices may also be thin-client computers, Internet-enabled gaming systems (e.g., computers having a Kinect gesture input device), etc., that can communicate over network(s) 1110. It may also include electronic devices such as a gaming console (with or without a Microsoft Xbox game console) and / or a personal messaging device.

[0135] 11 is shown with four client computing devices, any number of client computing devices may be supported. Other devices, such as devices with sensors, may interact with the server 1112.

[0136] The network(s) 1110 in the distributed system 1100 may be any type of network familiar to those skilled in the art capable of supporting data communications using any of a variety of available protocols, including, but not limited to, TCP / IP (Transmission Control Protocol / Internet Protocol), SNA (Systems Network Architecture), IPX (Internet Packet Exchange), AppleTalk, etc. By way of example only, the network(s) 1110 may be a local area network (LAN), an Ethernet-based network, a token ring, a wide area network, the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., a network operating under any of the Institute of Electrical and Electronics Engineers (IEEE) 1002.11 suite of protocols, Bluetooth and / or other wireless protocols), and / or any combination of these and / or other networks.

[0137] The servers 1112 may be configured as one or more general-purpose computers, dedicated server computers (including, by way of example, PC (personal computer) servers, UNIX servers, mid-range servers, mainframe computers, rack-mounted servers, etc.), server farms, server clusters, or other suitable arrangements and / or combinations. The servers 1112 may include one or more virtual machines running a virtual operating system, or other computing architectures including virtualization. One or more flexible pools of logical storage devices may be virtualized to maintain virtual storage devices for the servers. The virtual network may be controlled by the servers 1112 using software-defined networking. In various embodiments, the servers 1112 may be adapted to execute one or more services or software applications described in the preceding disclosure. For example, the servers 1112 may correspond to a server for executing the above-mentioned processes according to one embodiment of the present disclosure.

[0138] Server 1112 may run an operating system, including any of those listed above, as well as any commercially available server operating system. Server 1112 may also run any of a variety of other server and / or mid-tier applications, including HTTP (Hypertext Transfer Protocol) servers, FTP (File Transfer Protocol) servers, CGI (Common Gateway Interface) servers, JAVA servers, database servers, and the like. Examples of database servers include those commercially available from Oracle, Microsoft, Sybase, IBM (International Business Machines), and the like. This includes, but is not limited to, those that are

[0139] In some implementations, the server 1112 may run one or more applications for parsing and consolidating data feeds and / or event updates received from users of the client computing devices 1102, 1104, 1106, and 1108. By way of example, the data feeds and / or event updates may include, but are not limited to, Twitter feeds, Facebook updates, or real-time updates received from one or more third party information sources and continuous data streams, which may include real-time events associated with sensor data applications, stock ticker watches, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, automobile traffic monitoring, and the like. The server 1112 may also include one or more applications for displaying the data feeds and / or real-time events via one or more display devices of the client computing devices 1102, 1104, 1106, and 1108.

[0140] The distributed system 1100 may also include one or more databases 1114 and 1116. These databases may provide a mechanism for storing information such as identity information and other information used by embodiments of the present disclosure. The databases 1114 and 1116 may reside in a variety of locations. As an example, one or more of the databases 1114 and 1116 may reside on a non-transitory storage medium local to (and / or residing within) the server 1112. Alternatively, the databases 1114 and 1116 may be remote from the server 1112 and communicate with the server 1112 via a network-based or dedicated connection. In one set of embodiments, the databases 1114 and 1116 may reside within a storage area network (SAN). Similarly, any files necessary to perform functions ascribed to the server 1112 may be stored locally and / or remotely on the server 1112, as appropriate. In one set of embodiments, databases 1114 and 1116 may include relational databases, such as those provided by Oracle Corporation, adapted to store, update, and retrieve data in response to commands formatted according to SQL.

[0141] 12 is a diagram illustrating an example of a computer system 1200 that may be used to implement an embodiment of the present disclosure. In some embodiments, the computer system 1200 may be used to implement any of the various servers and computer systems described above. As shown in FIG. 12, the computer system 1200 includes various subsystems, including a processing subsystem 1204 that communicates with multiple peripheral subsystems via a bus subsystem 1202. These peripheral subsystems may include a processing acceleration unit 1206, an input / output subsystem 1208, a storage subsystem 1218, and a communication subsystem 1224. The storage subsystem 1218 may include a tangible computer-readable storage medium 1622 and a system memory 1210.

[0142] Bus subsystem 1202 provides a mechanism for allowing the various components and subsystems of computer system 1200 to communicate with each other as intended. Although bus subsystem 1202 is shown diagrammatically as a single bus, alternative embodiments of the bus subsystem may utilize multiple buses. Bus subsystem 1202 may be any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. For example, such architectures may include an Industry Standard Architecture (ISA) bus, a MicroChannel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, a Peripheral Component Interconnect (PCI) bus that may be implemented as a mezzanine bus manufactured in accordance with the IEEE P1386.1 standard, and the like.

[0143] The processing subsystem 1204 controls the operation of the computer system 1200 and may include one or more processing units 1232, 1234, etc. The processing units may include one or more processors, including single-core or multi-core processors, one or more cores of a processor, or a combination thereof. In some embodiments, the processing subsystem 1204 may include one or more special purpose co-processors, such as a graphics processor or a digital signal processor (DSP). In some embodiments, some or all of the processing units of the processing subsystem 1204 may be implemented using customized circuitry, such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[0144] In some embodiments, the processing units in the processing subsystem 1204 can execute instructions stored in the system memory 1210 or on the computer readable storage medium 1222. In various embodiments, the processing units can execute various program or code instructions and can maintain multiple simultaneously executing programs or processes. At any one time, some or all of the program code being executed can reside in the system memory 1210 and / or on the computer readable storage medium 1222, including, in some cases, on one or more storage devices. With suitable programming, the processing subsystem 1204 can provide the various functions described above for dynamically modifying documents (e.g., web pages) in response to usage patterns.

[0145] In particular embodiments, a processing acceleration unit 1206 may be provided to accelerate the overall processing performed by the computer system 1200, to perform customized processing, or to offload portions of the processing performed by the processing subsystem 1204.

[0146] Input / output subsystem 1208 may include devices and mechanisms for inputting information into computer system 1200 and / or outputting information from or through computer system 1200. In general, use of the term "input device" is intended to include all possible types of devices and mechanisms for inputting information into computer system 1200. User interface input devices may include, for example, keyboards, pointing devices (such as a mouse or trackball), touch pads or touch screens integrated into a display, scroll wheels, click wheels, dials, buttons, switches, keypads, audio input devices with voice command recognition systems, microphones, and other types of input devices. User interface input devices may also include motion-sensing and / or gesture recognition devices (such as a Microsoft Kinect® motion sensor) that allow a user to control and interact with the input device, a Microsoft Xbox® 360 game controller, and devices that provide an interface for receiving input using gestures and verbal commands. The user interface input devices may also include an eye gesture recognition device (such as a Google Glass® blink detector) that detects eye movements from the user (e.g., "blinking" while taking a picture and / or making a menu selection) and translates the eye gestures as input to the input device (e.g., Google Glass®). Additionally, the user interface input devices may include a voice recognition sensing device that allows the user to interact with a voice recognition system (e.g., the Siri® navigator) via voice commands.

[0147] Other examples of user interface input devices include three-dimensional (3D) mice, joysticks, and Examples of user interface input devices include, but are not limited to, digital or pointing sticks, game pads and graphic tablets, and audio / visual devices (such as speakers, digital cameras, digital camcorders, portable media players, webcams, image scanners, fingerprint scanners, barcode readers 3D scanners, 3D printers, laser range finders, and eye tracking devices). Additionally, user interface input devices may include medical imaging input devices, such as, for example, computed tomography, magnetic resonance imaging, position emission tomography, medical ultrasound examination devices, etc. User interface input devices may also include audio input devices, such as, for example, MIDI keyboards and digital musical instruments.

[0148] The user interface output devices may include a display subsystem, indicator lights, or non-visual displays (such as audio output devices). The display subsystem may be a cathode ray tube (CRT), a flat panel device (such as one using a liquid crystal display (LCD) or plasma display), a projection device, a touch screen, and the like. In general, use of the term "output device" is intended to include all possible types of devices and mechanisms for outputting information from computer system 1200 to a user or to another computer. For example, user interface output devices may include, but are not limited to, various display devices that visually convey text, graphics, and audio / visual information (such as monitors, printers, speakers, headphones, automobile navigation systems, plotters, audio output devices, and modems).

[0149] The storage subsystem 1218 provides a repository or data store for storing information used by the computer system 1200. The storage subsystem 1218 provides a tangible, non-transitory computer-readable storage medium for storing the basic programming and data constructs that provide the functionality of some embodiments. Software (programs, code modules, instructions) that, when executed by the processing subsystem 1204, provide the above-described functionality may be stored in the storage subsystem 1218. This software is executable by one or more processing units of the processing subsystem 1204. The storage subsystem 1218 may also provide a repository for storing data used in accordance with the present disclosure.

[0150] The storage subsystem 1218 may include one or more non-transient memory devices, including volatile and non-volatile memory devices. As shown in FIG. 12, the storage subsystem 1218 includes a system memory 1210 and a computer-readable storage medium 1222. The system memory 1210 may include several memories, including a volatile main random access memory (RAM) for storing instructions and data during program execution, and a non-volatile read-only memory (ROM) or flash memory in which fixed instructions are stored. In some implementations, a basic input / output system (BIOS), including basic routines that help transfer information between elements within the computer system 1200, such as during start-up, may be stored in the ROM. The RAM may include data and / or program modules currently being operated and executed by the processing subsystem 1204. In some implementations, the system memory 1210 may include multiple different types of memories, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0151] By way of example, and not limitation, as shown in FIG. 12, system memory 1210 may store application programs 1212 (which may include client applications, web browsers, mid-tier applications, relational database management systems (RDBMS), etc.), program data 1214, and an operating system 1216. By way of example, operating system 1216 may include various versions of Microsoft Windows, Apple Macintosh, and / or Linux operating systems, various commercially available UNIX or UNIX-like operating systems (including, but not limited to, various GNU / Linux operating systems, Google Chrome OS, etc.), and / or mobile operating systems (such as iOS, Windows Phone, Android OS, BlackBerry 10 OS, and Palm OS operating systems).

[0152] The computer readable storage medium 1222 may store programming and data structures that provide the functionality of some embodiments. Software (programs, code modules, instructions) that, when executed by the processing subsystem 1204, cause the processor to provide the above-mentioned functionality may be stored in the storage subsystem 1218. By way of example, the computer readable storage medium 1222 may include non-volatile memory, such as a hard disk drive, a magnetic disk drive, an optical disk drive (CD ROM, DVD, Blu-ray disk, etc.), or other optical media. The computer readable storage medium 1222 may include, but is not limited to, a Zip drive, a flash memory card, a Universal Serial Bus (USB) flash drive, a Secure Digital (SD) card, a DVD disk, and a digital video tape, etc. The computer-readable storage medium 1222 may also include solid-state drives (SSDs) based on non-volatile memory (such as flash memory-based SSDs, enterprise flash drives, solid-state ROMs, etc.), SSDs based on volatile memory (such as solid-state RAM, dynamic RAM, static RAM, etc.), DRAM-based SSDs, magnetoresistive RAM (MRAM) SSDs, and hybrid SSDs using a combination of DRAM-based SSDs and flash memory-based SSDs. The computer-readable medium 1222 may provide storage of computer-readable instructions, data structures, program modules, and other data for the computer system 1200.

[0153] In particular embodiments, storage subsystem 1200 may also include a computer readable storage medium reader 1220 further connectable to a computer readable storage medium 1222. Along with, and optionally in combination with, system memory 1210, computer readable storage medium 1222 may collectively represent remote, local, fixed, and / or removable storage devices and media for storing computer readable information.

[0154] In certain embodiments, computer system 1200 may provide support for running one or more virtual machines. Computer system 1200 may execute programs such as a hypervisor to facilitate configuration and management of virtual machines. Each virtual machine may be assigned memory, computational (e.g., processors, cores), input / output, and networking resources. Each virtual machine may run its own operating system, which may be the same or different from the operating systems executed by other virtual machines executed by computer system 1200. Thus, multiple operating systems may be executed simultaneously by computer system 1200. Each virtual machine generally operates independently from other virtual machines.

[0155] The communications subsystem 1224 provides an interface to other computer systems and networks. The communications subsystem 1224 serves as an interface for sending and receiving data between other systems and the computer system 1200. For example, the communications subsystem 1224 may be used by the computer system 1200 to communicate with a cluster. The communications subsystem 1224 may enable the establishment of a communications channel over the Internet to one or more client devices for sending and receiving information to and from the client device. Additionally, the communications subsystem 1224 may be used to convey notifications of successful login or notifications to re-enter a password from a privileged account manager to a requesting user.

[0156] The communications subsystem 1224 may support both wired and / or wireless communication protocols. For example, in certain embodiments, the communications subsystem 1224 may include a radio frequency (RF) transceiver component for accessing wireless voice and / or data networks (e.g., using cellular telephone technology, advanced data network technology (such as 3G, 4G, or EDGE (enhanced data rates for global evolution)), WiFi (IEEE 802.11 family of standards), or other mobile communications technologies, or any combination thereof), a global positioning system (GPS) receiver component, and / or other components. In some embodiments, the communications subsystem 1224 may provide a wired network connection (e.g., Ethernet) in addition to or instead of a wireless interface.

[0157] The communications subsystem 1224 can receive and transmit data in a variety of formats. For example, in some embodiments, the communications subsystem 1224 can receive incoming communications in the form of structured and / or unstructured data feeds 1226, event streams 1228, event updates 1230, and the like. For example, the communications subsystem 1224 can be configured to receive (or transmit) data feeds 1226 in real time from users of social media networks and / or other communications services, such as Twitter feeds, Facebook updates, web feeds (such as Rich Site Summary (RSS) feeds), and / or real-time updates from one or more third party sources.

[0158] In particular embodiments, the communications subsystem 1224 may be configured to receive data in the form of a continuous data stream, which may include an event stream 1228 of real-time events and / or event updates 1230, which may be continuous or infinite in nature without a clear end. Examples of applications that generate continuous data may include, for example, sensor data applications, stock ticker watches, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, and automobile traffic monitoring.

[0159] The communications subsystem 1224 can also be configured to output structured and / or unstructured data feeds 1226, event streams 1228, event updates 1230, etc. to one or more databases that may communicate with one or more streaming data source computers coupled to the computer system 1200.

[0160] The computer system 1200 may be one of a variety of types, including a handheld portable device (e.g., an iPhone® mobile phone, an iPad® computing tablet, a PDA), a wearable device (e.g., a Google Glass® head-mounted display), a personal computer, a workstation, a mainframe, a kiosk, a server rack, or other data processing system.

[0161] Because the nature of computers and networks is constantly changing, the computer The description of computer system 1200 is intended to be exemplary only. Many other configurations are possible having more or fewer components than the system shown in Figure 12. Based on the disclosure and teachings provided herein, one of ordinary skill in the art will appreciate other aspects and / or methods for implementing the various embodiments.

[0162] The systems illustrated in the various figures may be provided in a variety of configurations, in some embodiments, the systems may be configured as distributed systems in which one or more components of the systems are distributed across one or more networks in one or more cloud infrastructure systems.

[0163] A cloud infrastructure system is a collection of one or more server computing devices, network devices, and / or storage devices. These resources may be divided by a cloud service provider and allocated in some manner to its customers. For example, a cloud service provider (such as Oracle Corporation of Redwood Shores, California) may offer various types of cloud services, including, but not limited to, one or more services offered under the Software as a Service (SaaS) category, services offered under the Platform as a Service (PaaS) category, services offered under the Infrastructure as a Service (IaaS) category, or services in other categories of services, including hybrid services. Examples of SaaS services include Oracle Fusion Examples of PaaS services include, but are not limited to, the ability to build and deliver a suite of on-demand applications, such as applications. SaaS services allow customers to utilize applications that run on cloud infrastructure systems without having to purchase software for those applications. Examples of PaaS services include, but are not limited to, services for organizations (such as Oracle) to consolidate existing applications onto a shared common architecture, as well as the ability to build new applications that leverage shared services provided by platforms such as Oracle Java Cloud Services (JCS) and Oracle Database Cloud Services (DBCS). IaaS services may facilitate the management and control of basic computing resources, such as storage, network, and other basic computing resources, for customers who utilize the services provided by SaaS and PaaS platforms.

[0164] 13 is a simplified block diagram illustrating one or more components of a system environment 1300 in which services provided by one or more components of an embodiment system may be provided as cloud services, according to one embodiment of the disclosure. In the illustrated embodiment, the system environment 1300 includes one or more client computing devices 1304, 1306, and 1308 that may be used by users to interact with a cloud infrastructure system 1302 that provides cloud services. These client computing devices may be configured to run a web browser, a proprietary client application (e.g., a client application such as Oracle Forms, or some other application, etc.) that may be used by users of the client computing devices to interact with the cloud infrastructure system 1302 and use the services provided by the cloud infrastructure system 1302.

[0165] It should be understood that the cloud infrastructure system 1302 depicted in the figure may have other components than those depicted. Additionally, the illustrated embodiment is only one example of a cloud infrastructure system that may incorporate an embodiment of the present disclosure. In some other embodiments, cloud infrastructure system 1302 may have more or fewer components than illustrated, may combine two or more components, or may have a different configuration or arrangement of components.

[0166] The client computing devices 1304 , 1306 , and 1308 may be devices similar to those described above for the client computing devices 1102 , 1104 , 1106 , and 1108 .

[0167] Although the example system environment 1300 is shown with three client computing devices, any number of client computing devices may be supported. Other devices, such as devices with sensors, may interact with the cloud infrastructure system 1302.

[0168] Network(s) 1310 may facilitate communication and exchange of data between clients 1304, 1306, and 1308 and cloud infrastructure system 1302. Each network may be any type of network familiar to those skilled in the art capable of supporting data communications using any of a variety of commercially available protocols, including those described above for network(s) 1110.

[0169] Cloud infrastructure system 1302 may comprise one or more computers and / or servers, which may include those described above for server 1112.

[0170] In certain embodiments, the services provided by a cloud infrastructure system may include a number of services (such as online data storage and backup solutions, web-based email services, hosted office suites and document collaboration services, database processing, managed technical support services, etc.) that are made available to users of the cloud infrastructure system on demand. The services provided by the cloud infrastructure system can dynamically scale to meet the needs of its users. A specific instantiation of a service provided by a cloud infrastructure system is referred to herein as a "service instance." In general, any service that is made available to users from a cloud service provider's system over a communications network such as the Internet is referred to as a "cloud service." In a public cloud environment, the servers and systems that make up the cloud service provider's system are distinct from the customer's own on-premise servers and systems. For example, the cloud service provider's system can host applications, and users can order and use the applications on demand over a communications network such as the Internet.

[0171] In some examples, services in a computer network cloud infrastructure may include protected computer network access to storage, hosted databases, hosted web servers, software applications, or other services provided to users by cloud vendors or as otherwise known in the art. For example, a service may include password-protected access to remote storage on the cloud over the Internet. As another example, a service may include a web-services-based hosted relational database and script for personal use by a networked developer. As another example, a service may include access to an email software application hosted on a cloud vendor's website.

[0172] In certain embodiments, cloud infrastructure system 1302 may include a suite of application, middleware, and database service offerings that are delivered to customers in a self-service, subscription-based, elastically scalable, reliable, highly available, and secure manner. One example of such a cloud infrastructure system is the Oracle Public Cloud (Oracle Public Cloud) offered by the present assignee. Public Cloud).

[0173] In various embodiments, cloud infrastructure system 1302 may be adapted to automatically provision, manage, and track customer subscriptions to services provided by cloud infrastructure system 1302. Cloud infrastructure system 1302 may provide cloud services through different deployment models. For example, the services may be provided under a public cloud model, where cloud infrastructure system 1302 is owned by an organization that sells cloud services (e.g., owned by Oracle Corporation) and the services are made available to the general public or to companies in different industries. As another example, the services may be provided under a private cloud model, where cloud infrastructure system 1302 is operated solely for a single organization and may provide services for one or more entities within that organization. Cloud services may also be provided under a community cloud model, where cloud infrastructure system 1302 and the services provided by cloud infrastructure system 1302 are shared by several organizations in an associated community. Cloud services may also be provided under a hybrid cloud model, which is a combination of two or more different models.

[0174] In some embodiments, the services offered by cloud infrastructure system 1302 may include one or more services offered under a Software-as-a-Service (SaaS) category, a Platform-as-a-Service (PaaS) category, an Infrastructure-as-a-Service (IaaS) category, or other service categories including hybrid services. A customer may order one or more services offered by cloud infrastructure system 1302 via a subscription order. Cloud infrastructure system 1302 then performs processing to provide the services in the customer's subscription order.

[0175] In some embodiments, the services provided by the cloud infrastructure system 1302 may include, but are not limited to, application services, platform services, and infrastructure services. In some examples, application services may be provided by the cloud infrastructure system via a SaaS platform. The SaaS platform may be configured to provide cloud services that fall under the SaaS category. For example, the SaaS platform may provide the ability to build and deliver a suite of on-demand applications on an integrated development and deployment platform. The SaaS platform may manage and control the underlying software and infrastructure for delivering the SaaS services. By utilizing the services provided by the SaaS platform, customers may utilize applications that run on the cloud infrastructure system. Customers may use separate licenses and A customer may obtain application services without having to purchase software and support. A variety of different SaaS services may be offered, including, but not limited to, services that provide solutions for sales performance management, enterprise integration, and business flexibility for large organizations.

[0176] In some embodiments, platform services may be provided by the cloud infrastructure system through a PaaS platform. The PaaS platform may be configured to provide cloud services that fall under the PaaS category. Examples of platform services may include, but are not limited to, services that allow an organization (such as Oracle) to integrate existing applications on a shared common architecture, and the ability to build new applications that leverage shared services provided by the platform. The PaaS platform may manage and control the underlying software and infrastructure to provide the PaaS services. Customers may obtain the PaaS services provided by the cloud infrastructure system without having to purchase separate licenses and support. Examples of platform services include, but are not limited to, Oracle Java Cloud Services (JCS) and Oracle Database Cloud Services (DBCS).

[0177] By utilizing the services provided by the PaaS platform, customers can use programming languages ​​and tools supported by the cloud infrastructure system and can also control the deployed services. In some embodiments, the platform services provided by the cloud infrastructure system can include database cloud services, middleware cloud services (e.g., Oracle Fusion middleware services), and Java cloud services. In one embodiment, the database cloud services can support a shared service deployment model that allows organizations to pool database resources and provide database-as-a-service to customers in the form of a database cloud. In the cloud infrastructure system, the middleware cloud services can provide a platform for customers to develop and deploy various business applications, and the Java cloud services can provide a platform for customers to deploy Java applications in the cloud infrastructure system.

[0178] A variety of different infrastructure services may be provided in a cloud infrastructure system by an IaaS platform that facilitates management and control of basic computing resources (such as storage, network and other basic computing resources) for customers that utilize the services provided by the SaaS and PaaS platforms.

[0179] In particular embodiments, cloud infrastructure system 1302 may also include infrastructure resources 1330 for providing resources used to provide various services to customers of the cloud infrastructure system. In one embodiment, infrastructure resources 1330 may include a pre-integrated and optimized combination of hardware (such as servers, storage, and networking resources) for running the services offered by the PaaS and SaaS platforms.

[0180] In some embodiments, the resource in the cloud infrastructure system 1302 Resources may be shared by multiple users and dynamically reallocated per demand. Furthermore, resources may be allocated to users in different time zones. For example, the cloud infrastructure system 1302 may allow a first set of users in a first time zone to utilize resources of the cloud infrastructure system for a particular time, and then allow reallocation of the same resources to another set of users located in a different time zone to maximize utilization of the resources.

[0181] In certain embodiments, multiple internal shared services 1332 are provided that are shared by various components or modules of cloud infrastructure system 1302 and by services provided by cloud infrastructure system 1302. These internal shared services may include, but are not limited to, security and identity services, integration services, enterprise repository services, enterprise manager services, virus scanning and whitelist services, high availability, backup and recovery services, services to enable cloud support, email services, notification services, file transfer services, and the like.

[0182] In particular embodiments, cloud infrastructure system 1302 may provide comprehensive management of cloud services (e.g., SaaS, PaaS, and IaaS services) in the cloud infrastructure system. In one embodiment, cloud management functions may include functions such as provisioning, managing, and tracking customer subscriptions received by cloud infrastructure system 1302.

[0183] In one embodiment, as shown in the figure, cloud management functionality may be provided by one or more modules, such as an order management module 1320, an order orchestration module 1322, an order provisioning module 1324, an order management and monitoring module 1326, and an identity management module 1328. These modules may include or be provided using one or more computers and / or servers, which may be general purpose computers, dedicated server computers, server farms, server clusters, or any other suitable arrangement and / or combination.

[0184] In an example of operation 1334, a customer may interact with cloud infrastructure system 1302 using a client device, such as client device 1304, 1306, or 1308, by requesting one or more services offered by cloud infrastructure system 1302 and placing an order for a subscription to one or more services offered by cloud infrastructure system 1302. In particular embodiments, the customer may access cloud user interfaces (UIs), namely cloud UI 1312, cloud UI 1314, and / or cloud UI 1316, and place a subscription order via these UIs. Order information received by cloud infrastructure system 1302 in response to the customer placing an order may include information identifying the customer and the one or more services offered by cloud infrastructure system 1302 to which the customer wishes to subscribe.

[0185] After an order is placed by a customer, the order information is received via the cloud UI 1312, 1314 and / or 1316.

[0186] In operation 1336, the order is stored in the order database 1318. The leader database 1318 may be one of several databases operated by the cloud infrastructure system 1302 and in coordination with other system elements.

[0187] At operation 1338, the order information is forwarded to the order management module 1320. In some examples, the order management module 1320 can be configured to perform billing and accounting functions related to the order, such as validating the order and booking the order upon validation.

[0188] At operation 1340, information regarding the order is communicated to the order orchestration module 1322. The order orchestration module 1322 may utilize the order information to orchestrate the provisioning of services and resources for the order placed by the customer. In some examples, the order orchestration module 1322 may use the services of the order provisioning module 1324 to orchestrate the provisioning of resources to support the subscribed services.

[0189] In certain embodiments, the order orchestration module 1322 enables management of business processes associated with each order and applies business logic to determine whether the order should proceed to provisioning. In operation 1342, upon receiving an order for a new subscription, the order orchestration module 1322 sends a request to the order provisioning module 1324 to allocate resources and configure those resources needed to fulfill the subscription order. The order provisioning module 1324 enables the allocation of resources for services ordered by a customer. The order provisioning module 1324 provides a level of abstraction between the cloud services provided by the cloud infrastructure system 1302 and the physical implementation layer used to provision resources to provide the requested services. This allows the order orchestration module 1322 to be decoupled from implementation details such as whether the services and resources are actually provisioned on the fly or whether the services and resources are pre-provisioned and only allocated / allocated upon request.

[0190] At operation 1344, once the services and resources are provisioned, a notification of the provided services may be sent by the order provisioning module 1324 of the cloud infrastructure system 1302 to the customer on the client devices 1304, 1306 and / or 1308. At operation 1346, the customer's subscription orders may be managed and tracked by the order management and monitoring module 1326. In some examples, the order management and monitoring module 1326 can be configured to collect usage statistics for the services in the subscription orders, such as the amount of storage used, the amount of data transferred, the number of users, and the amount of system up and down time.

[0191] In particular embodiments, cloud infrastructure system 1302 may include an identity management module 1328. Identity management module 1328 may be configured to provide identity services, such as access management and authorization services in cloud infrastructure system 1302. In some embodiments, identity management module 1328 may control information about customers who wish to use services provided by cloud infrastructure system 1302. Such information may be used to authenticate the identities of such customers. The identity management module 1328 may also include management of descriptive information about each customer, as well as information about how and by whom that descriptive information can be accessed and modified.

[0192] Although specific embodiments of the present disclosure have been described, various modifications, variations, alternative configurations, and equivalents are also encompassed within the scope of the present disclosure. The embodiments of the present disclosure are not limited to operating in a particular specific data processing environment, but can freely operate in multiple data processing environments. Furthermore, while the embodiments of the present disclosure have been described using a particular sequence of transactions and steps, it should be apparent to those skilled in the art that the scope of the present disclosure is not limited to the sequence of transactions and steps described. Various features and aspects of the above-described embodiments may be used individually or together.

[0193] Furthermore, while embodiments of the present disclosure have been described using a particular combination of hardware and software, it should be appreciated that other combinations of hardware and software are within the scope of the present disclosure. Embodiments of the present disclosure may be implemented solely in hardware, solely in software, or using a combination thereof. The various processes described herein may be implemented on the same processor, or on different processors in any combination. Thus, while components or modules are described as being configured to perform certain operations, such configuration may be achieved, for example, by designing electronic circuitry to perform the operations, by programming a programmable electronic circuit (such as a microprocessor) to perform the operations, or by any combination thereof. Processes may communicate using a variety of techniques, including but not limited to conventional techniques for inter-process communication, and different process pairs may use different techniques, or the same process pair may use different techniques at different times.

[0194] The specification and drawings are therefore to be regarded in an illustrative rather than a restrictive sense. It will be apparent, however, that additions, subtractions, deletions and other modifications and alterations may be made without departing from the broader spirit and scope of the appended claims. Accordingly, although specific embodiments of the disclosure have been described, they are not intended to be limiting. Various modifications and equivalents are within the scope of the following claims. These modifications include any relevant combination of the disclosed features.

[0195] Several exemplary items of this disclosure are described below. Item 1: A method comprising: receiving, by at least one of the plurality of computer processors, a safety plan including a list of resources and respective actions based at least in part on a deployment configuration file; Following receipt of instructions for approval of the safety plan, Preparing at least one of a plurality of computer processors to perform an operation corresponding to at least one of a list of resources according to a deployment configuration file; comparing the motion to a safety plan; executing, by at least one of the plurality of computer processors, an action in accordance with a determination that the action is part of the safety plan; Following a determination that the action is not part of the safety plan, At least one of the plurality of computer processors stopping the deployment; and at least one of the plurality of computer processors sending a notification that the deployment does not comply with the safety plan.

[0196] Item 2. The method of item 1, wherein the multiple processors are associated with an execution target.

[0197] Item 3. The method of item 2, wherein the execution target includes a geographic region. Item 4. The method of item 1, wherein the deployment configuration file is received from a central cloud infrastructure orchestration service.

[0198] Item 5. The method of item 4, wherein the deployment configuration file is generated by a user of a central cloud infrastructure orchestration service.

[0199] Item 6. The method of item 1, wherein each operation includes at least one of a create operation, an update operation, or a delete operation performed on or associated with at least one resource of the list of resources.

[0200] Item 7. The method of item 1, further comprising at least one of the plurality of computer processors halting the deployment pursuant to receiving disapproval of the safety plan or non-receiving approval of the safety plan.

[0201] Item 8. The method of item 1, further comprising at least one of the plurality of computer processors generating a local plan having local operations according to the deployment configuration file, and comparing the operations to the safety plan includes comparing one of the local operations to the safety plan.

[0202] Item 9. A computer-readable storage medium having instructions written thereon that, when executed by a computer processor, cause the computer processor to perform an operation, the operation being: receiving a safety plan including a list of resources and respective actions based at least in part on a deployment configuration file; Following receipt of instructions for approval of the safety plan, preparing to execute an action corresponding to at least one of a list of resources according to a deployment configuration file; Comparing the movement to the safety plan; performing the action pursuant to a determination that the action is part of the safety plan; and Stopping the deployment pursuant to a determination that the action is not part of the safety plan; and sending a notification that the deployment is not compliant with the safety plan.

[0203] Item 10. The computer-readable storage medium of item 9, wherein the deployment configuration file is generated by a central cloud infrastructure orchestration service.

[0204] Item 11. The central cloud infrastructure orchestration service is configured to generate a list of safety plans, each safety plan in the list of safety plans corresponding to a specific execution target or a specific deployment phase. A computer readable storage medium.

[0205] Item 12. The computer-readable storage medium of item 11, wherein the central cloud infrastructure orchestration service is further configured to contemporaneously generate the list of safety plans prior to receiving approval or disapproval of the list of safety plans.

[0206] Item 13. The computer-readable storage medium of item 12, wherein the central cloud infrastructure orchestration service is configured to generate a plurality of deployment configuration files, each deployment configuration file of the plurality of deployment configuration files being interpreted for each respective execution target on the list of safety plans.

[0207] Item 14. The computer-readable storage medium of item 10, wherein the operations further include sending the safety plan to a central cloud infrastructure orchestration service.

[0208] Item 15. Central cloud infrastructure orchestration services are Rendering the safety plan on a user interface of the user computer; Item 15. The computer-readable storage medium of item 14, configured to receive approval or disapproval of the safety plan.

[0209] Item 16. A system comprising: A processor; and a memory storing instructions, the instructions, when executed by the processor, causing the system to: receiving a safety plan including a list of resources and respective actions based at least in part on a deployment configuration file; Following receipt of instructions for approval of the safety plan, prepares to execute an action corresponding to at least one of a list of resources according to a deployment configuration file, Compare the behavior to the safety plan, Implementing the action pursuant to a determination that the action is part of the safety plan; Following a determination that the action is not part of the safety plan, Stop the deployment, A system that is configured to send notifications that a deployment is not compliant with the safety plan.

[0210] Item 17. The system of item 16, wherein the action is one of a plurality of actions corresponding to at least one of a list of resources according to a deployment configuration file, and each action of the plurality of actions is compared to the safety plan at least until the deployment is completed or an abort of the deployment occurs.

[0211] Item 18. The system of item 16, wherein the action is not part of a safety plan that is based at least in part on a behavior change that occurred in one of the resources before the action is performed.

[0212] Item 19. The system of item 16, wherein the deployment configuration file is generated by a central cloud infrastructure orchestration service.

[0213] Item 20. The system of item 19, wherein the notification is sent to a user of a central cloud infrastructure orchestration service.

[0214] Item 21. An apparatus comprising a means for performing the steps according to any one of items 1 to 8. Item 22. A method comprising: receiving, by at least one of the plurality of computer processors, a configuration file for deployment to a first execution target and a second execution target; and generating, by at least one of the plurality of computer processors, a first safety plan for the first execution target, the first safety plan including a first list of resources and respective operations associated with deployment on the first execution target based at least in part on the configuration file, the method further comprising: receiving, by at least one of the plurality of computer processors, an acknowledgement of the first safety plan; and generating, by at least one of the plurality of computer processors, a second safety plan for a second execution target, the second safety plan including a second list of resources and respective operations associated with deployment on the second execution target based at least in part on the configuration file, the method further comprising: at least one of the plurality of computer processors determining whether the second safety plan is a subset of the first safety plan; In accordance with a determination that the second safety plan is a subset of the first safety plan, at least one of the plurality of computer processors automatically approving the second safety plan based at least in part on the approval of the first safety plan; and at least one of the plurality of computer processors transmitting the automatically approved second safety plan to a second execution target.

[0215] Item 23. Pursuant to a determination that the second safety plan is not a subset of the first safety plan, a notification indicating that the second safety plan is not a subset of the first safety plan; 23. The method of claim 22, further comprising providing a user interface for presenting a second list of resources and respective actions associated with the second safety plan.

[0216] Item 24. The method of item 23, further comprising determining at least one difference between the second list of resources and respective actions of the second safety plan and the first list of resources and respective actions of the first safety plan.

[0217] Item 25. The method of item 24, further comprising providing a user interface for presenting the at least one difference above all other visual representations of the second list of resources and respective actions of the second safety plan.

[0218] Item 26. The method of item 22, wherein determining whether the second safety plan is a subset of the first safety plan includes at least one of the plurality of computer processors determining whether each resource and each action of a second list of resources and each action of the second safety plan is listed in the first list of resources and each action of the first safety plan.

[0219] Item 27. The method of item 22, wherein the first execution target is a connection region of a cloud infrastructure orchestration service.

[0220] Item 28. The method according to item 27, wherein the second execution target is a disconnected area of ​​a cloud infrastructure orchestration service. Item 29. The disconnected region includes an environment configured to receive instructions to deploy to the disconnected region, the disconnected region further configured to not transmit any data outside of the disconnected region. 29. The method according to claim 28,

[0221] Item 30. The method of item 28, wherein the second safety plan is for deployment on a second execution target.

[0222] Item 31. The method of item 30, wherein the second safety plan is automatically approved further pursuant to a determination that the deployment at the first execution target was successful based at least in part on the first safety plan.

[0223] Item 32. A computer-readable storage medium having instructions written thereon that, when executed by a computer processor, cause the computer processor to execute the instructions, the instructions comprising: receiving a configuration file for deployment to a first execution target and a second execution target; generating a first safety plan for the first execution target, the first safety plan including a first list of resources and respective operations associated with deployment on the first execution target based at least in part on the configuration file, the instructions further comprising: receiving approval of the first safety plan; and generating a second safety plan for a second execution target, the second safety plan including a second list of resources and respective actions associated with deployment on the second execution target based at least in part on the configuration file, the instructions further comprising: determining whether the second safety plan is a subset of the first safety plan; In accordance with a determination that the second safety plan is a subset of the first safety plan, automatically approving the second safety plan based at least in part on the approval of the first safety plan; and transmitting the automatically approved second safety plan to a second execution target.

[0224] Item 33. The Order further provides that pursuant to a determination that the second safety plan is not a subset of the first safety plan: a notification indicating that the second safety plan is not a subset of the first safety plan; Item 33. The computer-readable storage medium of item 32, comprising preparing a user interface for presenting a second list of resources and respective actions associated with the second safety plan.

[0225] Item 34. The computer-readable storage medium of item 33, the instructions further including determining at least one difference between the second list of resources and respective actions of the second safety plan and the first list of resources and respective actions of the first safety plan.

[0226] Item 35. The computer-readable storage medium of item 34, the instructions further including preparing a user interface for presenting the at least one difference above all other visual representations of the second list of resources and respective actions of the second safety plan.

[0227] Item 36. The computer-readable storage medium of item 32, wherein the second execution target is a disconnected region of a cloud infrastructure orchestration service, the disconnected region including an environment configured to receive instructions to deploy to the disconnected region, the disconnected region further configured to not transmit any data outside of the disconnected region.

[0228] Item 37. The second safety plan is for deployment on a second execution target. Item 37. The computer-readable storage medium of item 36, wherein the second safety plan is automatically approved further pursuant to a determination that the deployment at the first execution target based at least in part on the first safety plan was successful.

[0229] Item 38. A system comprising: A processor; and a memory storing instructions, the instructions, when executed by the processor, receiving a configuration file for deployment to a first execution target and a second execution target; Configuring the system to generate a first safety plan for a first execution target, the first safety plan including a first list of resources and respective actions associated with deployment on the first execution target based at least in part on the configuration file, the instructions further comprising: Receive approval of the first safety plan; and configuring the system to generate a second safety plan for a second execution target, the second safety plan including a second list of resources and respective actions associated with deployment on the second execution target based at least in part on the configuration file, the instructions further comprising: determining whether the second safety plan is a subset of the first safety plan; In accordance with a determination that the second safety plan is a subset of the first safety plan, automatically approving the second safety plan based at least in part on the approval of the first safety plan; The system is configured to send the automatically approved second safety plan to a second execution target.

[0230] Item 39. The system, pursuant to a determination that the second safety plan is not a subset of the first safety plan, presenting a notice indicating that the second safety plan is not a subset of the first safety plan; Item 39. The system of item 38, configured to provide a user interface for presenting a second list of resources and respective actions associated with the second safety plan.

[0231] Item 40. The system further: determining at least one difference between a second list of resources and respective actions of the second safety plan and the first list of resources and respective actions of the first safety plan; Item 40. The system of item 39, configured to provide a user interface for presenting the at least one difference over all other visual representations of the second list of resources and respective actions of the second safety plan.

[0232] Item 41. The system of item 38, wherein the second execution target is a disconnected region of a cloud infrastructure orchestration service, the disconnected region including an environment configured to receive instructions to deploy to the disconnected region, the disconnected region further configured to not transmit any data outside of the disconnected region, the second safety plan for deployment at the second execution target, the second safety plan being automatically approved further pursuant to a determination that the deployment at the first execution target was successful based at least in part on the first safety plan.

[0233] Item 42. An apparatus comprising a means for the steps according to any one of items 22 to 31.

Claims

1. 1. A method comprising: receiving, by at least one of the plurality of computer processors, a configuration file for deployment to a first execution target and a second execution target; and generating, by at least one of the plurality of computer processors, a first safety plan for the first execution target, the first safety plan including a first list of resources and respective operations associated with deployment on the first execution target based at least in part on the configuration file, the method further comprising: receiving, by at least one of the plurality of computer processors, an approval of the first safety plan; and generating, by at least one of the plurality of computer processors, a second safety plan for the second execution target, the second safety plan including a second list of resources and respective actions associated with deployment on the second execution target based at least in part on the configuration file, the method further comprising: at least one of the plurality of computer processors determining whether the second safety plan is a subset of the first safety plan; in response to determining that the second safety plan is a subset of the first safety plan, at least one of the plurality of computer processors automatically approving the second safety plan based at least in part on the approval of the first safety plan; and at least one of the plurality of computer processors transmitting the automatically approved second safety plan to the second execution target.

2. upon determining that the second safety plan is not a subset of the first safety plan, a notification indicating that the second safety plan is not a subset of the first safety plan; The method of claim 1 , further comprising: providing a user interface for presenting the second list of resources and respective actions associated with the second safety plan.

3. 3. The method of claim 2, further comprising determining at least one difference between the second list of resources and respective actions of the second safety plan and the first list of resources and respective actions of the first safety plan.

4. 4. The method of claim 3, further comprising preparing the user interface to present the at least one difference above all other visual representations of the second list of resources and respective actions of the second safety plan.

5. 10. The method of any one of the preceding claims, wherein determining whether the second safety plan is a subset of the first safety plan includes at least one of the plurality of computer processors determining whether each resource and each action of the second list of resources and each action of the second safety plan is listed in the first list of resources and each action of the first safety plan.

6. 10. The method of any one of the preceding claims, wherein the first execution target is a connection region of a cloud infrastructure orchestration service.

7. The method of claim 6 , wherein the second execution target is a disconnected region of the cloud infrastructure orchestration service.

8. 8. The method of claim 7, wherein the disconnected region includes an environment configured to receive instructions to deploy to the disconnected region, the disconnected region further configured to not transmit any data outside of the disconnected region.

9. The method of claim 7 or 8, wherein the second safety plan is for deployment on the second execution target.

10. 10. The method of claim 9, wherein the second safety plan is automatically approved further pursuant to a determination that a deployment at the first execution target based at least in part on the first safety plan is successful.

11. 1. A computer-readable storage medium having instructions written thereon that, when executed by a computer processor, cause the computer processor to execute the instructions, the instructions comprising: receiving a configuration file for deployment to a first execution target and a second execution target; generating a first safety plan for the first execution target, the first safety plan including a first list of resources and respective actions associated with deployment on the first execution target based at least in part on the configuration file, the instructions further comprising: receiving approval of the first safety plan; and generating a second safety plan for the second execution target, the second safety plan including a second list of resources and respective actions associated with deployment on the second execution target based at least in part on the configuration file, the instructions further comprising: determining whether the second safety plan is a subset of the first safety plan; in response to determining that the second safety plan is a subset of the first safety plan, automatically approving the second safety plan based at least in part on the approval of the first safety plan; and transmitting the automatically approved second safety plan to the second performance target.

12. The instructions further comprise, in response to a determination that the second safety plan is not a subset of the first safety plan, a notification indicating that the second safety plan is not a subset of the first safety plan; 12. The computer-readable storage medium of claim 11, further comprising providing a user interface for presenting the second list of resources and respective actions associated with the second safety plan.

13. 13. The computer-readable storage medium of claim 12, wherein the instructions further comprise determining at least one difference between the second list of resources and respective actions of the second safety plan and the first list of resources and respective actions of the first safety plan.

14. The instructions further include a second association between resources and respective actions of the second safety plan.

14. The computer-readable storage medium of claim 13, further comprising preparing the user interface to present the at least one difference above all other visual representations of a list.

15. 15. The computer-readable storage medium of claim 11, wherein the second execution target is a disconnected region of the cloud infrastructure orchestration service, the disconnected region including an environment configured to receive instructions to deploy to the disconnected region, the disconnected region further configured not to transmit any data outside of the disconnected region.

16. 16. The computer-readable storage medium of claim 15, wherein the second safety plan is for the deployment at the second execution target, and the second safety plan is automatically approved further pursuant to a determination that the deployment at the first execution target is successful based at least in part on the first safety plan.

17. 1. A system comprising: A processor; and a memory storing instructions that, when executed by the processor, receiving a configuration file for deployment to a first execution target and a second execution target; Configuring the system to generate a first safety plan for the first execution target, the first safety plan including a first list of resources and respective actions associated with deployment on the first execution target based at least in part on the configuration file, the instructions further comprising: receiving approval of the first safety plan; configuring the system to generate a second safety plan for the second execution target, the second safety plan including a second list of resources and respective actions associated with deployment on the second execution target based at least in part on the configuration file, the instructions further comprising: determining whether the second safety plan is a subset of the first safety plan; in response to determining that the second safety plan is a subset of the first safety plan, automatically approving the second safety plan based at least in part on the approval of the first safety plan; The system is configured to transmit the automatically approved second safety plan to the second execution target.

18. The system further comprises, in response to determining that the second safety plan is not a subset of the first safety plan, presenting a notification indicating that the second safety plan is not a subset of the first safety plan; 20. The system of claim 17, further configured to provide a user interface for presenting the second list of resources and respective actions associated with the second safety plan.

19. The system further comprises: determining at least one difference between the second list of resources and respective actions of the second safety plan and the first list of resources and respective actions of the first safety plan; 20. The system of claim 18, further configured to prepare the user interface for presenting the at least one difference above all other visual representations of the second list of resources and respective actions of the second safety plan.

20. 20. The system of claim 17, wherein the second execution target is a disconnected region of the cloud infrastructure orchestration service, the disconnected region including an environment configured to receive instructions to deploy to the disconnected region, the disconnected region further configured to not transmit any data outside of the disconnected region, the second safety plan is for deployment at the second execution target, and the second safety plan is automatically approved further pursuant to a determination that the deployment at the first execution target is successful based at least in part on the first safety plan.

21. Apparatus comprising means for the steps according to any one of claims 1 to 10.

Images