Techniques for using directed acyclic graphs for deployment instructions

Abstract

To provide a method of, a system for, and a storage medium for utilizing a directed acyclic graph for deployment instructions.SOLUTION: A method realized by a computer includes various operations. A computing device executes instructions for executing a plurality of times of analysis of setting data relating to deployment to generate a first DAG (directed acyclic graph). The first DAG is used for deploying a first resource based on the analysis. The computing device generates a second DAG for use in deployment of an execution target based on the analysis. The second DAG designates a dependency relation among the execution targets for deployment. The computing device further generates a connection list data structure based on the analysis and traverses the connection list data structure to thereby deploy the computing system.SELECTED DRAWING: Figure 15

Description

Technical Field

[0001] This application is a regular application and claims the benefit and priority of U.S. Provisional Application No. 62 / 963,477, filed on January 20, 2020, titled "TECHNIQUES FOR UTILIZING DIRECTED ACYCLIC GRAPHS FOR DEPLOYMENT INSTRUCTIONS", and U.S. Regular Application No. 16 / 953,262, filed on November 19, 2020, titled "TECHNIQUES FOR UTILIZING DIRECTED ACYCLIC GRAPHS FOR DEPLOYMENT INSTRUCTIONS" under 35 U.S.C. § 119(e). The entire disclosure of each of these applications is hereby incorporated by reference for all purposes.

Background Art

[0002] Background Today, cloud infrastructure services use many individual services for provisioning and deploying code and settings (respectively) across many areas of cloud infrastructure services. In particular, considering that provisioning is generally declarative and code deployment is imperative, these tools require a significant amount of manual effort to use. In addition, as the number of service teams and regions increases, cloud infrastructure services also need to continue to grow. As a strategy for some cloud infrastructure services to deploy to more and smaller regions, there is a strategy to break down spending by region, but it has not been able to cope well.

Summary of the Invention

Means for Solving the Problems

[0003] Summary This paper discloses techniques for using directed acyclic graphs for deployment instructions. In some embodiments, the methods implemented by a computer may include a variety of operations. Instructions for performing the analysis of configuration data related to deployment may be executed by a computing device. The computing device may generate a first directed acyclic graph (DAG), which is used to deploy a first resource (e.g., a software service) based on the analysis. A second DAG may be generated for deploying execution targets based on phases, which specifies the dependencies between the execution targets of the deployment. The computing device may also generate a linked list data structure based on the analysis, and the computing system may be deployed by traversing the linked list data structure.

[0004] Other embodiments disclose a system for using a DAG for deployment instructions. This system may comprise one or more processors and one or more memories storing computer-executable instructions, which, when executed by the one or more processors, configure the one or more processors to perform various operations. A computing device may execute instructions for performing one or more analyses of configuration data related to the deployment of a computing system. The computing device may generate a first DAG, which is used to deploy a first resource based at least in part on one or more analyses. The computing device may use multiple executioners based at least in part on one or more analyses. A second DAG may be generated for deploying the get, the second DAG specifying the dependencies between the deployment execution targets. The computing device may generate a linked list data structure based at least in part on one or more parsing runs, the linked list data structure specifying the dependencies between multiple deployment phases. The computing device may then deploy the computing system based at least in part on traversing the linked list data structure, the second DAG, and the first DAG.

[0005] Other embodiments disclose a computer-readable storage medium for using a DAG for deployment instructions, which can store computer-executable instructions. When executed by one or more processors, the computer-executable instructions cause those processors to perform various operations. A computing device may execute instructions for performing one or more analyses of configuration data related to the deployment of a computing system. The computing device may generate a first DAG, which is used to deploy a first resource based at least in part on one or more analyses. The computing device may generate a second DAG for deploying multiple execution targets based at least in part on one or more analyses, which specifies dependencies between the execution targets of the deployment. The computing device may generate a linked list data structure based at least in part on one or more analyses, which specifies dependencies between multiple deployment phases. The computing device can then deploy the computing system based at least in part on traversing the linked list data structure, the second DAG, and the first DAG.

[0006] Other embodiments disclose an apparatus which may comprise means for performing steps of the method disclosed herein.

[0007] In other embodiments, a computer program product is disclosed. This computer program product may include computer instructions. When executed by a processor, these computer instructions perform the steps of the method disclosed herein.

[0008] To facilitate understanding of descriptions of specific elements or actions, the leading one or more digits in a reference number indicate the drawing number in which that element is first introduced. [Brief explanation of the drawing]

[0009] [Figure 1] This is a block diagram of an architecture for implementing at least some elements of a cloud infrastructure orchestration service according to at least one embodiment. [Figure 2] This is a block diagram of an architecture for implementing at least some elements of a cloud infrastructure orchestration service according to at least one embodiment. [Figure 3] This is a flowchart illustrating an exemplary Flock according to at least one embodiment. [Figure 4] This is a flowchart illustrating an exemplary Flock according to at least one embodiment. [Figure 5] This is a user interface that presents information about a release including multiple phases and multiple execution targets, relating to at least one embodiment. [Figure 6] This is an exemplary code segment for defining a list and sequence of phases, relating to at least one embodiment. [Figure 7] This is an exemplary data structure that may be generated by CIOS (Cloud Infrastructure Orchestration Service) to hold lists and sequences related to one or more phases, relating to at least one embodiment. [Figure 8]This is an exemplary code segment for defining a list and order of execution targets, relating to at least one embodiment. [Figure 9] This is an exemplary data structure that may be generated by CIOS (Cloud Infrastructure Orchestration Service) to hold a list and order related to one or more execution targets related to a phase, according to at least one embodiment. [Figure 10] This is an exemplary code segment for constructing explicit and implicit dependencies between resources of an execution target, relating to at least one embodiment. [Figure 11] This is an exemplary code segment for constructing explicit and implicit dependencies between resources of an execution target, relating to at least one embodiment. [Figure 12] This is an exemplary directed acyclic graph corresponding to resources in a cloud computing system, relating to at least one embodiment. [Figure 13] This flowchart illustrates an exemplary process for orchestrating the execution of a task, which involves a dependency on at least one capability, according to at least one embodiment. [Figure 14] An exemplary processing flow of a computer infrastructure orchestration service according to at least one embodiment, traversing a linked list, a directed acyclic graph of execution targets, and a directed acyclic graph of capabilities. [Figure 15] This is a flowchart of the process for deploying a computing system using a directed acyclic graph in a computer infrastructure orchestration service, according to at least one embodiment. [Figure 16] This is a block diagram of a distributed system relating to at least one embodiment. [Figure 17]This is a block diagram of one or more components of a system environment according to at least one embodiment, and these components enable the provision of services offered by one or more components of the system of the embodiment as a cloud service. [Figure 18] This is a block diagram of an exemplary computer system in which various embodiments of the present disclosure may be implemented. [Modes for carrying out the invention]

[0010] Detailed explanation In some cases, IaaS (Infrastructure as a Service) is a specific type of cloud computing. IaaS can be configured to deliver virtualized computing resources over a public network (e.g., the Internet). In some cases, IaaS is one of three main classifications (or subcategories) of cloud computing services. Many people consider the other main classifications to be SaaS (Software as a Service) and PaaS (Platform as a Service), and SaaS may be seen as a broader classification that includes both PaaS and IaaS, while others may see IaaS as a subcategory of PaaS.

[0011] In the IaaS model, a cloud computing provider can host infrastructure components such as servers, storage devices, network nodes (e.g., hardware), deployment software, and platform virtualization (e.g., the hypervisor layer).

[0012] In some cases, IaaS providers may supply various services associated with these infrastructure components (e.g., billing, monitoring, logging, security, load balancing, and clustering). Therefore, these services are part of the policy. Since it can be a driving service, IaaS users will be able to implement policies that drive load distribution to maintain application availability and the performance of the application.

[0013] In some cases, IaaS customers may access resources and services through a WAN (Wide Area Network) such as the Internet and can use the cloud provider's services to install the remaining elements of the application stack. For example, a user can log in to the IaaS platform to create a VM (Virtual Machine), install an OS (Operating System) on each VM, deploy middleware such as a database, create storage buckets for workload and backup, and even install enterprise software on the VM. Then, the customer can use the provider's services to perform various functions, including load distribution of network traffic, troubleshooting application problems, performance monitoring, disaster recovery management, and the like.

[0014] In most cases, the cloud computing model requires the participation of a cloud provider. The cloud provider may be a third-party service specializing in the provision (e.g., sale) of IaaS, but it doesn't have to be. Also, an entity may choose to deploy a private cloud and become a provider of its own infrastructure services.

[0015] In some cases, IaaS deployment is the process of deploying a new application or a new version onto a pre-configured application server, etc. It may also include the process of preparing the server (e.g., installing libraries, daemons, etc.). This is often managed by the cloud provider at a lower layer than the hypervisor layer (e.g., servers, storage, network hardware, and virtualization). Therefore, the customer may be responsible for handling application deployment (e.g., on self-service virtual machines, etc., which can be spun up on demand), middleware, and / or application deployment.

[0016] In some examples, IaaS provisioning might refer to acquiring the computers or virtual hosts to be used, and even installing the necessary libraries or services on them. In most cases, a deployment does not include provisioning, and provisioning will need to be done first.

[0017] In some cases, IaaS provisioning presents two distinct challenges. First, there's the initial challenge of provisioning an initial set of infrastructure before doing anything. Second, once everything is provisioned, there's the challenge of evolving the existing infrastructure (e.g., adding new services, modifying services, removing services, etc.). In some cases, these two challenges can be addressed by allowing the infrastructure configuration to be defined declaratively. That is, one or more configuration files can define the infrastructure (e.g., what components are needed and how they interact). Thus, the overall topology of the infrastructure (e.g., which resources depend on which resources and how they interact with each other) can be described declaratively. In some cases, once the topology is defined, a workflow can be generated to create and / or manage each of the different components described in the configuration files.

[0018] In some embodiments, IaaS provisioning may include generating a DAG (Directed Acyclic Graph). The DAG may include any appropriate number of nodes and edges. It can be a finite directed graph. Each edge is directed from one node to another. Nodes and edges can be arranged so as not to form a directed cycle. That is, the DAG is defined such that, starting from any node, no consistently directed sequence of edges eventually circulates back to that same node. IaaS provisioning may include parsing configuration files corresponding to one or more resources of the system (e.g., services, software resources). A separate DAG may be generated for each resource. The DAG for each resource can define that resource's dependency on the capabilities of one or more other resources of the system (e.g., services, software resources). For example, a first DAG can be generated and used to deploy a first resource (e.g., a software service such as an email service configured to manage electronic messages for one or more users, also referred to as a "computing service"). The first DAG may show the first resource's dependency on the capabilities provided by a second resource (e.g., another software service different from the first resource, such as an identity service configured to verify / authenticate a user's identity based on previously obtained credentials). The first resource and / or the second resource may each be one of several services provided by the computing system. "Capability" may refer to a part of the functionality of a given resource (for example, the capability of the second resource to verify / authenticate a user's identity). Processes may be instantiated to traverse the DAG. If a process reaches a node in the DAG corresponding to a capability that is currently unavailable, it may publish an indication to the scheduling service that the process has reached a dependency on that capability and will wait for that particular capability to become available before proceeding. As various resources of the system are deployed and / or started, these resources may publish indications to the scheduling service that various capabilities are available when those capabilities become available.As used herein, the terms “start up,” “starting,” and “started” refer to the process of performing a startup sequence consisting of actions corresponding to a particular resource (e.g., a software / computing service, a computing device, etc.). Deploying a resource (e.g., a software service) may include starting and / or making available at least some of the functions that the resource provides (e.g., one or more capabilities). If the scheduling service determines that a particular capability is available, it may resume the process from where it last terminated (e.g., immediately after it announced that the capability was needed). This process may restart the traverse by regenerating the DAG (Directed Agglutination Loop) (e.g., from the last accessed node). By utilizing a DAG for each resource, the system can manage dependencies between the capabilities of different resources, eliminating the need for human operators to manually start complex systems.

[0019] In some examples, infrastructure can have many interconnected elements. For example, there may be one or more VPCs (Virtual Private Clouds), also known as core networks (which could be, for example, configurable computing resources and / or on-demand pools of shared computing resources). In some examples, there may also be one or more security group rules and one or more VMs (Virtual Machines) provisioned to define how network security is set up. Other infrastructure elements, such as load balancers and databases, may also be provisioned. As more infrastructure elements are requested and / or added, the infrastructure will evolve incrementally.

[0020] As mentioned above, one way to provision infrastructure is to describe it declaratively. Therefore, the configuration file describes each of the infrastructure components mentioned above and how they interact with each other. A configuration file can be a declarative file that simply describes the configuration. The configuration file can describe the resources and relevant fields necessary to create an element, and may describe other elements that reference previously described elements. In some examples, a provisioning tool may then generate a workflow for creating and managing the elements described in the configuration file.

[0021] In some cases, the provisioning tool's workflow can be configured to execute various commands. One function it can perform is view reconciliation. In view reconciliation, the provisioning tool can compare a current view of the infrastructure (e.g., the expected state of the infrastructure) to the actual behavior of the infrastructure. In some cases, performing the view reconciliation function may also involve querying various resource providers or infrastructure resources to identify which resources are actually operational. Another function that the provisioning tool can perform is plan generation. In plan generation, the provisioning tool can compare the infrastructure components that are actually operational to how the provisioning tool would look in a desired state (e.g., a desired configuration). That is, the plan generation function can determine what changes need to be made to bring resources to the latest expected state. In some cases, a third function is the execution (e.g., apply) function. In the execution function, the provisioning tool can execute the plan generated by the plan generation function.

[0022] Generally, provisioning tools can be configured to retrieve configuration files, parse the declarative information contained within them, and programmatically / automatically determine the order in which resources must be provisioned to execute a plan. For example, if a VPC needs to be started before security group rules and VMs are started, the provisioning tool can make this decision and ensure that the startups are performed in the determined order without user intervention and / or without requiring that information to be included in the configuration file.

[0023] In some cases, continuous deployment techniques can be employed to enable the deployment of infrastructure code across various virtual computing environments. In addition, the techniques described can enable infrastructure management within these environments. In some examples, a service team may write code that is desired to be deployed to one or more production environments, often many different production environments (e.g., across various different geographical locations, and possibly even across the entire world). However, in some examples, the infrastructure to which the code will be deployed must first be set up. In some cases, provisioning can be done manually, provisioning tools may be used to provision resources, and / or, once the infrastructure is provisioned, deployment tools may be used to deploy the code.

[0024] As mentioned above, there are generally two different tools, each used to address the provisioning of infrastructure resources and the deployment of code to control those resources, respectively, with manual orchestration between the tools. However, at scale, manual implementations inevitably suffer from discrepancies. Therefore, automated tools capable of both provisioning and deploying virtual infrastructure enable more efficient and reliable techniques for implementing virtual cloud environments.

[0025] In some cases, when two tools are used, problems can arise if a user manually makes code changes between the provisioning and deployment phases. As described herein, the technique of using a single tool for both provisioning and deployment can mitigate such problems by automating the process and eliminating the opportunity to manually change the code. In some cases, a slight change in coding by one user could potentially cause a major problem in the deployment phase. In some cases, when an operator performs an operation in a new region for the first time (for example, with a typo in the code), an object with the typo may remain. If the application is deployed with that typo and the application is not affected by the typo (for example, it works for the time being), then additional code changes in the future could be affected by that typo and crash the entire system. Therefore, the technique provided herein can eliminate the provisioning and deployment gap, which can often lead to problems.

[0026] Generally, modeling a deployment is declarative, allowing the use of configuration files to declare infrastructure resources. For example, CRUD (create, read, update, and delete) commands are commonly used to generate deployment files using the common REST (Representational State Transfer) concept (e.g., a REST API (Application Programming Interface)). However, deployment itself generally does not follow the same concept. In addition, while infrastructure provisioning tools tend to be very powerful and / or expressive, tools for deployment tend to be much more limited in terms of the operations they can perform (e.g., tools for deployment are imperative, as opposed to declarative). Thus, for a long time, there has been a need for a single tool that can address both functional requirements (e.g., provisioning and deploying infrastructure elements) within a cloud environment.

[0027] Several examples describe the technologies for implementing CIOS (Cloud Infrastructure Orchestration Services). These technologies can be configured to manage both the provisioning and deployment of infrastructure resources within a cloud environment, as briefly mentioned above. In some cases, CIOS may include two service classes: a central component and regional components (e.g., CIOS Central and CIOS Regional). The following terms will be used throughout this specification.

[0028] ● Infrastructure Components - Long-term components of the infrastructure that support the code in operation.

[0029] Examples: Deployment applications, load balancers, DNS (Domain Name System) entries, object storage buckets, etc.

[0030] ● Artifact-deployment application or Kubernetes Configuration information applied to code deployed in the Engine cluster or to infrastructure components (hereinafter referred to as "config"). These may be read-only resources.

[0031] ● Deployment tasks – short-term tasks often related to deploying or testing code. In addition, deployment tasks are modeled as resources. These resources do not exist longer than the release that created them.

[0032] ○Examples: "deploy $artifact to $environment", "watch $alarm for 10 minutes", "execute $testSuite", or "wait for $manualApproval" For example, CIOS can model the deployment of a deployment orchestrator as the creation of a resource that transitions to an available state when completed.

[0033] ○CIOS maintains the state of its cloud infrastructure service declarative provisioners, and therefore CIOS can control the lifecycle of these short-term resources, as that lifecycle is related to releases.

[0034] ● Resources - Resources that can be CRUD (Create, Read, Uninstall, Delete). ○CIOS models each of the constructs listed above as resources. The next section will explain this modeling in detail.

[0035] ●Flock - A CIOS model that encapsulates the control area and all components. Its primary purpose is to model the ownership of infrastructure components and to refer to those infrastructure components.

[0036] ●Flock Configuration - Describes all the infrastructure components, artifacts, and deployment tasks associated with a single service.

[0037] Each Flock has exactly one Flock setting. The Flock setting is checked in to determine the source of control.

[0038] ○Flock configuration is declarative. It requires CIOS to provide the realm, region, ad, and artifact versions as input.

[0039] ○Flock has a fine granularity - Flock consists of a single service that supports the infrastructure.

[0040] ●Status - A snapshot of the current state of all resources in Flock. ●Release - A tuple consisting of the specific version of the Flock configuration and the specific versions of all artifacts it references.

[0041] ○We assume that the release describes a state that does not yet exist. ●Release Plan - A series of steps CIOS takes to transition all regions from their current state to the state described in the release.

[0042] ○The release plan has a finite number of steps and clearly defined start and end times.

[0043] ●Application - This is a noun. One attempt to execute a release plan. The execution changes the current state of Flock.

[0044] CIOS can be described as an orchestration layer that applies configurations to downstream systems (for example, globally). It is designed to enable global infrastructure provisioning and code deployment without manual intervention from service teams (for example, possibly after initial approval). Higher levels of CIOS responsibilities include, but are not limited to, the following:

[0045] ● Provides teams with a view of the current state of resources managed by CIOS, including in-flight change operations.

[0046] ● Assist the team in planning and releasing new changes. ● Coordinate operations across various downstream systems within a region to execute approved release plans without human intervention.

[0047] ● Coordinate operations across regions / realms to execute approved release plans globally.

[0048] In some cases, CIOS addresses onboarding by allowing teams to provide CIOS with configuration information via checked-in code. In addition, CIOS can automate other operations, making it more critical than operations in previous implementations. In some cases, CIOS addresses pre-deployment by providing teams with the ability to automatically deploy and test code. In some cases, CIOS can address writing CM (Change Management) policies by allowing teams to automatically generate plans for rolling out new artifacts (e.g., globally) when they build new artifacts. This can be done by examining the current state of each region and the current CIOS configuration (which can itself be an artifact). In addition, teams can examine these plans and iterate through them by changing CIOS configurations and requesting CIOS to rewrite the plans. Once satisfied with the plans, teams can create “releases” that reference the plans. They can then score the plans to approve or reject them. Teams can still write CMs, but these are merely pointers to CIOS plans. Thus, teams can reduce the time they spend thinking about plans. Since the plan is machine-generated, its accuracy improves. The plan is too detailed for humans to understand, but it can be displayed through an advanced UI (user interface).

[0049] In some cases, CIOS can handle CM execution by automatically executing deployment plans. Once a release plan is created and approved, engineers are no longer required to participate in CM unless CIOS initiates a rollback. This may, in some cases, require teams to automate tasks that are currently manual. In some cases, CIOS can handle CM (Change Management) rollbacks by automatically generating a plan to revert Flock to its original (e.g., pre-release) state if it detects a decline in service health during operation. In some cases, CIOS can handle deploying sudden / strategic changes by receiving release plans that are limited in scope to a subset of regions and / or a subset of resources managed by CIOS, and then executing the plan.

[0050] In addition, CIOS can support the primitives necessary to define fully automated global deployments. For example, CIOS can measure service health by monitoring alarms and running integration tests. CIOS can help teams quickly define rollback behavior when a service goes down, and then have it executed automatically. CIOS can automatically generate and display release plans and track approvals. In some cases, the language teams use to describe the desired deployment behavior may be declarative. CIOS can combine code deployment capabilities and infrastructure configuration (e.g., provisioning) into a single system. CIOS can also be regional. It supports flexible ordering between components and within a region. Teams can indicate ordering through checked-in configurations. Teams can programmatically call the CIOS Planning and Release APIs.

[0051] Figure 1 shows an architecture 100 illustrating the techniques for implementing at least CIOS Central 102. In some examples, CIOS Central 102 may be a service that handles "Flock" level operations. CIOS Central 102 has several responsibilities, but is not limited to those listed below.

[0052] ● Responsible for acting as the authentication gateway for Flock metadata changes and release operations.

[0053] ● Store reliable mappings of Flock metadata in the deployment artifacts and the Flock CIOS repository.

[0054] ● Coordinate global releases across phases and targets. ● Synchronization to enforce policies such as "Only one ongoing release to Flock at a time."

[0055] ● Detects changes to Flock settings and artifacts, and triggers the generation of releases in response to those changes.

[0056] In some examples, SCVMS (Source Code Version Control / Management Service) 104 can be configured to store trusted Flock configurations, allowing CIOS Central 102 to subscribe to ANS (Artifact Notification Service) 106, and CIOS Central 102 will be notified about the creation of new artifacts. CIOS Central 102 can then map the changes that will reach the affected Flocks and, if necessary, begin release planning. In addition, in some examples, CIOS Central 102 can invoke APS (Artifact Push Service) before a release to the target, ensuring that all artifacts necessary for a successful release are presented in the target region before the release.

[0057] In some examples, a customer (e.g., an engineer) 108 can call CIOS Central 102 for CRUD Flock and / or releases to view the status of ongoing CIOS activity. The Flock Management Service 110 may include one or more APIs for manipulating Flocks, the View / Plan / Approval Service 112 may include CRUD APIs for creating and approving plans, and CRUD APIs for viewing a central copy of all state of resources managed by CIOS, the Change Monitoring Service 114 may watch SCVMS 104 to see changes made to Flock configurations and receive notifications about changes made to other artifacts from ANS 106, and the State Ingester Service 116 may create a copy of the regional state in CIOS Central DB (database) 118 so that View / Plan / Approval 112 can publish it. In some examples, CIOS Central DB 118 may be a DB for Flocks, plans, and states. Flock information may be reliable information, while others may be old copies of data from CIOS Regional 120. CIOS Central 102 provides any appropriate portion and / or any number of user interfaces (e.g., user interfaces 500-1300) for presenting any appropriate data regarding Flock, releases, infrastructure components, artifacts, etc. It can be configured in this way. In some embodiments, CIOS Central 102 may present data for one or more releases through any suitable interface. A release may include any suitable combination of tasks for one or more infrastructure components and / or one or more code changes to one or more applications (e.g., artifacts). Some examples of user interfaces provided by CIOS Central 102 are described below in Figures 5 to 13.

[0058] In some examples, engineer 108 can create a list of Flocks by making API calls to the Flock management service 110 (for example, through the ingress proxy fleet 122). The protocol used for such API calls may be HTTPS (Hypertext Transport Protocol Secure). Relevant ACLs (Access Control Lists) for this operation may include the LAN (Local Area Network) 124 or other private connections. For example, CIOS may manage / control network connectivity instead of using the public internet (e.g., dedicated connections, leased connections, and / or private connections) to connect a customer's on-premises data center or network to CIOS. In addition, authentication and authorization (for example, engineer 108) may be performed by a reservation system portal that allows users to manage machine infrastructure (e.g., reservation services). In some cases, CIOS Central 102 may store Flock metadata, plans, and status in the Central DB 118 using JDBC (Java Database Connectivity), etc. In some examples, ANS 106 can be configured to notify the change monitoring service 114 when new artifacts are published. ANS106 may utilize HTTPS, and both authentication and authorization may be handled by the mutual transport layer security service. In addition, the change monitoring service 114 may, in some cases, poll SCVMS104 for changes in Flock settings. This polling may be performed using SSH (Secure Shell) or other protocols. Authentication for the change monitoring service 114 may be handled by the CIOS system account, and authorization may be handled by SCVMS104.

[0059] In some examples, Engineer 108 can use the View / Plan / Approval Service 112 to perform one or more of the following actions: Engineer 108 can perform plans and / or approvals by calling CIOS Central 102 to generate and approve plans. Engineer 108 can view the status of ongoing global CIOS operations by calling CIOS Central 102. In addition, Engineer 108 can call CIOS Central 102 to view a copy of the state of global resources managed by CIOS. These API calls (or similar) can be made via the HTTPS protocol or a similar protocol. In addition, relevant ACLs can be controlled by LAN 124, and both authentication and authorization can be handled by the Reservation Service. In some examples, the View / Plan / Approval Service 112 can request planning and promote plan approval for all regions of CIOS Regional 120 (e.g., using HTTPS). Relevant ACLs can be controlled using security lists managed by the WAN (Wide Area Network) Gateway 126. Authentication can be handled by mutual transport layer security, and authorization can be handled by various identity policies. Furthermore, the status Ingester service 116 can monitor the CIOS regional 120 to check for changes in job status or state, and CIOS can provide these central views on request (for example, using HTTPS). ACLS for this can also be handled by the WAN gateway 126, and both authentication and authorization can be handled by mutual transport layer security services.

[0060] Figure 2 shows Architecture 200 to illustrate the techniques for implementing at least CIOS Regional 202. In some examples, much of the work in CIOS Regional 202 may involve declarative provisioning and planning, as well as approved release application work. In some cases, each instance of CIOS Regional 202 may have a regional frontend that can handle “run target” level operations. It can be configured to perform the following operations:

[0061] ● Handles all CIOS authentication related to operations received from CIOS Central 102.

[0062] ● Implement a rule that, for a given execution target, only one "execution" (planning / importing resources / applying a plan) may be in progress at a time.

[0063] ● Manages the storage of binary artifacts for declarative provisioning artifacts used as inputs and outputs during the execution of declarative infrastructure provisioning. Examples of inputs include declarative infrastructure provisioning configuration files and input state files. The output is typically the final state file.

[0064] ● Request the CIOS executor to perform a given execution task, and then poll the CIOS executor for the results.

[0065] In some cases, the CIOS frontend may rely on the CIOS Executor 206 (also referred to herein as the “Scheduler”), which can handle actual execution. In some examples, the CIOS Executor operates at the “execution” level and can perform the following actions:

[0066] ● Track the pool of available worker nodes. ●Query incoming job requests and assign them to qualified workers when they become available.

[0067] ● Track worker status and run updates to report to clients.

[0068] ● Dead nodes can be detected via the leasing protocol, and the functionality of tasks assigned to dead nodes can be stopped depending on the task status.

[0069] ● Provides facilities for canceling / terminating / pausing / resumeing execution, and allows these functions to be mapped to the facilities to pass cancellation / termination / resume information to worker nodes.

[0070] In some cases, a CIOS executor may rely on CIOS workers that can assign tasks to perform, and can provide facilities that allow workers to update the progress of jobs. Worker services operate at the "task" granularity. Each worker is an agent that performs the tasks assigned to it and reports the status and output of those tasks. Each worker can perform the following actions:

[0071] ● Polling the Executor Worker API for the assigned work items and performing actions to match the assigned state to its local state.

[0072] ○ Start a container to poll for task items that do not exist locally. ○ Terminate any containers that are running locally and do not have a corresponding task item assigned to them.

[0073] ● Report the job status. ● Stage input / output for job container execution.

[0074] ● Start and monitor the declarative infrastructure provisioning container to perform the actual work of the release for the execution target.

[0075] A CIOS worker may depend on a CIOS executor to poll for work from the CIOS executor's worker endpoint and report results to the CIOS executor's worker endpoint. The worker may depend on the executor for all coordination. In addition, a CIOS worker may depend on CIOS Regional 202, in which a worker service reads input from one or more APIs related to a regional frontend service and writes output to those APIs. Examples of inputs include configuration files, startup state files, and import mappings. Examples of outputs include declarative provisioning processes, output of declarative provisioning state files, and import of result states.

[0076] In some examples, CIOS Regional 202 may be a regional service for managing regional instances / deployments of CIOS. CIOS Regional 202 covers the responsibility of authoritatively storing and managing plans and state relevant to a particular region. Regional DB 204 may be a CIOS DB for state and plans in a specific region. This is a trusted copy of a subset of regions of the central DB 118 in Figure 1. Scheduler 206 may be responsible for managing the worker fleet capacity, assigning tasks to workers, and tracking task status. In some cases, Task DB 208 is another CIOS DB for task status. The data in this DB is, in most cases, operational data. In addition, worker 210 may be a fleet of JVMs (Java Virtual Machines) managing a declarative provisioning image. These receive instructions from Scheduler 206 and communicate the results to both Scheduler 206 and CIOS Regional 202. The CIOS container 212 can perform declarative provisioning operations in its own private Docker 214 container. This container does not need to contain any secrets. In addition, in some examples, to avoid secrets being embedded in the declarative provisioning image, the signing proxy 216 can be configured to prevent secrets from being leaked through the declarative provisioning tool. Instead, CIOS can perform request signing or the proxy can initiate an mTLS (Mutual Transport Layer Security) service. This also facilitates the use of FIPS-compliant cryptographic libraries.

[0077] In some examples, CIOS Central 102 can call CIOS Regional 202 to create and approve plans, monitor job status (service principals), and extract declarative provisioner status (service principals). Ingress proxy 218 can be configured as an ACL, and various identity policies may be used for authentication and authorization. Alternatively, in some examples, ingress proxy 218 can be configured to distribute the load of incoming requests, plans, etc. This can be replaced with a configured load balancer. In some cases, CIOS regional 202 may request scheduler 206 to execute a declarative provisioner, and this execution may be performed. Worker 210 can confirm with scheduler 206 what should be running and report the status to scheduler 206 upon completion. In some cases, mTLS may handle both authentication and authorization for CIOS regional 202 and worker 210. In addition, if a declarative provisioner needs to be executed, worker 210 will perform this execution within a Docker container by interacting with local Docker 214. Authentication at this stage may be handled by a local Unix® socket. The Docker protocol may be used for this final step, but HTTPS may be used for the preceding steps.

[0078] In some embodiments, CIOS Regional 202 may be configured to provide any suitable portion and / or any number of user interfaces (e.g., user interfaces 500-1300) for presenting any suitable data regarding Flocks, releases, infrastructure components, artifacts, etc. In some embodiments, CIOS Regional 202 may present data regarding one or more releases through any suitable interface. A release may include any suitable combination of tasks regarding one or more infrastructure components, and / or one or more code changes to one or more applications (e.g., artifacts). Some examples of user interfaces provided by CIOS Regional 202 are described below in Figures 5-13.

[0079] In some examples, the CIOS container 212 allows the declarative provisioner to interact with the signing proxy 216 (via API), while the signing proxy 216 believes that the signing proxy 216 is calling various CIOS services. The signing proxy 216 listens on one ephemeral port for each instance of that declarative provisioner's call that only the declarative provisioner knows about. The signing proxy 216 can initiate signature or mTLS requests and pass the declarative provisioner's call to other CIOS services within the service enclave. In some cases, the signing proxy 216 can also communicate with one or more public CIOS services 220. For example, the signing proxy 216 utilizes the internal endpoint of a public service, where possible. For services that do not have an internal endpoint, it must utilize the egress proxy 222 to reach the external endpoint. The use of this signature proxy 216 does not necessarily have to be for inter-region communication; for example, the egress proxy whitelist in each region may only whitelist the public IP range of that region. In some examples, worker 210 may then hold state and logs from declarative provisioners in CIOS regional 202, so that these are leaked to CIOS central 102.

[0080] When using CIOS, there are four typical customer experience phases: onboarding, pre-release, global release, and strategic release. In the pre-release phase, examples of events that occur between the new artifact being built and the artifact released in Release 1 (e.g., R1) include: This will replace some or most of the current change management process. Once the relevant artifacts are built, CIOS can automatically generate a release using "all the latest versions in Flock." A release is a specific version of the Flock configuration with specific inputs (e.g., artifact version, realm, region, and advertisement). A release includes one roll-forward plan per region and metadata describing the region ordering. Each regional plan is a set of actions that a declarative provisioner can take to implement a Flock configuration in that region. Teams with a pre-release environment can use CIOS to automatically release and test software in that environment. Teams can configure CIOS to automatically test rollback plans. Teams can inspect and approve releases through the CIOS UI. Teams can approve some, but not all, of the regional plans in a release. If "All Latest Versions" does not provide a suitable plan, the team can request CIOS to generate a plan for selected artifact versions.

[0081] For a global release phase, here's an example of how a team might run tomorrow's version of today's "normal CM": Once the release is approved, CIOS propels the approved regional plan down to each region. CIOS operates independently within each region, enforcing the approved plan. CIOS only performs the set of actions explicitly described in the plan for that region; it will stop working rather than "think for itself." The CIOS UI displays the team's progress of the execution. If manual approval is required, the CIOS UI prompts the team. If the execution fails due to a failure in CIOS or a downstream service, CIOS can notify the team and prompt for the next steps (e.g., abort, retry). CIOS will retry, but the failure in the downstream system may outweigh the willingness to retry. If the execution fails due to poor service health or a test failure, CIOS helps the team roll Flock back to its starting state. CIOS notifies the team (e.g., by calling) when it's time to initiate an automated rollback. The team must approve this rollback plan, after which CIOS will implement it.

[0082] In the strategic release phase, here's an example of how a team might execute a "new CM" for tomorrow's version: When generating a plan, the team might request CIOS to target plans for specific resources in several ways: topologically (e.g., realm, region, ad, etc.), by resource type (e.g., "metrics configuration only" or "deployment of deployment orchestration services only"), or in combination of the above (e.g., isolated). The team approves the strategic release in the same way as a global release. CIOS orchestrates the strategic release in the same way. If the team needs to deploy a strategic release despite a valid global release being available, CIOS will stop the execution of the global release in the target region before starting the execution of the strategic release.

[0083] In some cases, the state of a declarative provisioner (for example, typically a file) is a trusted record of the set of resources managed by the declarative provisioner. This includes a mapping between the logical identifier of each resource from the configuration file and the actual identifier of the resource. When the declarative provisioner creates a resource, under certain types of failures, the actual identifier may no longer be recorded in the state. This happens when the actual identifier no longer belongs to the declarative provisioner. These may be called "orphaned resources."

[0084] For most resources, orphanedness represents garbage. A declarative provisioner will launch an instance that it had forgotten about (for example), but instead of it being executed the next time, the declarative provisioner will launch a different instance. For resources with uniqueness constraints or client-provided identifiers, orphanedness causes the declarative provisioner to delay progress. This prevents further progress. For example, if a declarative provisioner creates a user named "nglass" and it becomes orphaned due to a failure, the next execution of the declarative provisioner will attempt to create "nglass," but the creation will fail because a user with that username already exists. In some cases, orphaning is only a problem when adding new resources to the state. In some cases, the update behavior of a declarative provisioner may recover naturally from failures in logging updates and deletions.

[0085] CIOS must be robust in the event of a downstream service failure or a failure of CIOS itself. Since CIOS can apply changes using declarative provisioners, it can be said to be robust in executing declarative provisioners and maintaining their state. Declarative provisioner providers perform "small" retries sufficient to avoid outages lasting several minutes. For example, a cloud provider might retry for up to 30 minutes. If a downstream system outage lasts longer than 30 minutes, the declarative provisioner will fail. When a declarative provisioner fails, it records all changes made during normal operation in its state before terminating. For retries, CIOS must re-execute the declarative provisioner. Re-executing the declarative provisioner also allows CIOS to be retried if CIOS itself fails. In some cases, CIOS can loop through the following actions:

[0086] ● Update - Declarative provisioners call the GET API to retrieve the most recent snapshot of all resources described in the state.

[0087] ● Plan - Taking into account the most recently updated current state, the declarative provisioner generates a plan (a series of specific API calls) that can achieve the desired state.

[0088] ●Application - A declarative provisioner executes a series of steps included in the plan. CIOS can always perform all three of these steps when running a declarative provisioner. Update operations help recover from updates or deletions that were not recorded. CIOS inspects the results of the plan execution and compares them to the approved release plan. If the newly generated plan includes behavior that was not included in the approved release plan, CIOS may fail and notify the service team.

[0089] Figure 3 shows a DAG (Directed Acyclic Graph) 200 to illustrate an exemplary Flock 302. It can describe the code / configuration progression from check-in to production for a single Flock configuration in CIOS, from the initial test environment deployment to the final production environment deployment. Internally, CIOS calls each element in the progression of ETs (Execution Targets). This is present throughout the internal API but does not leak into the Flock configuration. CIOS executes the ETs based on the DAG 200 defined in the Flock configuration. Each ET (e.g., ET-1, ET-2, ET-3, ET-4, ET-5, ET-6, and ET-7) is generally a copy of one of the services described by the Flock configuration.

[0090] Figure 4 is a DAG400 illustrating the exemplary Flock402. In the Flock configuration, CIOS is quite dogmatic about how teams represent this progress. Teams must model their progress using cloud infrastructure tenancies and regions. Teams should not model their progress using realms. CIOS allows teams to use many tenancies within a realm and many regions within a tenancy. However, CIOS prohibits teams from using the same region twice within a single tenancy (but each time being different). (The same region can be used twice within a single realm in a tenancy.) DAG400 shows a variation of DAG300 in Figure 3, represented using tenancies and regions. This embodiment is an embodiment for an overlay service where the pre-production environment ET is in the production environment region. Service Enclave Service Release 1 has an unstable tenancy and a stable tenancy. In DAG400, IAD is the regional airport code for Dulles Airport in Washington, D.C., YYZ is the regional airport code for Toronto in Ontario, PHX, LHR, and FRA are the regional airport codes for Phoenix, London, and Frankfurt, respectively, and LUF and LFI are the airport codes for two different air force bases.

[0091] In one embodiment, CIOS and / or other technologies described herein are improved versions of Terraform (a declarative provisioning tool), Tanden (a code generation tool), and ODO (Oracle Deployment Orchestrator). In addition, in some examples, CIOS and / or other technologies described herein can be implemented using at least some of the Terraform, Tanden, and ODO tools.

[0092] Figure 5 shows a user interface (UI) 500 presenting information about a release including multiple phases and multiple execution targets, according to at least one embodiment. The UI 500 may include a phase area 502 and an execution target area 504. The phase area 502 may show an ordered list of phases, such as phase 506 (e.g., "R_n"), phase 507 (e.g., "R_s"), phase 508 (e.g., "R_e"), and phase 509 (e.g., "R_w"). In some embodiments, the phases may be displayed in a left-to-right order, where the leftmost phase completes before the phase to its right, and so on. In some embodiments, the ordered list of phases may be stored in any suitable data structure, such as a linked list. An exemplary linked list is described in further detail in Figure 7.

[0093] Each phase may be associated with multiple tasks (for example, multiple tasks including deploying one or more infrastructure resources for one or more execution targets). The list of phases shown in UI 500 includes four phases, but the phase area 502 may include any appropriate number of phases for deploying infrastructure resources in one or more execution targets. In some embodiments, the ordered list of phases presented within the phase area 502 may be horizontally scrollable. The phase area 502 may show the number of phases 510, the status 511, the number of completed execution targets and / or the number of execution targets 513, and the Flock configuration identifier 514. The number of phases 510 may indicate the number of phases included in the linked list, and the status 511 may indicate the status of the release. As shown in Figure 5, the status 511 may indicate "Applying", but the status 511 may be any appropriate status indicator (e.g., "Before Start", "Completed", "Failed", etc.). As shown in Figure 5, the number of completed execution targets and / or the number of execution targets 513 indicates that, for the execution targets, 24 deployments have been completed out of a total of 57 execution targets. The number of completed execution targets and / or the number of execution targets 512 may be presented on the UI 500 in any appropriate way. The Flock configuration identifier 514 is presented on the UI 500 as "13380fb2832", but the Flock configuration 514 may be presented on the UI 500 in any appropriate way to convey the unique identifier of the Flock configuration file corresponding to the release shown in Figure 5.

[0094] Each sub-area corresponding to a phase has a phase identifier (for example, phase identifier 51 6) The system may include a total execution target indicator (e.g., total execution target indicator 518), timestamp information (e.g., timestamp information 520), an execution target tracker area (e.g., execution target tracker area 522), and other suitable information relating to the phase. For example, identifier 516 (e.g., "R_s") may be any suitable alphanumeric string to uniquely identify the phase. In some embodiments, the total execution target indicator 518 may be presented adjacent to identifier 516. The total execution target indicator 518 indicates the total number of execution targets associated with a given phase (e.g., 14).

[0095] The timestamp information 520 may include a start time and / or end time, respectively, associated with the time the phase started and / or completed. In some embodiments, the timestamp information 520 may include any appropriate indicators to show the status of the corresponding phase (e.g., "Before Start", "Failed", "Completed", etc.). In some embodiments, the phase area 502 may display any appropriate information about the phase. Information about each phase may be presented similarly, the presentation of phase information may be in a different format, and / or may include more or less information than that shown in Figure 5.

[0096] An execution target tracker area (for example, execution target tracker area 522 in phase 506) may be presented within each phase. The execution target tracker area in each phase may include one or more execution target indicators (for example, execution target indicators 524, 526, and 528). Each execution target indicator may include a number indicating the number of tasks to be performed simultaneously (for example, deployments to the corresponding execution targets). For example, execution target indicator 524 indicates that a deployment to a particular execution target may be performed. Execution target indicator 526, positioned to the right of execution target indicator 524, indicates that after the completion of the first task corresponding to execution target indicator 524, another deployment to a different execution target will be performed. Execution target indicator 528, positioned to the right of execution target indicator 526, indicates that after the completion of the second task corresponding to execution target indicator 524, twelve separate deployments to twelve different execution targets will be performed. The combination of execution target indicators 524, 526, and 528 collectively represents a collapsed directed acyclic graph (for example, the DAG900 described later in Figure 9).

[0097] In some embodiments, execution target indicators 524-528 may correspond to a data structure configured to hold a list of execution targets associated with the phase and the order in which tasks (e.g., deployments to each execution target) are performed. Details of an exemplary data structure for holding this list and order will be described later in relation to Figure 9. Each execution target indicator may include a ring (e.g., ring 530). The ring may be divided into any appropriate number of parts corresponding to the number of execution targets associated with the execution target indicator. In the illustrated example, ring 530 may be divided into 12 equal parts. Each part of ring 530 may correspond to a specific execution target and may be colored in a color corresponding to the status of the tasks corresponding to that specific execution target. As an example, ring 530 may indicate that 9 deployments to execution targets have been completed (e.g., by 9 green parts). The remaining 3 parts of ring 530 (e.g., colored white) may indicate that 3 tasks corresponding to 3 execution targets have not yet been started. As another example, the remaining 3 parts of ring 530 may be colored in a different color (e.g., yellow), and This indicates that the corresponding task has started but is not yet complete. Therefore, by looking at ring 530, the user can quickly and visually identify the current status of the task associated with the execution target indicator 528. By looking at the execution target indicators in the execution target tracker area 522, the user can see that deployments to 11 execution targets are complete and 3 are in progress. In some embodiments, each execution target indicator may correspond to a node in a DAG (directed acyclic graph) (e.g., DAG900 in Figure 9) associated with a given phase (e.g., "R_s").

[0098] The execution target area 504 may include an execution target column 532, a progress column 534, and an operation column 536. The execution target column 532 may include a list of targets (e.g., execution targets) corresponding to tasks (e.g., deployments) executed in relation to a given phase (e.g., phase 506 as shown in the figure). In some embodiments, the execution target column 532 may be stored chronologically or in any other suitable way for storing task executions. The progress column 534 may include a list of progress indicators showing the status (e.g., "successfully completed," "before starting," "in progress," etc.) corresponding to each execution target. The operation column 536 may include a list of operations performed on the execution targets shown in the execution target column 532. For example, the list of operations included in the operation column 536 may include CRUD (create, read, update, and delete) operations, etc.

[0099] Figure 6 is an exemplary code segment 600 for defining a list and order of phases, relating to at least one embodiment. As shown in Figure 6, the code segment 600 defines four phases. Each phase is defined as a resource of type “phase” and is assigned an identifier (e.g., “R_n”, “R_s”, “R_e”, and “R_n”). Resources 602, 604, 606, and 608 correspond to their respective phases, as shown in code segment 600. Each phase may contain one or more variables, one of which may contain an indicator to indicate the execution order of each phase. In some embodiments, the indicator may indicate that there is a dependency on and / or no dependency on one or more other phases. For example, indicator 618 (e.g., predecessors=[] on line 5) may indicate no relationship with other phases. This may be interpreted by the system as defining that the first phase is executed. Indicator 620 (for example, predecessors=[phase.R_n.variable1]) can be used to indicate a dependency on another phase (for example, phase "R_n") by including the assignment of values ​​corresponding to variables related to that other phase (for example, phase "R_n"). Similarly, indicator 622 may indicate a dependency on another phase (for example, "R_s") (for example, by including the assignment of values ​​corresponding to variables related to "R_s"), and indicator 624 may indicate a dependency on yet another phase (phase "R_e") (by including the assignment of values ​​corresponding to variables related to "R_e").

[0100] In some embodiments, the configuration file corresponding to the release may include a code segment 600. The configuration file may be parsed / traversed one or more times as part of the execution of a preprocessing procedure. In one of these traversals, the code segment 600 may be analyzed to generate a data structure that can hold the IDs and order of executions corresponding to one or more phases. Figure 7 shows an example of a data structure that may be used to hold this information (for example, a linked list).

[0101] Use indicators 618-624 to construct the specific order in which the phases will be executed. This is possible. For example, as shown in Figure 6, phase "R_n" is executed first, followed by phases "R_s", "R_e", and "R_w". You will see that dependencies can indicate one or more other phases that are completed before the task associated with a given phase begins. In Figure 6, four phases are defined, but you will see that any appropriate number of phases can be defined in the same way.

[0102] Figure 7 shows an exemplary data structure (e.g., linked list 700) generated by CIOS (Cloud Infrastructure Orchestration Service) to hold a list and order related to one or more phases. Linked list 700 can be generated to identify the phases and execution order defined in code segment 600 in Figure 6. As shown in Figure 7, linked list 700 contains four nodes 702, 704, 706, and 708. Nodes 702, 704, 706, and 708 may each correspond to one of the four phases defined in code segment 600. Each node in the linked list may correspond to a data object configured to store any appropriate information corresponding to a given phase. For example, a given node may store any appropriate number of variables, identifiers, data structures, pointers, references, etc., corresponding to a particular phase. As a non-restrictive example, node 702 corresponding to phase "R_n" may store three variables defined in code segment 600 as corresponding to phase "R_n". Each node in the linked list 700 may contain any appropriate number of variables corresponding to a given phase.

[0103] Each node in the linked list may contain a pointer / reference to another node in the linked list (if there are multiple phases). For example, node 702 may contain a reference to node 704. Node 704 may contain a reference to node 706. Node 706 may contain a reference to node 708. Node 708 may be indicated (for example, by a null pointer) as the end node of linked list 700. In some embodiments, these pointers / references may be identified based on indicators 618-624 of code segment 600.

[0104] In some embodiments, at runtime, CIOS may identify the order in which particular phases are performed based at least in part on traversing a linked list 700 starting from node 702 (e.g., the starting node). Any suitable combination of data stored within each corresponding node may be used to perform these phases. Once the operation corresponding to a given phase is completed, CIOS may traverse to the next phase and repeat this process any appropriate number of times until the operation corresponding to the end node of the linked list 700 (e.g., node 708) is completed. In some embodiments, if the operation at a given node is unsuccessful (e.g., an error occurs), CIOS may not traverse to the next node, but instead may abort the deployment and return a notification to alert the user of this situation.

[0105] Each node in the linked list 700 may correspond to a data structure configured to identify and hold the execution order corresponding to one or more execution targets. Figures 8 and 9 describe the definition and use of such data structures in more detail. It will be seen that the linked list 700 may provide the information that the UI 500 in Figure 5 uses to present any appropriate information related to the phase (e.g., identifier 516, timestamp information 520, execution target 518, etc.).

[0106] Figure 8 is an exemplary code segment 800 for defining a list and order of execution targets (e.g., execution targets corresponding to a particular phase) according to at least one embodiment. As shown in Figure 8, the code segment 800 defines four execution targets. Each execution target is a resource of type “execution target”. They are defined as execution targets and are assigned identifiers (e.g., "us-la", "us-sj", "us-sf", and "us-sd"). Resources 802-808 each correspond to a unique execution target, as shown in code segment 800. Each execution target may be associated with one or more variables, one of which may include an indicator to indicate the execution order of each phase. In some embodiments, the indicator may indicate that there is one or more dependencies on one or more other execution targets and / or no dependencies. For example, indicator 818 (e.g., predecessors=[] on line 5) may indicate no relationship with other phases. This may be interpreted by the system as defining that the first execution target is executed. Indicator 620 (e.g., predecessors=[execution_target.us-la.variable1]) may be used to indicate a dependency on another execution (e.g., phase "us-la") by including an assignment of a value corresponding to a variable related to another execution target (e.g., execution target "us-la"). Similarly, indicator 822 may indicate a dependency on another phase (for example, the execution target "us-sj" by including the assignment of values ​​corresponding to variables related to the execution target "us-sj"), and indicator 624 may indicate a dependency on yet another phase (for example, the execution target "us-sf" by including the assignment of values ​​corresponding to variables related to the execution target "us-sf").

[0107] In some embodiments, the code segment 800 may be included in a configuration file corresponding to a release (e.g., the same or different configuration file containing the code segment 800). The configuration file may be parsed / traversed one or more times as part of the execution of a preprocessing procedure. In one of these traversals, the code segment 800 may be analyzed to generate a data structure that can hold the IDs and order of execution targets corresponding to a given phase. Figure 9 shows an example of a data structure that may be used to hold this information (e.g., a directed acyclic graph (DAG)).

[0108] Indicators 818-824 can be used to construct a specific order in which execution targets are executed. For example, as shown in Figure 8, tasks related to execution target "us-la" are executed first, followed by tasks related to execution target "us-sj", then tasks related to execution target "us-sf", and finally tasks related to execution target "us-sd". It can be seen that dependencies can indicate one or more other execution targets whose corresponding tasks are completed before tasks related to a given execution target begin. In Figure 8, four execution targets are defined, but it can be seen that any appropriate number of phases can be defined in the same way. In some embodiments, execution targets may share common dependencies (e.g., the same predecessor definitions). Common dependencies can be used to indicate that tasks related to execution targets that share these common dependencies may be executed simultaneously.

[0109] Figure 9 shows an exemplary data structure (e.g., a Directed Acyclic Graph (DAG) 900) that CIOS (Cloud Infrastructure Orchestration Service) may generate to hold a list and order relating to one or more execution targets related to a phase, according to at least one embodiment. DAG 900 may be one of several DAGs generated in response to the CIOS parsing of a release-related configuration file one or more times. Each node of DAG 900 may correspond to one execution target. As shown in Figure 9, DAG 900 includes six nodes (e.g., nodes 902, 904, 906, 908, 910, and 912). Each of these nodes may correspond to one of six execution targets defined in a manner similar to that described in code segment 800 of Figure 8. Each node of DAG 900 corresponds to an execution target This can correspond to a data object configured to store appropriate information. For example, a given node may store any number of appropriate variables, identifiers, data structures, pointers, references, etc., corresponding to a specific execution target.

[0110] Each node in DAG900 may contain a pointer / reference to one or more nodes in DAG90. For example, node 902 may contain a reference to node 904. Node 904 may contain references to nodes 906-910. Nodes 906-910 may each contain a reference to node 912. Node 912 may be indicated (for example, by a null pointer) as an end node of DAG900. In some embodiments, these pointers / references may be identified based on indicators similar to indicators 818-824 described above with respect to Figure 8. In some embodiments, nodes 906, 908, and 910 may contain a common dependency on node 904 so that tasks related to nodes 906-910 can be executed at least partially simultaneously. In some embodiments, node 912 may correspond to an execution target that depends on nodes 906-910. Therefore, tasks related to the execution target corresponding to node 912 can only be executed after all tasks related to the execution targets corresponding to nodes 906-910 have been completed.

[0111] In some embodiments, DAG900 may be associated with a node in a linked list 700 as described above with respect to Figure 7. That is, one or more execution targets (for example, identified and represented by a node in DAG900) may be associated with a particular phase (for example, a node) in linked list 700. In some embodiments, a node in linked list 700 may contain a reference to DAG900, or one or more nodes in DAG900 may contain a reference to a node in linked list 700.

[0112] In some embodiments, CIOS may traverse configuration files (e.g., code segment 800 in Figure 8) and generate a DAG900 from this traverse. The generation of the DAG900 may be completed as part of a preprocessing procedure performed before or at runtime. Once the operation corresponding to a given execution target is completed, CIOS may traverse to the next execution target(s), and repeat this process any appropriate number of times until the operation corresponding to one or more end nodes of the DAG900 (e.g., node 912) is completed. In some embodiments, if the operation corresponding to a given node is not performed successfully (e.g., an error occurs), CIOS may not traverse to the next node, but instead may return a notification to alert the user to this situation.

[0113] Each node in DAG900 may correspond to a data structure configured to identify and hold the execution order corresponding to one or more resources (e.g., services, software modules, etc.). Figures 10-12 illustrate the definition and use of such a data structure (e.g., a DAG of capabilities) in more detail. As an example, each node in DAG900 may correspond to another DAG (e.g., DAG1200 in Figure 12, which shows a list and order of resources and / or capabilities and identifies the order of task execution). It will be seen that DAG900 may provide the information that UI500 in Figure 5 uses to present any appropriate information related to the execution target indicators 524-528. Thus, the information shown in the execution target tracking area 522 may represent a simplified version of DAG900 (a DAG corresponding to a given phase, such as phase "R_s"). In a simplified version of the DAG, the concurrently running nodes of DAG900 can be condensed into a single node (see, for example, execution target indicator 528, which shows that 12 nodes of the DAG are condensed into a single node). In some embodiments, CIOS uses at least part of traversing DAG900 to access infrastructure resources and / or release software. The software artifact can be deployed. The specific tasks and their specific order are identified with reference to Figures 10-12.

[0114] Figure 10 is an exemplary code segment 1000 for constructing explicit and implicit dependencies between resources of an execution target, relating to at least one embodiment. Code segment 1000 includes two modules 1002 and 1004 and a resource 1006, as shown in Figure 10. Modules 1002 and 1004 each include names 1008 and 1010, indicated as "apps_example1" and "apps_example2," respectively. Modules may include names of any appropriate length, containing any appropriate alphanumeric characters(s). Modules 1002 and 1004 may define the applications / services that the user wants to launch or provision. Modules 1002 and 1004 can be used to deploy applications to available domain 1 and available domain 2, respectively. Resource 1006 may include a multi-parameter list, including a resource type 1012, indicated as "type" in Figure 10, and a resource name 1014, indicated as "executor" in Figure 10. Resource type 1012 may be any type suitable for deployment, and resource name 1014 may be any name.

[0115] Resource 1006 may be a capability and may include implicit dependencies, explicit dependencies, or both. As shown in code segment 1000, resource 1006 attempts to assign the variable "variable1" to a value equal to "module.apps_example1.variable1" and a value accessible through a module called apps_example1 (e.g., module 1002). This is intended to demonstrate an implicit dependency formed between resource 1006 and module 1002. The formed implicit dependency may prevent resource 1006 from operating before module 1002 has completed its deployment. The process responsible for invoking resource 1006 may receive notification that module 1002 has completed its deployment. This notification may be sent by a scheduler (e.g., scheduler 206 in Figure 2) and may be received by the process responsible for invoking resource 1006. Since resource 1006, shown in Figure 10, does not directly define the dependency between module 1002 and resource 1006, the implicit dependency that is formed is considered implicit.

[0116] In contrast, resource 1006 includes an explicit dependency, as shown on line 29 of Figure 10, which explicitly defines the dependency between resource 1006 and the app_example2 module. As shown on line 29 of code segment 1000, resource 1006 includes "depends_on=apps_example2.variable2". Based on the code on line 29, an explicit dependency may be formed, which may prevent resource 1006 from being deployed until apps_example2 is successfully deployed. Once apps_example2 is successfully deployed, a notification may be sent by the scheduler, which may be received by the process responsible for deploying resource 1006. Note that while code segment 1000 in Figure 10 includes one resource 1006 containing one implicit dependency and one explicit dependency, a person skilled in the art will see that any combination of resource 1006, the implicit dependency, and the explicit dependency may be used to achieve the goals of a CIOS user.

[0117] CIOS (or a declarative infrastructure provisioner, such as the CIOS declarative provisioning tool mentioned above) can be used to analyze the configuration file containing code segment 1000. Through this analysis, CIOS (or a declarative provisioner) The provisioning provisioner may generate a directed acyclic graph (DAG) for each resource, module, and / or capability, which compiles and defines an ordered list of dependencies on other resources, modules, and / or capabilities. While attempting to deploy a resource, the CIOS may traverse the DAG to identify when a resource depends on another resource, module, and / or capability. The DAG for each resource may specify implicit dependencies, explicit dependencies, or a combination thereof, and can be used to start or deploy the corresponding resource with the CIOS.

[0118] Figure 11 is an exemplary code segment 1100 for constructing explicit and implicit dependencies between resources of an execution target, according to at least one embodiment. The code segment 1100 shown in Figure 11 includes four resources 1102, 1104, 1106, and 1108. Each of the resources 1102, 1104, 1106, and 1108 may correspond to a capability and may include implicit dependencies, explicit dependencies, or a combination thereof.

[0119] Resource 1102, shown in Figure 11, includes a name 1110 labeled "object_storage", a type 1112 labeled "type1" (for example, indicating that the resource is a capability), and several variables (for example, variables 1-3; however, more or fewer variables may be used). Resource 1104, shown in Figure 11, includes a name 1116 labeled "worker" and an explicit dependency 1118 labeled "depends_on=type1.object_storage". The parameter list for resource 1102 includes an identifier 1115. Identifier 1115 may be used to refer to that resource (or name 1110 may be used similarly). Statement 1118 forms an explicit dependency to resource 1102 by referencing type1.object_storage. This explicit dependency means that resource 1104 does not need to be deployed until resource 1102 has completed its deployment. Resource 1106, shown in Figure 11, includes an identifier 1120 labeled "peacock", a type (e.g., "type1" indicating capability), and several variables (e.g., variable1 and variable2). Resource 1108, also shown in Figure 11, includes a type (e.g., "type4"), a name 1124 labeled "LB", and a statement 1126 ("count=type1.peacock.exists."). The use of statement 1126 can create an implicit dependency on resource 1106. Resource 1108 does not use an explicit dependency construct (e.g., "depends_on"), but an implicit dependency exists because it attempts to assign a value to the variable "count" equal to whether the capability "peacock" exists (determined from the statement type1.peacock.exists). Therefore, resource 1108 does not need to be deployed until resource 1106 "peacock" is deployed, as there is an assignment attempted on line 18.The code segment 1100 in Figure 11 includes four resources 1102, 1104, 1106, and 1108, each containing one implicit dependency and one explicit dependency. However, a person skilled in the art will see that any combination of resources, implicit dependencies, and explicit dependencies can be used to achieve the goals of the CIOS user.

[0120] Using CIOS (or a declarative infrastructure provisioner such as the CIOS declarative provisioning tool mentioned above), a configuration file containing code segment 1100 can be parsed. Through this parsing, CIOS (or a declarative provisioning provisioner) can generate a directed acyclic graph (DAG) for each resource, module, and / or capability. Each resource, module, and / or capability compiles and defines an ordered list of dependencies on other resources, modules, and / or capabilities. While attempting to deploy a resource, CIOS traverses the DAG to determine the resource It is possible to identify when a resource depends on another resource, module, and / or the capabilities of another resource. A per-resource DAG may specify implicit dependencies, explicit dependencies, or a combination thereof, and can be used to start or deploy the corresponding resource with CIOS.

[0121] Figure 12 shows an exemplary Directed Acyclic Graph (DAG) 1200 corresponding to a resource (e.g., resource A) in a cloud computing system, according to at least one embodiment. As shown in the figure, the DAG 1200 can be a finite directed graph containing any appropriate number of nodes (e.g., six nodes in Figure 12) and edges (e.g., seven edges in Figure 12). Each edge is directed from one node to another, as shown in Figure 12. The nodes and edges may be arranged so as not to form directed cycles. That is, the DAG 1200 is defined such that, starting from any node, no consistently directed sequence of edges eventually cycles back to that same node. The last node (e.g., node "6") may point to a null value or may indicate the endpoint of the DAG.

[0122] While DAG1200 shows six nodes and seven edges, a DAG may contain any appropriate number of nodes and directed edges. In some embodiments, each node corresponds to a set of operations (for example, operations to perform tasks such as deploying and / or starting a resource such as resource A), or a set of capabilities on which the next node depends, consisting of multiple operations. The directed edges of each DAG define the order in which these operations are performed and / or the dependencies between a subset of operations associated with a node and a subset of capabilities associated with the node immediately preceding that node.

[0123] As a simple example, nodes 1, 2, 5, and 6 of DAG1200 represent nodes corresponding to four distinct sets of operations. Based on the edges shown in Figure 1200, the operations of each node are performed in the order corresponding to nodes 1, 2, 5, and 6. Nodes 3 and 4 represent multiple nodes that individually match one or more dependencies. For example, node 3 may correspond to a dependency on a capability related to a different resource (e.g., resource B) for the operation corresponding to node 5. Similarly, node 4 may correspond to a dependency on a capability related to a different resource (e.g., resource C) for the operation corresponding to node 5. In some embodiments, different capability nodes may be used for different resources (e.g., nodes that identify dependencies on one / more capabilities of a particular resource), and all dependencies may be specified using a single node, regardless of how many resources a dependency refers to. Thus, in some embodiments, a dependency corresponding to resource B (e.g., identified in node 3) and a dependency corresponding to resource C (e.g., identified in node 4) may be combined into a single node.

[0124] DAG1200 is traversed in a manner that illustrates in more detail with respect to Figures 10-12, and can orchestrate the execution of operations to launch and / or deploy resources in a cloud computing environment, in relation to one or more dependencies on the capabilities of other resources (or the other resources themselves).

[0125] Figure 13 is a flowchart illustrating an exemplary process 1300 for orchestrating the execution of a task (e.g., deploying resources) that involves a dependency on at least one capability (e.g., the capability of different resources), relating to at least one embodiment. As shown in Figure 13, the process flow 1300 includes a scheduler 1302 (e.g., scheduler 206 in Figure 2), a worker 1304 (e.g., worker 210 in Figure 2), and an IP process 1306 (e.g., CIOS container 212 in Figure 2).

[0126] In 1308, the scheduler 1302 may receive a task to deploy infrastructure resources in a region, and the scheduler 1302 may send data related to that task to the worker 1304. In some embodiments, the scheduler 1302 may instantiate the worker 1304 to handle the deployment of resources (e.g., services).

[0127] In 1310, worker 1304 may instantiate IP process 1306. IP process 1306 may be configured to run an instance of a declarative infrastructure provisioner (for example, the declarative provisioning tool Terraform mentioned above).

[0128] In 1312, the IP process 1306 may parse configuration files related to the deployment (for example, configuration files containing code segments 1000 and / or 1100 in Figures 10 and 11) to generate a directed acyclic graph (DAG) for a particular resource. Through the parsing of the configuration, the IP process 1306 (declarative infrastructure provisioner) may identify any appropriate number of implicit and / or explicit dependencies of other resources to their capabilities. Once identified, the IP process 1306 constructs a DAG (for example, according to the implicit and / or explicit dependencies identified during the parsing) that specifies a task for starting and / or deploying the resource, having one or more nodes corresponding to the capabilities on which the resource depends.

[0129] In 1314, the IP process 1306 may initiate a traverse of the DAG, and upon reaching various nodes of the DAG, it may perform at least part of the deployment and / or initiation of a particular resource. Any appropriate operation may be performed according to at least one node of the DAG to make part of the functionality corresponding to the resource available. Multiple parts of the functionality corresponding to the resource may become available. In some embodiments, the IP process 1306 may send a notification (not shown) to the scheduler 1302 indicating that one or more capabilities of a resource are currently available. At least one node of the DAG may correspond to the capabilities of one or more other resources. Upon reaching these types of nodes, the IP process 1306 may check whether the capabilities are available. If they are available, the IP process 1306 may proceed with the traverse of the DAG.

[0130] In 1316, the IP process 1306 may reach a node in the DAG that corresponds to one or more capabilities of one or more other resources. In some embodiments, the IP process 1306 may determine that at least one capability associated with the node is not yet available.

[0131] In response to determining in 1320 that at least one capability associated with a node is unavailable, the IP process 1306 may send data to the scheduler 1302 indicating that one or more capabilities on which the resource corresponding to the IP process 1306 depends have been determined to be unavailable.

[0132] In 1322, the IP process 1306 may terminate after storing state information indicating which operations and / or which nodes of the DAG have been completed and / or which specific node of the DAG was last accessed by the IP process 1306. The IP process 1306 terminates, is forcibly terminated, is temporarily stopped, or execution is aborted.

[0133] In 1324, the scheduler 1302 determines that a specific resource should be restarted. It may store information indicating that it was listening for one or more specific capabilities required for the purpose of calling and / or deploying.

[0134] At 1326, the scheduler 1302 may receive one or more notifications indicating that one or more capabilities on which the resource was listening have become available. In some embodiments, the scheduler 1302 may receive various notifications from other IP processes about various capabilities of the corresponding resource as they become available. The scheduler 1302 may maintain one or more records of these available capabilities and / or one or more records of various capabilities on which the resource is currently listening. From these one or more records, the scheduler 1302 may identify that a particular one or more capabilities on which the resource corresponding to IP process 1306 was listening have become available. Thus, the scheduler 1302 may proceed to 1328.

[0135] In step 1328, in response to determining that the capability on which the resource corresponding to the IP process 1306 depends has become available, the scheduler 1302 may return to step 1308. In step 1308, the scheduler 1302 sends data related to the original task (e.g., deploying the resource) to worker 1304. In some embodiments, the scheduler 1302 may instantiate a new worker, or it may continue to use the previous worker 1304 (illustrated) to handle the task related to the resource. Worker 1304 may instantiate an IP process (not shown) which may be configured to execute and parse a configuration file to generate a DAG about the resource. The IP process may access stored state information to identify the last accessed node in the DAG (e.g., a node corresponding to one or more capabilities on which the resource was waiting for a response). Once one or more capabilities are available, the IP process may proceed traversing the DAG in a similar manner to that described above, performing operations at each node, executing parts of tasks, or checking the capabilities on which subsequent parts of tasks depend, until the operations at the end nodes of the DAG are complete.

[0136] For all resources of a task, a process similar to the one described above may be executed. For example, when deploying a system with multiple resources (e.g., multiple services), process 1300 may be executed on behalf of each resource to deploy each resource of the system.

[0137] Figure 14 shows an exemplary process flow 1400 for (for example, CIOS) to perform a release, relating to at least one embodiment. In event number 1, scheduler 1402 (for example, scheduler 206 in Figure 2) may send a task to worker 1404 (for example, worker 210 in Figure 2). This task may include deploying a computing system or a subset thereof, such as deploying infrastructure resources to a set of execution targets. The task may require traversing linked lists (for example, the linked list 700 example in Figure 7), DAGs (for example, DAGs 900 and 1200 in Figures 9 and 12, respectively), combinations thereof, or other suitable tasks for deploying the computing system. Worker 1404 may receive this task from scheduler 206. Worker 1404 may be one worker node in a fleet of worker nodes. A fleet of worker nodes may include any appropriate number of worker nodes for deploying the computing system. The scheduler 1402 may select worker 1404 based at least partly on its capacity. For example, if worker 1404 has the most computing power in the fleet of worker nodes, the scheduler 1402 may choose to assign a task to worker 1404.

[0138] In event number 2, worker 1404 may parse / traverse configuration file 1406 one or more times. Configuration file 1406 may contain instructions for deploying the computing system, and by performing one or more parses, resources or other capabilities that are to be started or otherwise deployed for deploying the computing system may be identified. As a non-limiting example, configuration file 1406 may contain code segments 600, 800, 1000, and 1100 in Figures 6, 8, 10, and 11, respectively.

[0139] In event number 3, information from configuration file 1406 may be sent to IP process 1408 (for example, IP process 212 in Figure 2). IP process 1408 may receive information from configuration file 1406 based on one or more parses / traverses performed by worker 1404. This information may include a set of capabilities, execution targets, or other suitable resources for deploying a computing system.

[0140] In event number 4, in response to receiving this information from configuration file 1406, IP process 1408 may determine the order in which capabilities, or other suitable resources for deploying the computing system, will be deployed. IP process 1408 may generate a linked list of phases 1410 (for example, linked list 700 in Figure 7), a DAG of execution targets 1412 (for example, a DAG of 900 in Figure 9), a DAG of capabilities 1414 (for example, a DAG of 1200 in Figure 12), or other suitable lists, graphs, or data structures. The linked list of phases 1410, the DAG of execution targets 1412, and the DAG of capabilities 1414 (collectively referred to as the “Release Data Structure”) may be generated in any suitable order.

[0141] The release data structure can be used to identify and determine the order in which to perform the tasks of a release. For example, each node in the linked list of phases 1410 corresponds to a separate instance of the execution target's DAG (e.g., the example execution target's DAG 1412), where each node of the execution target's DAG corresponds to a capability's DAG (e.g., the example capability's DAG 1414). The IP process 1408 may identify the corresponding execution target's DAG, starting from the first node of the linked list 1400. It may also identify the corresponding capability's DAG by utilizing the first node of the execution target's DAG. Following the capability's DAG, the IP process 1408 may perform tasks associated with that capability's DAG, and upon completion, it may traverse the next nodes of the execution target's DAG to identify the next corresponding capability's DAG. It may also traverse each node of the execution target's DAG, and upon completion of the tasks corresponding to these nodes, the IP process 1408 may traverse the next nodes of the linked list 1400 to identify the next phase. This process can be repeated any appropriate number of times until all tasks related to each execution target associated with the final phase of the release are completed.

[0142] For example, in event number 5, the first node of the linked phase list 1410 is reached. IP process 1408 identifies the execution target DAG corresponding to the first node.

[0143] In event number 6, the first node of the execution target DAG1412 is reached. The capable DAG1414 can be identified at least in part on the basis that it is related to the first node of the execution target DAG1412.

[0144] In event number 7, tasks are performed on a given execution target, at least in part, based on traversing the DAG 1414 of the capabilities. Once these tasks are completed, IP process 1408 traverses the next node of the execution target's DAG and the corresponding capabilities. The IP process 1408 may determine the DAG of capabilities and execute tasks according to its traverse of the DAG of capabilities. This process may continue until the task related to the last node of the execution target DAG 1412 is executed. The IP process 1408 may then traverse the next node of the linked list of phases 1410. Operations 5-7 of events may be repeated any appropriate number of times until all tasks related to all execution targets related to the last node of the linked list of phases 1410 are executed. Once the tasks, execution targets, and / or phases are complete, the IP process 1408 may update or have UI 500 in Figure 5 updated to display the current state of the tasks, execution targets, and / or phases.

[0145] In event number 8, IP process 1408 sends a signal to scheduler 1402 indicating that the release crossover is complete. Scheduler 1402 may receive a signal from IP process 1408 and may broadcast a notification indicating that the computing system is ready for use.

[0146] Figure 15 is a flowchart of process 1500 for deploying a computing system using a DAG in CIOS, according to at least one embodiment. In block 1502, CIOS may execute an instruction to parse configuration data related to the deployment of the computing system one or more times. The configuration data may be contained in a configuration file, and when parsing the configuration data one or more times, CIOS may determine a set of tasks to be performed to deploy the computing system. Tasks included in this set of tasks may include deploying infrastructure resources at the execution target, or other suitable tasks for deploying the computing system.

[0147] In block 1504, CIOS generates a first DAG (for example, DAG1200 in Figure 12) for deploying resources based on a configuration file. The first DAG may be a DAG of capabilities, which may outline a specific order in which capabilities are started or deployed. The first DAG may contain any appropriate number of tasks for deploying capabilities and any appropriate number of dependencies between the capabilities being deployed.

[0148] In block 1506, CIOS generates a second DAG (for example, DAG900 in Figure 9) that defines the dependencies between execution targets for deployment based on the configuration file. The second DAG may also be a DAG of execution targets that specifies the order in which the execution targets are deployed. The second DAG may contain any appropriate number of execution targets for deploying the computing system and any appropriate number of dependencies for deploying the execution targets.

[0149] In block 1508, CIOS generates a linked list (for example, linked list 700 in Figure 7) that specifies the dependencies between phases based on the configuration file. These phases may include a first DAG, a second DAG, or a combination thereof, and the linked list may determine the order in which the phases are executed. The phases in the linked list may be executed sequentially and may not include phases executed in parallel. The linked list may include any appropriate number of phases for deploying the computing system.

[0150] In block 1510, CIOS may deploy the computing system by traversing the first DAG, the second DAG, and the linked list. CIOS may traverse the linked list, and the first and second DAGs may be traversed simultaneously (i.e., the first and second DAGs may be included in the linked list). Successful traversal of the linked list, the first DAG, and the second DAG may result in a successful deployment of the computing system.

[0151] System Example Figures 16 to 18 illustrate exemplary environments for realizing various embodiments of the present disclosure. Figure 16 is a simplified diagram of a distributed system 1600 for realizing an embodiment of the present disclosure. In the illustrated embodiment, the distributed system 1600 comprises one or more client computing devices 1602, 1604, 1606, and 1608. One or more client computing devices 1602, 1604, 1606, and 1608 are configured to run and operate client applications such as web browsers and proprietary clients (e.g., Oracle Forms) on one or more networks 1610. A server 1612 may be connected to the remote client computing devices 1602, 1604, 1606, and 1608 via the network 1610 in a communicative manner.

[0152] In various embodiments, server 1612 may be configured to run one or more services or software applications, such as services and applications that provide identity management services. In certain embodiments, server 1612 may also provide other services or software applications that may include non-virtual and virtual environments. In some embodiments, these services may be provided to users of client computing devices 1602, 1604, 1606, and / or 1608 as web-based services or cloud services, or under a Software as a Service (SaaS) model. Users operating client computing devices 1602, 1604, 1606, and / or 1608 can then use one or more client applications to interact with server 1612 and utilize the services provided by these components.

[0153] In the configuration shown in Figure 16, the software components 1618, 1620, and 1622 of system 1600 are shown as being implemented on server 1612. In other embodiments, one or more components of system 1600 and / or one or more services provided by these components may be implemented by one or more client computing devices 1602, 1604, 1606, and / or 1608. A user operating a client computing device can then use one or more client applications to access the services provided by these components. These components may be implemented as hardware, firmware, software, or a combination thereof. It should be understood that various different system configurations are possible, and these may differ from the distributed system 1600. Therefore, the embodiment shown in Figure 16 is an example of a distributed system for implementing the embodiment and is not intended to be limiting.

[0154] Client computing devices 1602, 1604, 1606, and / or 1608 may include various types of computer systems. For example, a client computing device may include a palm-sized portable device (e.g., iPhone®, mobile phone, iPad®, computing tablet, personal digital assistant (PDA)) or a wearable device (e.g., Google Glass® head-mounted display) running software such as Microsoft Windows® Mobile and / or various mobile operating systems such as iOS, Windows Phone®, Android, BlackBerry 10, Palm OS. The device may support various applications such as various internet-related apps, email, SMS (Short Message Service) applications, and may use various other communication protocols. Devices may include, for example, general-purpose personal computers, including personal computers and / or laptop computers running various versions of Microsoft Windows, Apple Macintosh®, and / or Linux® operating systems. Client computing devices may be workstation computers running various commercially available UNIX® or UNIX-like operating systems, including, for example, various GNU / Linux® operating systems such as Google Chrome OS. Client computing devices may also include electronic devices that can communicate over network(s) 1610, such as thin client computers, internet-enabled gaming systems (e.g., Microsoft Xbox gaming consoles with or without Kinect® gesture input devices), and / or personal messaging devices.

[0155] The distributed system 1600 in Figure 16 is shown with four client computing devices, but any number of client computing devices may be supported. Other devices, such as sensor-equipped devices, may interact with the server 1612.

[0156] The network(s) 1610 in the distributed system 1600 may be any type of network familiar to those skilled in the art, capable of supporting data communication using various available protocols, including but not limited to TCP / IP (Transmission Control Protocol / Internet Protocol), SNA (Systems Network Architecture), IPX (Internet packet exchange), AppleTalk, etc. Just as an example, the network(s) 1610 may be a LAN (Local Area Network), an Ethernet®-based network, a Token Ring, a Wide Area Network, the Internet, a virtual network, a VPN (Virtual Private Network), an intranet, an extranet, a PSTN (Public Switched Telephone Network), an infrared network, a wireless network (for example, a network operating under any of the protocols of the IEEE (Institute of Electrical and Electronics) 1002.16 suite, Bluetooth®, and / or other wireless protocols), and / or any combination thereof, as well as / or other networks.

[0157] Server 1612 may consist of one or more general-purpose computers, dedicated server computers (including, for example, PC (Personal Computer) servers, UNIX servers, midrange servers, mainframe computers, rack-mount servers, etc.), server farms, server clusters, or other appropriate arrangements and / or combinations. Server 1612 may include one or more virtual machines running a virtual operating system, or other computing architectures with virtualization. One or more flexible pools of logical storage can be virtualized to maintain virtual storage for the server. A virtual network can be controlled by Server 1612 using software-defined networking. In various embodiments, Server 1612 may be configured to run one or more services or software applications described in the above disclosure. For example, Server 1612 may correspond to a server for performing the processing described above according to the embodiments of this disclosure.

[0158] Server 1612 uses one of the above operating systems and commercially available Server 1612 may run an operating system that includes any of the server operating systems. Additionally, Server 1612 may run various additional server applications and / or mid-tier applications, including HTTP (Hypertext Transport Protocol) servers, FTP (File Transfer Protocol) servers, CGI (Common Gateway Interface) servers, Java® servers, and database servers. Examples of database servers include, but are not limited to, those commercially available from Oracle, Microsoft, Sybase, IBM (International Business Machines), and others.

[0159] In some implementations, server 1612 may include one or more applications for analyzing and aggregating data feeds and / or event updates received from users of client computing devices 1602, 1604, 1606, and 1608. For example, data feeds and / or event updates may include, but are not limited to, Twitter® feeds, Facebook® updates, or real-time updates received from one or more third-party information sources and continuous data streams, and such one or more third-party information sources and continuous data streams may include real-time events related to sensor data applications, tickers, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, and automobile traffic monitoring. Server 1612 may also include one or more applications for displaying data feeds and / or real-time events via one or more display devices on client computing devices 1602, 1604, 1606, and 1608.

[0160] The distributed system 1600 may also include one or more databases 1614 and 1616. These databases may provide a mechanism for storing information such as user identification information and other information used in embodiments of this disclosure. Databases 1614 and 1616 may reside in various locations. For example, one or more of databases 1614 and 1616 may reside on a non-temporary storage medium local to (and / or located on) server 1612. Alternatively, databases 1614 and 1616 may reside in a location remote from server 1612 and communicate with server 1612 via a network-based or dedicated connection. In one embodiment, databases 1614 and 1616 may reside on a SAN (Storage Area Network). Similarly, any files necessary for performing functions originating from server 1612 may be stored, as appropriate, in a local location on server 1612 and / or in a location remote from server 1612. In one embodiment, databases 1614 and 1616 may include relational databases, such as Oracle-provided databases, which are configured to store, update, and retrieve data in response to commands in SQL format.

[0161] Figure 17 shows an exemplary computer system 1700 that may be used to implement embodiments of the present disclosure. In some embodiments, the computer system 1700 can be used to implement any of the various servers and computer systems described above. As shown in Figure 17, the computer system 1700 comprises various subsystems, including a processing subsystem 1704 that communicates with several peripheral subsystems via a bus subsystem 1702. These peripheral subsystems may include a processing acceleration device 1706, an I / O subsystem 1708, a storage subsystem 1718, and a communication subsystem 1724. The storage subsystem 1718 may include a tangible computer-readable storage medium 1722 and a system memory 1710.

[0162] The bus subsystem 1702 provides a mechanism for various components and subsystems of the computer system 1700 to communicate with each other as intended. Although the bus subsystem 1702 is illustrated as a single bus, other embodiments of the bus subsystem may utilize multiple buses. The bus subsystem 1702 may be one of several types of bus structures, including a memory bus for the memory controller, peripheral buses, and local buses using various bus architectures. For example, such architectures may include ISA (Industry Standard Architecture) buses, MCA (Micro Channel Architecture) buses, EISA (Enhanced ISA) buses, VESA (Video Electronics Standards Association) local buses, and PCI (Peripheral Component Interconnect) buses, which can be implemented as Mezzanine buses manufactured in accordance with the IEEE P1386.1 standard.

[0163] The processing subsystem 1704 controls the operation of the computer system 1700 and may comprise one or more processing units 1732, 1734, etc. The processing units may include one or more processors, including single-core or multi-core processors, one or more cores of a processor, or a combination thereof. In some embodiments, the processing subsystem 1704 may include one or more dedicated coprocessors, such as a graphics processor or a digital signal processor (DSP). In some embodiments, some or all of the processing units of the processing subsystem 1704 may be implemented using custom circuitry, such as application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs).

[0164] In some embodiments, the processing unit included in the processing subsystem 1704 can execute instructions stored in system memory 1710 or on a computer-readable storage medium 1722. In various embodiments, the processing unit can execute various program or code instructions and maintain multiple concurrently running programs or processes. At any given time, some or all of the program code being executed may reside in system memory 1710 and / or on a computer-readable storage medium 1710, potentially including on one or more storage devices. Through appropriate programming, the processing subsystem 1704 can provide the various functions described above for dynamically modifying documents (e.g., web pages) according to usage patterns.

[0165] In certain embodiments, a processing acceleration device 1706 may be provided for performing customized processing or for reducing the load of some of the processing performed by the processing subsystem 1704 so as to speed up the overall processing performed by the computer system 1700.

[0166] The I / O subsystem 1708 may include devices and mechanisms for inputting information into the computer system 1700, and / or devices and mechanisms for outputting information via or from the computer system 1700. Generally, the term “input device” is intended to include all kinds of devices and mechanisms for inputting information into the computer system 1700. User interface input devices may include, for example, pointing devices such as keyboards, mice or trackballs, touchpads or touchscreens integrated into displays, scroll wheels, click wheels, dials, buttons, switches, keypads, voice input devices with voice command recognition systems, microphones, and other types of input devices. The input device may include motion detection devices such as a Microsoft Kinect® motion sensor and / or gesture recognition devices, a Microsoft Xbox® 360 game controller, and a device that provides an interface for accepting input using gesture commands and voice commands, enabling the user to control and interact with the input device. The user interface input device may also detect the user's eye movements (e.g., blinking while taking a photograph and / or making a menu selection) and eye behavior (eye The user interface input device may include an eye gesture recognition device, such as a Google Glass® blink detection device, which transforms gestures into input to an input device (e.g., Google Glass®). In addition, the user interface input device may include a voice recognition detection device that enables the user to interact with a voice recognition system (e.g., Siri® Navigator) by voice commands.

[0167] Other user interface input devices include, but are not limited to, three-dimensional (3D) mice, joysticks or pointing sticks, gamepads and graphic tablets, as well as audio / visual devices such as speakers, digital cameras, digital camcorders, portable media players, webcams, image scanners, fingerprint scanners, barcode readers, 3D scanners, 3D printers, laser rangefinders, and eye-tracking devices. In addition, user interface input devices may include, for example, medical imaging input devices such as computed tomography, magnetic resonance imaging, positron emission tomography, and ultrasound devices. Furthermore, user interface input devices may also include, for example, audio input devices such as MIDI keyboards and digital musical instruments.

[0168] The user interface output device may include non-visual display devices such as display subsystems, indicator lights, or audio output devices. The display subsystem may include flat panel display devices such as those using cathode ray tubes (CRTs), liquid crystal displays (LCDs), or plasma displays, projection devices, touchscreens, etc. In general, the use of the term “output device” is intended to include all kinds of devices and mechanisms for outputting information from the computer system 1700 to a user or another computer. For example, the user interface output device may include, but is not limited to, a variety of display devices that visually convey text, graphics, and audio / video information, such as monitors, printers, speakers, headphones, car navigation systems, drawing devices, audio output devices, and modems.

[0169] The storage subsystem 1718 provides a repository or datastore for storing information used by the computer system 1700. The storage subsystem 1718 provides a tangible, non-temporary, computer-readable storage medium for storing basic programming structures and data structures that provide functionality in several embodiments. Software (programs, code modules, instructions) that, when executed by the processing subsystem 1704, provides the aforementioned functionality may be stored in the storage subsystem 1718. The software may be executed by one or more processing units of the processing subsystem 1704. The storage subsystem 1718 may also provide a repository for storing data used in accordance with this disclosure.

[0170] The storage subsystem 1718 may include one or more non-temporary memory elements, including volatile and non-volatile memory elements. As shown in Figure 17, the storage subsystem 1718 comprises a system memory 1710 and a computer-readable storage medium 1722. The system memory 1710 stores instructions and data during program execution. The system memory may include several types of memory, including volatile main RAM (Random Access Memory) for storing information, and non-volatile ROM (Read Only Memory) or flash memory for storing fixed instructions. In some implementations, a BIOS (Basic Input / Output System) containing basic routines that help transfer information between elements within the computer system 1700 during startup, etc., may be stored in ROM. The RAM may include data and / or program modules that the processing subsystem 1704 is currently operating and executing. In some implementations, the system memory 1710 may include several different types of memory, such as SRAM (Static Random Access Memory) or DRAM (Dynamic Random Access Memory).

[0171] As an example, as shown in Figure 17, system memory 1710 may include, but is not limited to, application programs 1712 such as client applications, web browsers, mid-tier applications, and relational database management systems (RDBMS), program data 1714, and an operating system 1716. As an example, operating system 1716 may include various versions of Microsoft Windows®, Apple Macintosh®, and / or Linux operating systems, various commercially available UNIX® or UNIX-like operating systems (including, but not limited to, various GNU / Linux operating systems, Google Chrome® OS, etc.), and / or mobile operating systems such as iOS, Windows® Phone, Android® OS, BlackBerry® 10 OS, and Palm® OS operating systems.

[0172] The computer-readable storage medium 1722 may store programming structures and data structures that provide the functionality of several embodiments. When executed by the processing subsystem 1704, software (programs, code modules, instructions) that provides the above-mentioned functionality to the processor may be stored in the storage subsystem 1718. As an example, the computer-readable storage medium 1722 may include non-volatile memory such as a hard disk drive, a magnetic disk drive, an optical disk drive such as a CD-ROM, DVD, or Blu-ray® disc, or other optical media. The computer-readable storage medium 1722 may also include, but is not limited to, a Zip® drive, a flash memory card, a USB (Universal Serial Bus) flash drive, an SD (Secure Digital) card, a DVD disc, or a digital videotape. The computer-readable storage medium 1722 may include non-volatile memory-based SSDs (Solid-State Drives) such as flash memory-based SSDs, enterprise flash drives, and solid-state ROMs; volatile memory-based SSDs such as solid-state RAM, dynamic RAM, and static RAM; DRAM-based SSDs; magnetoresistive RAM (MRAM) SSDs; and hybrid SSDs that use a combination of DRAM SSDs and flash memory-based SSDs. The computer-readable medium 1722 may provide storage for computer-readable instructions, data structures, program modules, and other data for the computer system 1700.

[0173] In certain embodiments, the storage subsystem 1700 may include a computer-readable storage medium reader 1720 which can be further connected to the computer-readable storage medium 1722. Together with the system memory 1710, and optionally in combination with the system memory 1710, the computer-readable storage medium 1722 can be a remote storage device, a local storage device, a fixed storage device, and / or a removable storage device. Furthermore, it can comprehensively represent storage media for storing computer-readable information.

[0174] In certain embodiments, the computer system 1700 may provide support for running one or more virtual machines. The computer system 1700 may run programs such as hypervisors to facilitate the configuration and management of virtual machines. Each virtual machine may be allocated memory, computer (e.g., processor, core), I / O, and networking resources. Each virtual machine may run its own operating system. This operating system may be the same as or different from the operating systems run by other virtual machines running on the computer system 1700. Thus, multiple operating systems may be run simultaneously by the computer system 1700. Each virtual machine generally runs separately from other virtual machines.

[0175] The communication subsystem 1724 provides interfaces to other computer systems and networks. It functions as an interface for receiving data from computer system 1700 and transmitting data from computer system 1700 to other systems. For example, the communication subsystem 1724 may enable computer system 1700 to establish a communication channel over the internet to one or more client computing devices for sending and receiving information. In addition, the communication subsystem 1724 can be used to transmit notifications of successful login or notifications from privileged account administrators prompting the requesting user to re-enter their password.

[0176] The communication subsystem 1724 may support both wired and / or wireless communication protocols. For example, in certain embodiments, the communication subsystem 1724 may include an RF (Radio Frequency) transmitting component for accessing a wireless voice network and / or data network (e.g., using cellular technology, 3G, 4G, or next-generation data network technologies such as EDGE (Enhanced Data Rates For Global Evolution), WiFi (IEEE 802.11 family of standards), other mobile object communication technologies, or any combination thereof), a GPS (Global Positioning System) receiving component, and / or other components. In some embodiments, the communication subsystem 1724 may provide wired network connectivity (e.g., Ethernet) in addition to, or instead of, a wireless interface.

[0177] The communication subsystem 1724 can send and receive data in various forms. For example, in some embodiments, the communication subsystem 1724 may receive input communications in the form of structured and / or unstructured data feeds 1726, event streams 1728, event updates 1730, etc. For example, the communication subsystem 1724 may be configured to receive (or send) data feeds 1726 in real time from users of social media networks and / or other communication services, such as Twitter® feeds, Facebook® updates, RSS (Rich Site Summary) feeds and / or real-time updates from one or more third-party information sources.

[0178] In certain embodiments, the communication subsystem 1724 may be configured to receive data in the form of a continuous data stream, which may include an event stream 1728 of real-time events and / or event updates 1730 that is continuous or infinite and essentially has no clear end. Applications could include, for example, sensor data applications, tickers, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, and vehicle traffic monitoring.

[0179] Furthermore, the communication subsystem 1724 may be configured to output structured and / or unstructured data feeds 1726, event streams 1728, event updates 1730, etc., to one or more databases that may be communicating with one or more streaming data source computers connected to the computer system 1700.

[0180] Computer system 1700 can be one of a variety of types, including palm-sized portable devices (e.g., iPhone® mobile phones, iPad® computing tablets, PDAs), wearable devices (e.g., Google Glass® head-mounted displays), personal computers, workstations, mainframes, kiosks, server racks, or other data processing systems.

[0181] Due to the ever-changing nature of computers and networks, the description of the computer system 1700 shown in Figure 17 is merely an example. Many other configurations with more or fewer components than the system shown in Figure 17 are possible. Based on the disclosures and teachings contained herein, those skilled in the art will be able to see other ways and / or methods for realizing various embodiments.

[0182] The systems shown in some of the drawings may be provided in various configurations. In some embodiments, these systems may be configured as distributed systems in which one or more components of the system are distributed across one or more networks in one or more cloud infrastructure systems.

[0183] A cloud infrastructure system is a collection of one or more server computing devices, network equipment, and / or storage devices. These resources can be divided by a cloud service provider and allocated to its customers in some way. For example, a cloud service provider such as Oracle Corporation in Redwood Shores, California, may offer various types of cloud services, including one or more services offered under the classification of SaaS (Software as a Service), services offered under the classification of PaaS (Platform as a Service), services offered under the classification of IaaS (Infrastructure as a Service), or other classifications including mixed services. Examples of SaaS services include, but are not limited to, the ability to build and deliver on-demand application sets such as Oracle Fusion applications. SaaS services enable customers to use applications running on the cloud infrastructure system without having to purchase software for the applications. Examples of PaaS services include JCS (Oracle Java Cloud Service) and DBCS (Oracle Database Cloud Service). IaaS services include, but are not limited to, services that enable organizations (such as Oracle) to consolidate existing applications onto a shared common architecture, and the ability to create new applications that leverage the shared services provided by the platform. IaaS services will make it easier for customers using services provided by SaaS and PaaS platforms to manage and control underlying computing resources, such as storage, networking, and other basic computing resources.

[0184] Figure 18 is a simplified block diagram of one or more components of a system environment 1800 in which services provided by one or more components of the system of the embodiment may be provided as cloud services, according to an embodiment of the present disclosure. In the illustrated embodiment, the system environment 1800 includes one or more client computing devices 1804, 1806, and 1808. One or more client computing devices 1804, 1806, and 1808 may be used by users to interact with a cloud infrastructure system 1802 that provides cloud services. The client computing devices may be configured to operate client applications, such as a web browser, a proprietary client application (e.g., Oracle Forms), or other applications. The client applications may be used by users of the client computing devices to interact with the cloud infrastructure system 1802 and to use services provided by the cloud infrastructure system 1802.

[0185] It should be understood that the cloud infrastructure system 1802 shown in the figure may have components other than those illustrated. Furthermore, the embodiment shown in the figure is only one example of a cloud infrastructure system that may incorporate embodiments of this disclosure. In some other embodiments, the cloud infrastructure system 1802 may have more or fewer components than those shown in the figure, may combine two or more components, or may have different configurations or arrangements of components.

[0186] Client computing devices 1804, 1806, and 1808 may be similar to the client computing devices described above with respect to 1602, 1604, 1606, and 1608.

[0187] The exemplary system environment 1800 is shown with three client computing devices, but any number of client computing devices may be supported. Other devices, such as sensor-equipped devices, may interact with the cloud infrastructure system 1802.

[0188] Network(s) 1810 can facilitate data communication and exchange between clients 1804, 1806, and 1808 and the cloud infrastructure system 1802. Each network(s) may be any type of network familiar to those skilled in the art, capable of supporting data communication using any of the various commercially available protocols, including the protocols described above with respect to network(s) 1810.

[0189] The cloud infrastructure system 1802 may consist of one or more computers and / or servers, which may include the computers and servers described above with respect to server 1812.

[0190] In certain embodiments, the services provided by the cloud infrastructure system may include a host of services that become available upon request from users of the cloud infrastructure system, such as online data storage and backup solutions, web-based email services, hosted office suite document integration services, database processing, and managed technical support services. The services provided by the cloud infrastructure system can be dynamically scaled to meet the needs of its users. Specific instances of the services provided by the cloud infrastructure system are referred to herein as “service instances.” Generally, these are accessible to users via communication networks such as the Internet. Any service from a cloud service provider's system that becomes available is referred to as a "cloud service." In a public cloud environment, the servers and systems that make up the cloud service provider's system are different from the customer's on-premises servers and systems. For example, a cloud service provider's system may host applications, and users can order and use those applications on demand via a communication network such as the internet.

[0191] In some examples, services in a computer network cloud infrastructure may include storage, hosted databases, hosted web servers, secure computer network access to software applications, or other services provided by the cloud vendor to the user, or other services not mentioned above that are well known in the art. For example, services may include password-protected access to remote storage on the cloud over the internet. Another example is that services may include a web service-based hosted relational database and scripting language middleware engine for private use by networked developers. Yet another example is that services may include access to an email software application hosted on the cloud vendor's website.

[0192] In certain embodiments, the cloud infrastructure system 1802 may include a suite of applications, middleware, and database service offerings delivered to customers in a self-service, subscription-based, scalable, reliable, and highly available secure manner. An example of such a cloud infrastructure system is the Oracle Public Cloud offered by the assignee of this application.

[0193] In various embodiments, the cloud infrastructure system 1802 may be configured to automatically provision, manage, and track customer subscriptions to the services provided by the cloud infrastructure system 1802. The cloud infrastructure system 1802 may provide cloud services through different deployment models. For example, the services may be provided under a public cloud model. In a public cloud model, the cloud infrastructure system 1802 is owned by an organization providing the cloud services (e.g., owned by Oracle), and the services are made available to the general public or to different industrial enterprises. As another example, the services may be provided under a private cloud model where the cloud infrastructure system 1802 runs for only one organization, and may provide services for one or more entities within that organization. The cloud services may also be provided under a community cloud model. In a community cloud model, the cloud infrastructure system 1802 and the services it provides are shared by several organizations within a relevant community. The cloud services may also be provided under a hybrid cloud model. A hybrid cloud model is a combination of two or more different models.

[0194] In some embodiments, the services provided by the cloud infrastructure system 1802 may include one or more services provided under other classifications, including SaaS (Software as a Service), PaaS (Platform as a Service), IaaS (Infrastructure as a Service), or mixed services. A customer may order one or more services provided by the cloud infrastructure system 1802 by ordering a subscription. The cloud infrastructure system 1802 then processes This is done to provide the service that is in the customer's subscription order.

[0195] In some embodiments, the services provided by the cloud infrastructure system 1802 may include, but are not limited to, application services, platform services, and infrastructure services. In some examples, application services may be provided by the cloud infrastructure system via SaaS services. A SaaS platform may be configured to provide cloud services that fall under the SaaS classification. For example, a SaaS platform may provide the functionality to build a suite of on-demand applications and deliver them to an integrated development / deployment platform. A SaaS platform may manage and control the underlying software and infrastructure for providing SaaS services. By using the services provided by the SaaS platform, customers can utilize applications running on the cloud infrastructure system. Customers can obtain application services without having to purchase separate licenses and support. A variety of different SaaS services may be provided. Examples include, but are not limited to, services that provide solutions for sales performance management, enterprise integration, and business flexibility for large organizations.

[0196] In some embodiments, platform services may be provided by a cloud infrastructure system via a PaaS platform. The PaaS platform may be configured to provide cloud services that fall under the PaaS classification. Platform services include, but are not limited to, services that enable organizations (such as Oracle) to consolidate existing applications onto a shared common architecture, and the ability to create new applications that leverage the shared services provided by the platform. The PaaS platform may manage and control the underlying software and infrastructure for providing PaaS services. Customers can obtain services provided by the PaaS cloud infrastructure system without having to purchase licenses and support separately. Platform services include, but are not limited to, JCS (Oracle Java Cloud Service), DBCS (Oracle Database Cloud Service), and others.

[0197] By utilizing the services provided by the PaaS platform, customers can adopt programming languages ​​and tools supported by the cloud infrastructure system and manage the deployed services. In some embodiments, the platform services provided by the cloud infrastructure system may include database cloud services, middleware cloud services (e.g., Oracle Fusion Middleware services), and Java cloud services. In one embodiment, the database cloud service may support a shared service deployment model, which allows an organization to pool database resources and provide the database to customers as a service in the form of a database cloud. Middleware cloud services may provide a platform for customers to develop and deploy various business applications, and Java cloud services may provide a platform for customers to deploy Java applications on the cloud infrastructure system.

[0198] In cloud infrastructure systems, IaaS platforms can provide a variety of different infrastructure services. These infrastructure services are provided by SaaS and PaaS platforms. It facilitates the management and control of underlying computing resources, such as storage, networking, and other basic computing resources, for customers using the system.

[0199] In certain embodiments, the cloud infrastructure system 1802 may include infrastructure resources 1830 for providing resources used to provide various services to customers of the cloud infrastructure system. In one embodiment, the infrastructure resources 1830 may include a pre-integrated optimal combination of hardware such as servers, storage, and networking resources, as well as other resources, for running services provided by the PaaS platform and SaaS platform.

[0200] In some embodiments, resources in the cloud infrastructure system 1802 can be shared by multiple users and dynamically reallocated on demand. In addition, resources can be allocated to users in different time zones. For example, the cloud infrastructure system 1830 can enable a first set of users in a first time zone to use the resources of the cloud infrastructure system for a specified number of hours, and then reallocate the same resources to another set of users located in a different time zone, thereby maximizing resource utilization.

[0201] In certain embodiments, several internal shared services 1832 may be provided, shared by different components or modules of the cloud infrastructure system 1802 and by the services provided by the cloud infrastructure system 1802. These internal shared services may include, but are not limited to, services that enable security / identity services, integration services, enterprise repository services, enterprise manager services, virus scanning / whitelisting services, highly available backup / recovery services, cloud support, email services, notification services, file transfer services, and the like.

[0202] In certain embodiments, the cloud infrastructure system 1802 may provide comprehensive management of cloud services (e.g., SaaS services, Paa services, and IaaS services) within the cloud infrastructure system. In one embodiment, the cloud management functionality may include functions for provisioning, managing, and tracking customer subscriptions received by the cloud infrastructure system 1802, etc.

[0203] In one embodiment, as shown in the figure, cloud management functionality may be provided by one or more modules, such as an order management module 1820, an order orchestration module 1822, an order provisioning module 1824, an order management / monitoring module 1826, and an identity management module 1828. These modules may include, or be provided using, one or more computers and / or servers. One or more computers and / or servers may be general-purpose computers, dedicated server computers, server farms, server clusters, or other appropriate arrangements and / or combinations.

[0204] In exemplary operation 1834, a customer using a client device such as client device 1804, 1806, or 1808 requests one or more services provided by the cloud infrastructure system 1802 and orders a subscription for one or more services provided by the cloud infrastructure system 1802. This allows interaction with the cloud infrastructure system 1802. In certain embodiments, the customer may access a cloud UI (user interface) such as cloud UI 1812, cloud UI 1814, and / or cloud UI 1816, and place a subscription order through these UIs. The order information received by the cloud infrastructure system 1802 in response to the customer's order may include information identifying this customer, and one or more services provided by the cloud infrastructure system 1802 that the customer intends to subscribe to.

[0205] After an order is placed by the customer, the order information is received via the cloud UI 1812, 1814, and / or 1816.

[0206] In operation 1836, this order is stored in the order database 1818. The order database 1818 may be one of several databases operated by the cloud infrastructure system 1818 and operated together with other system elements.

[0207] In operation 1838, order information is transferred to the order management module 1820. In some cases, the order management module 1820 may be configured to perform billing and accounting functions related to the order, such as order confirmation and registration of confirmed orders.

[0208] In operation 1840, information about the order is transmitted to the order orchestration module 1822. The order orchestration module 1822 may use this order information to orchestrate the provisioning of services and resources related to the order placed by the customer. In some cases, the order orchestration module 1822 may orchestrate the provisioning of resources to support services that are subscribed to using the services of the order provisioning module 1824.

[0209] In a particular embodiment, the order orchestration module 1822 enables the management of the workflow associated with each order and applies business logic to determine whether the order should proceed to provisioning. In operation 1842, upon receiving an order for a new subscription, the order orchestration module 1822 sends a request to the order provisioning module 1824 to allocate and configure the resources necessary to fulfill the subscription order. The order provisioning module 1824 enables the allocation of resources for the services subscribed to by the customer. The order provisioning module 1824 provides an abstraction between the cloud services provided by the cloud infrastructure system 1800 and the physical implementation layer used to provision resources to provide the requested services. Thus, the order orchestration module 1822 can be decoupled from implementation details such as whether the services and resources were actually provisioned on the fly or whether they were pre-provisioned and allocated only when requested.

[0210] In operation 1844, once services and resources are provisioned, the order provisioning module 1824 of the cloud infrastructure system 1802 may send notifications about the services provided to customers on client devices 1804, 1806, and / or 1808. In operation 1846, the order management / monitoring module 1826 may manage and track customer subscription orders. In some cases, the order management / monitoring module 1826 may manage subscriptions The system may be configured to collect usage statistics data related to the service in the order of the reply, such as the amount of storage used, the amount of data transferred, the number of users, and the system's uptime and downtime.

[0211] In certain embodiments, the cloud infrastructure system 1800 may include an identity management module 1828. The identity management module 1828 may be configured to provide identity services in the cloud infrastructure system 1800, such as access management and authorization services. In some embodiments, the identity management module 1828 may manage information about customers who wish to use the services provided by the cloud infrastructure system 1802. Such information may include information that authenticates the identity of such customers, and information that describes what operations those customers are authorized to perform on various system resources (e.g., files, directories, applications, communication ports, memory segments, etc.). The identity management module 1828 may also include managing descriptive information about each customer, and descriptive information about who can access and modify that descriptive information and how.

[0212] While specific embodiments of the Disclosure have been described, various modifications, alternatives, alternative configurations, and equivalents are also included within the scope of the Disclosure. The embodiments of the Disclosure are not limited to operation within a specific data processing environment, but can freely operate within multiple data processing environments. In addition, while specific sequences of transactions and steps have been used to illustrate the embodiments of the Disclosure, it should be apparent to those skilled in the art that the scope of the Disclosure is not limited to the described sequences of transactions and steps. Various features and aspects of the embodiments described above may be used individually or in combination.

[0213] Furthermore, while embodiments of the Disclosure have been described using specific combinations of hardware and software, it should be recognized that other combinations of hardware and software are also within the scope of the Disclosure. Embodiments of the Disclosure may be implemented using hardware alone, software alone, or a combination thereof. Various processes described herein can be implemented on different processors, either the same processor or any combination thereof. Thus, wherever a component or module is described as being configured to perform a particular operation, such configuration can be achieved, for example, by designing electronic circuitry to perform this operation, by programming programmable electronic circuitry (such as a microprocessor) to perform this operation, or by any combination thereof. Processes may communicate using a variety of techniques, including, but not limited to, prior art for inter-process communication, and different pairs of processes may use different techniques, and the same pair of processes may use different techniques at different times.

[0214] The specification and drawings should be considered, as appropriate, as examples only and not strict. However, it will be apparent that additions, subtractions, deletions, and other modifications and variations may be made to them without departing from the broader intent and scope set forth in the attached claims. Thus, specific embodiments of this disclosure have been described, but these are not intended to be limiting. Various modifications and equivalents are within the scope of the attached claims.

Claims

1. A method implemented by a computer, The computing device performs one or more analyses of one or more sets of configuration files associated with each of several services deployed across multiple execution targets in a cloud computing environment, The computing device generates a first graph for deploying the multiple services to one of the multiple execution targets of the cloud computing environment, the first graph being generated at least in part on performing one or more analyses of the configuration files, and the first graph showing the dependencies between tasks related to deploying the multiple services to the execution target, the method implemented by the computer further, The computing device includes generating a second graph for deploying the plurality of services to the plurality of execution targets, at least in part on performing the analysis of one or more of the configuration files, the plurality of execution targets include the execution targets, and the second graph identifies dependencies related to the different deployments of the plurality of services to each of the plurality of execution targets. A method implemented by a computer, further comprising the computing device deploying the plurality of services to the plurality of execution targets, at least in part on traversing the first graph and the second graph.

2. The computer-implemented method according to claim 1, wherein the computing device runs a cloud infrastructure orchestration service configured to provision infrastructure resources and deploy services to the infrastructure resources based at least part of a set of one or more configuration files, the set of one or more configuration files collectively describe the infrastructure resources to be provisioned and the services to be deployed via declarative instructions.

3. A method implemented by a computer according to claim 1 or 2, wherein the first graph and the second graph are directed acyclic graphs, the first graph identifies the dependency of the first resource on the capabilities of the second resource, and the first resource and the second resource are each of the services of the plurality of services, and the capabilities are part of the functionality of the second resource.

4. The method implemented by a computer according to any one of claims 1 to 3, further comprising the computing device generating a linked list data structure based at least in part on performing the parsing of the set of one or more configuration files, wherein the linked list data structure identifies dependencies between a plurality of deployment phases, each phase relating to a different set of execution targets to which the plurality of services are deployed.

5. The computer-implemented method according to claim 4, wherein the nodes of the linked list data structure refer to at least one node of the second graph, and at least one node of the second graph refers to a node of the first graph.

6. Deploying the multiple services to the multiple execution targets means Traversing up to the first node of the linked list data structure, Identifying the second graph from the first node of the linked list data structure, To traverse up to the first node of the second graph, Identifying the first graph from the first node of the second graph, Traversing to the first node of the aforementioned first graph, A computer-implemented method according to claim 5, further comprising traversing to a first node of the linked list data structure, traversing to the first node of the second graph, and traversing to the first node of the first graph, to expand at least a portion of the first service among the plurality of services.

7. Based at least in part on traversing up to the second node of the first graph, it is recognized that a portion of the deployed second service depends on the capabilities of the available third service, Recognizing that the aforementioned capabilities of the third service are unavailable, The computer-implemented method according to claim 6, further comprising delaying the deployment of the portion of the second service until the capacity of the third service becomes available.

8. A computing device, One or more processors, The computing device comprises one or more memories that store instructions executable by a computer, and when the instructions executable by the computer are executed by the one or more processors, the computing device is configured to: In a cloud computing environment, among multiple services deployed to multiple execution targets, each service is required to perform one or more analyses of one or more sets of configuration files associated with it. The cloud computing environment generates a first graph for deploying the multiple services to one of the multiple execution targets, the first graph is generated at least in part based on performing one or more analyses of the configuration files, the first graph shows the dependencies between tasks related to deploying the multiple services to the execution target, and the computing device further... Based at least in part on performing the analysis of one or more of the configuration files, a second graph is generated for deploying the multiple services to the multiple execution targets, the multiple execution targets include the execution targets, the second graph identifies dependencies related to the different deployments of the multiple services to each of the multiple execution targets, and further to the computing device, A computing device that deploys the plurality of services to the plurality of execution targets, at least in part on traversing the first graph and the second graph.

9. The computing device according to claim 8, wherein executing instructions executable by the computer causes the computing device to further execute a cloud infrastructure orchestration service configured to provision infrastructure resources and deploy services to the infrastructure resources based at least in part on a set of one or more configuration files, the set of one or more configuration files collectively describe the infrastructure resources to be provisioned and the services to be deployed via declarative instructions.

10. The computing device according to claim 8 or 9, wherein the first graph identifies the dependency of the first resource on the capabilities of the second resource, and the first resource and the second resource are each of the services of the plurality of services, and the capabilities are part of the functionality of the second resource.

11. The computing device according to any one of claims 8 to 10, wherein executing instructions executable by the computer is at least in part based on performing one or more parses of the set of one or more configuration files, thereby causing the computing device to further generate a linked list data structure, the linked list data structure identifies dependencies between a plurality of deployment phases, each phase relating to a different set of execution targets to which the plurality of services are deployed.

12. The computing device according to claim 11, wherein the nodes of the linked list data structure refer to at least one node of the second graph, and the at least one node of the second graph refers to a node of the first graph.

13. Deploying the multiple services to the multiple execution targets means Traversing up to the first node of the linked list data structure, Identifying the second graph from the first node of the linked list data structure, To traverse up to the first node of the second graph, Identifying the first graph from the first node of the second graph, Traversing to the first node of the aforementioned first graph, The computing device according to claim 12, further comprising traversing to the first node of the linked list data structure, traversing to the first node of the second graph, and traversing to the first node of the first graph, to deploy at least a portion of the first service among the plurality of services.

14. Executing instructions executable by the aforementioned computer further involves the computing device, Based at least in part on traversing up to the second node of the first graph, it is made to recognize that a portion of the second service being deployed depends on the capabilities of the available third service, To make it clear that the aforementioned capabilities of the third service are unavailable, The computing device according to claim 13, which delays the deployment of the portion of the second service until the capabilities of the third service become available.

15. A computer program for causing a computer to perform the method described in any one of claims 1 to 7.

Images