Techniques for utilizing a directed acyclic graph for deployment instructions

Abstract

Techniques are disclosed for utilizing a directed acyclic graph for deployment instructions. The computer-implemented method may include various operations. Instructions may be executed by a computing device to perform multiple analyses of configuration data related to deployment. The computing device may generate a first directed acyclic graph (DAG), where the first DAG is utilized to deploy a first resource based on the analysis. A second DAG may be generated based on the analysis for deploying execution targets, where the second DAG specifies dependencies between the execution targets of the deployment. The computing device may generate a linked list data structure based on the analysis and may deploy the computing system by traversing the linked list data structure.

Description

Technical Field

[0001] This application is a regular application and claims the benefit and priority of U.S. Provisional Application No. 62 / 963,477, filed on January 20, 2020, entitled "TECHNIQUES FOR UTILIZING DIRECTED ACYCLIC GRAPHS FOR DEPLOYMENT INSTRUCTIONS", and U.S. Regular Application No. 16 / 953,262, filed on November 19, 2020, entitled "TECHNIQUES FOR UTILIZING DIRECTED ACYCLIC GRAPHS FOR DEPLOYMENT INSTRUCTIONS", under 35 U.S.C. § 119(e). The entire contents of both are hereby incorporated by reference herein for all purposes.

Background Art

[0002] Background Today, cloud infrastructure services utilize many individual services for provisioning and deploying code and settings (respectively) across many areas of cloud infrastructure services. In particular, considering that provisioning is generally declarative and code deployment is imperative, these tools require a significant amount of manual effort to use. In addition, as the number of service teams and regions increases, cloud infrastructure services also need to continue to grow. As a strategy for some cloud infrastructure services to deploy to more and smaller regions, there is a strategy of regionalizing spending, but it has not been able to handle it well.

Summary of the Invention

Means for Solving the Problems

[0003] Summary Disclosed is a technique for utilizing a directed acyclic graph for deployment instructions. In some embodiments, a computer-implemented method may include various operations. Instructions for performing an analysis of configuration data related to deployment may be executed by a computing device. The computing device may cause a first DAG (directed acyclic graph) to be generated, the first DAG being utilized for deploying a first resource (e.g., a software service) based on the analysis. A second DAG for deploying execution targets based on phases may be generated, the second DAG specifying dependencies between execution targets of the deployment. The computing device may generate a linked list data structure based on the analysis and may deploy a computing system by traversing the linked list data structure.

[0004] In other embodiments, a system for utilizing a DAG for deployment instructions is disclosed. The system may comprise one or more processors and one or more memories storing computer-executable instructions, which, when executed by the one or more processors, configure the one or more processors to perform various operations. The computing device may execute instructions for performing one or more analyses of configuration data related to the deployment of a computing system. The computing device may cause a first DAG to be generated, which is utilized to deploy a first resource based at least in part on the performance of the one or more analyses. The computing device may generate a second DAG for deploying a plurality of execution targets based at least in part on the performance of the one or more analyses, where the second DAG specifies dependencies between the execution targets of the deployment. The computing device may generate a linked list data structure based at least in part on the performance of the one or more analyses, where the linked list data structure specifies dependencies between a plurality of deployment phases. And the computing device may deploy the computing system based at least in part on traversing the linked list data structure, the second DAG, and the first DAG.

[0005] In other embodiments, a computer-readable storage medium for utilizing a DAG for deployment instructions that can store computer-executable instructions is disclosed. The computer-executable instructions, when executed by one or more processors, cause the one or more processors to perform various operations. A computing device may execute instructions for performing one or more analyses of configuration data related to the deployment of a computing system. The computing device may generate a first DAG, which is utilized to deploy a first resource based at least in part on the performance of the one or more analyses. The computing device may generate a second DAG for deploying a plurality of execution targets based at least in part on the performance of the one or more analyses, and the second DAG specifies dependencies between the execution targets of the deployment. The computing device may generate a linked list data structure based at least in part on the performance of the one or more analyses, and the linked list data structure specifies dependencies between a plurality of deployment phases. And the computing device may deploy the computing system based at least in part on traversing the linked list data structure, the second DAG, and the first DAG.

[0006] In other embodiments, an apparatus is disclosed. The apparatus may comprise means for performing the steps of the methods disclosed herein.

[0007] In other embodiments, a computer program product is disclosed. The computer program product may include computer instructions that, when executed by a processor, perform the steps of the methods disclosed herein.

[0008] To facilitate understanding of the description of a particular element or operation, the topmost one or more digits represented by a reference number refer to the number of the drawing in which the element is first introduced.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

DETAILED DESCRIPTION OF THE INVENTION

[0010] Detailed Description In some examples, IaaS (infrastructure as a service) is a particular type of cloud computing. IaaS can be configured to provide virtualized computing resources over a public network (e.g., the Internet). In some examples, IaaS is one of three main categories (or sub - categories) of cloud computing services. Many people consider the other main categories to be SaaS (Software as a Service) and PaaS (Platform as a Service), and SaaS may be considered a broader category that includes both PaaS and IaaS, and some may consider IaaS a sub - category of PaaS.

[0011] In an IaaS model, a cloud computing provider can host infrastructure components (e.g., servers, storage, network nodes (e.g., hardware), deployment software, platform virtualization (e.g., hypervisor layer), etc.).

[0012] In some cases, an IaaS provider can supply various services associated with these infrastructure components (e.g., billing, monitoring, logging, security, load balancing, and clustering, etc.). Thus, since these services can be policy - driven services, IaaS users will be able to implement policies that drive load balancing to maintain application availability and the performance of the application.

[0013] In some cases, an IaaS customer may access resources and services through a WAN (Wide Area Network) such as the Internet and install the remaining elements of the application stack using the services of the cloud provider. For example, a user can log in to the IaaS platform to create a VM (Virtual Machine), install an OS (Operating System) on each VM, deploy middleware such as a database, create storage buckets for workloads and backups, and even install enterprise software on the VM. Subsequently, the customer can use the provider's services to perform various functions, including load balancing of network traffic, troubleshooting application problems, monitoring performance, and managing disaster recovery.

[0014] In most cases, the cloud computing model requires the participation of a cloud provider. The cloud provider may be a third-party service that specializes in providing (for example, selling) IaaS, but this is not necessary. Also, an entity may choose to deploy a private cloud and become a provider of its own infrastructure services.

[0015] In some examples, an IaaS deployment is the process of placing a new application or new version on a prepared application server or the like. It may also include the process of preparing the server (for example, installing libraries, daemons, etc.). This is often managed by the cloud provider at a layer below the hypervisor layer (for example, servers, storage, network hardware, and virtualization). Thus, the customer may be responsible for handling, for example, (OS), middleware, and / or application deployments (such as self-service virtual machines that can be spun up based on requests, etc.).

[0016] In some cases, IaaS provisioning can refer to obtaining the computers or virtual hosts to be used and even installing the necessary libraries or services on them. In most cases, deployment does not include provisioning, and provisioning will need to be done first.

[0017] In some cases, there are two different issues with IaaS provisioning. First, there is the initial challenge of provisioning the initial set of infrastructure before doing anything else. Second, once everything is provisioned, there is the challenge of evolving the existing infrastructure (e.g., adding new services, changing services, deleting services, etc.). In some cases, these two challenges will be addressed by enabling the infrastructure configuration to be defined declaratively. That is, the infrastructure (e.g., which components are needed and how they interact) can be defined by one or more configuration files. Thus, the overall topology of the infrastructure (e.g., which resources depend on which resources and how they interact with each other) can be described declaratively. In some cases, once the topology is defined, a workflow can be generated to create and / or manage the different components described in the configuration file.

[0018] In some embodiments, IaaS provisioning may include generating a DAG (Directed Acyclic Graph). The DAG can be a finite directed graph that includes any suitable number of nodes and edges. Each edge is directed from one node to another. The nodes and edges can be arranged so as not to form a directed cycle. That is, the DAG is defined such that after starting from any node, a consistent series of directed edges will not ultimately loop back to that same node. IaaS provisioning may include parsing configuration files corresponding to one or more resources of the system (such as services, software resources, etc.). A separate DAG can be generated for each resource. The DAG for each resource can define the dependency of that resource on the capabilities of one or more other resources of the system (such as services, software resources, etc.). For example, a first DAG can be generated and used to deploy a first resource (such as a software service, also referred to as a "computing service", like an email service configured to manage electronic messages for one or more users). The first DAG can indicate the dependency of the first resource on the capabilities provided by a second resource (such as another software service different from the first resource, like an ID service configured to verify / authenticate a user's ID based on previously obtained credentials). The first resource and / or the second resource can each be one of the multiple services provided by the computing system. "Capability" can refer to a part of the function of a given resource (such as the capability of the second resource to verify / authenticate a user's ID). A process for traversing the DAG can be instantiated. When the process reaches a node in the DAG corresponding to a currently unavailable capability, since the process has reached the dependency on that capability, it can announce to the scheduling service that it is waiting for that specific capability to become available before proceeding. When various resources of the system are deployed and / or started, these resources can announce to the scheduling service that these capabilities are available when the various capabilities become available.As used herein, the terms "starting", "being started", and "having been started" refer to the process of performing a startup sequence composed of operations corresponding to specific resources (e.g., software / computing services, computing devices, etc.). Deploying a resource (e.g., a software service) may include starting up and / or making available at least some of the functions (e.g., one or more capabilities) provided by the resource. If the scheduling service determines that a particular capability has become available, the process may resume from the point where it last ended (e.g., immediately after announcing that the capability is needed). This process may regenerate the DAG and resume traversal (e.g., from the last accessed node). By using a DAG for each resource, the system can manage the dependencies between the capabilities of different resources so that a human operator does not need to manually start up a complex system.

[0019] In some examples, the infrastructure may have many interconnected elements. For example, there may be one or more VPCs (virtual private clouds), also known as core networks (which may be an on-demand pool of configurable computing resources and / or shared computing resources). Also, in some examples, there may be one or more security group rules provisioned to define how network security is set up, and one or more VMs (virtual machines). Other infrastructure elements, such as load balancers, databases, etc., may be provisioned. As more infrastructure elements are desired and / or added, the infrastructure will evolve incrementally.

[0020] As described above, one way to provision infrastructure is to declaratively describe the infrastructure. Thus, the configuration file may be a declarative file that only describes each of the above infrastructure components and how they interact with each other. The configuration file can describe the resources and related fields necessary to create the elements, and other elements that reference the previously described elements may be described. In some examples, the provisioning tool may then generate a workflow for creating and managing the elements described in the configuration file.

[0021] In some cases, the workflow of the provisioning tool can be configured to execute various commands. One function that can be executed is View Reconciliation. In View Reconciliation, the provisioning tool can compare the current view of the infrastructure (e.g., the expected state of the infrastructure) with the actual operation of the infrastructure. In some cases, executing the View Reconciliation function may involve querying various resource providers or infrastructure resources to identify which resources are actually in operation. Another function that the provisioning tool can execute is plan generation. In plan generation, the provisioning tool can compare the actually operating infrastructure components with how the provisioning tool wants the state to look (e.g., the desired configuration). That is, the plan generation function can determine what changes need to be made to bring the resources to the latest expected state. In some cases, a third function is the execution (e.g., application) function. In the execution function, the provisioning tool can execute the plan generated by the plan generation function.

[0022] Generally, a provisioning tool can be configured to obtain a configuration file, parse the declarative information contained within the configuration file, and programmatically / automatically determine the order in which resources must be provisioned in order to execute a plan. For example, if a VPC needs to be launched before the rules of a security group and a VM are launched, the provisioning tool can make this determination and perform the launches in the determined order without user intervention and / or without the need to include that information in the configuration file.

[0023] In some cases, continuous deployment techniques can be employed to enable the deployment of infrastructure code across various virtual computing environments. In addition to this, the described techniques can enable infrastructure management within these environments. In some examples, a service team can write code that is desired to be deployed to one or more production environments, often many different production environments (e.g., spanning various different geographical locations and, in some cases, worldwide). However, in some examples, the infrastructure to which the code is to be deployed must first be set up. In some cases, provisioning can be done manually, a provisioning tool can be utilized to provision resources, and / or a deployment tool can be utilized to deploy the code once the infrastructure has been provisioned.

[0024] As described above, there are generally two different tools, and these tools are used to handle each of the provisioning of infrastructure resources and the deployment of code for controlling infrastructure resources while manual orchestration is performed between the tools. However, when implemented on a large scale, in a manual implementation, deviations always occur. Therefore, a more efficient and reliable technology for implementing a virtual cloud environment becomes possible with an automated tool that can perform both the provisioning and deployment of virtual infrastructure.

[0025] In some examples, when two tools are used, problems may occur if a user manually modifies the code between the provisioning phase and the deployment phase. As described herein, a technique of using one tool for both provisioning and deployment can reduce such problems by automating the process and eliminate the opportunity to manually modify the code. There may also be cases where a slight change in a single user's coding can cause major problems in the deployment phase. In some examples, when an operator first operates in a new region (e.g., there is an input error in the code), an object coded with an input error may remain as it is. If the application is deployed with that input error and the application is not affected by that input error (e.g., it operates temporarily), future additional code changes may be affected by that input error and crash the entire system. Therefore, the technology provided herein can eliminate the provisioning - deployment gap that can often cause problems.

[0026] Generally, modeling a deployment is declarative such that configuration files can be used to declare infrastructure resources. For example, CRUD (Create, Read, Update, and Delete) commands are commonly used to generate deployment files using common REST (Representational State Transfer) concepts (e.g., REST APIs (Application Programming Interfaces)). However, the deployment itself generally does not follow the same concepts. In addition, while infrastructure provisioning tools tend to be very powerful and / or expressive tools, tools for deployment tend to be much more limited in terms of the operations they can perform (e.g., deployment tools are imperative as opposed to declarative). Thus, there has long been a need for one tool that can handle both functional requirements (e.g., provisioning and deployment of infrastructure elements) within a cloud environment.

[0027] In some examples, techniques for implementing CIOS (Cloud Infrastructure Orchestration Service) are described. These techniques can be configured to manage both the provisioning and deployment of infrastructure resources within a cloud environment, as briefly described above. In some cases, CIOS can include two service classes: a central component and a regional component (e.g., CIOS Central and CIOS Regional). Throughout the specification, the following terms are used.

[0028] ● Infrastructure component - Long-lived parts of the infrastructure that support the running code.

[0029] ○ Examples: Deployment applications, load balancers, DNS (Domain Name System) entries, object storage buckets, etc.

[0030] ● Artifact - Deployment application, or code deployed in a Kubernetes Engine cluster, or configuration information applied to infrastructure components (hereinafter referred to as "configuration (config)"). These may be read-only resources.

[0031] ● Deployment task - A short-term task often related to deploying or testing code. In addition, deployment tasks are modeled as resources. Such resources do not exist longer than the release that creates the resource.

[0032] ○ Example: "deploy $artifact to $environment", "watch $alarm for 10 minutes", "execute $testSuite", or "wait for $manualApproval" ○ For example, CIOS can model the deployment of a deployment orchestrator as the creation of a resource that transitions to an available state when completed.

[0033] ○ CIOS can control the life cycle of these short-term resources because CIOS holds the state of its cloud infrastructure service declarative provisioner and this life cycle is related to the release.

[0034] ● Resource - A resource that can be CRUDed. ○ CIOS models each of the constructs listed above as resources. The following section will explain this modeling in detail.

[0035] ● Flock - A model of CIOS that encapsulates control areas and all components. It exists mainly to model the ownership of infrastructure components and to refer to such infrastructure components.

[0036] ● Flock Configuration - Describes all the infrastructure components, artifacts, and deployment tasks related to a single service.

[0037] ○ Each Flock has exactly one Flock Configuration. The Flock Configuration is checked in to obtain the source of control.

[0038] ○ Flock Configuration is declarative. It requires CIOS to provide the realm, region, advertisement, and version of the artifact as input.

[0039] ○ The granularity of Flock is fine - A Flock is composed of one service and supports the infrastructure.

[0040] ● State - A current snapshot of the state of all resources in a Flock. ● Release - A tuple composed of a specific version of the Flock Configuration and the specific versions of all the artifacts it references.

[0041] ○ Assume that the release describes a state that does not yet exist. ● Release Plan - A series of steps that CIOS takes to transition all regions from the current state to the state described by the release.

[0042] ○ The release plan has a finite number of steps and a clear start time and end time.

[0043] ● Application - This is a noun. One attempt to execute the release plan. The execution changes the current state of the Flock.

[0044] CIOS can be described as an orchestration layer that applies settings to downstream systems (e.g., globally). It is designed to enable the provisioning of global infrastructure and the deployment of code without manual intervention by the service team (e.g., in some cases, after initial approval). CIOS has a high level of responsibility, including but not limited to the following.

[0045] ● Provide the team with a view of the current state of the resources managed by CIOS, including in-flight change operations.

[0046] ● Assist the team in planning and releasing new changes. ● Coordinate operations between various downstream systems within a region to execute an approved release plan without human intervention.

[0047] ● Coordinate operations between regions / realms to execute an approved release plan globally.

[0048] In some cases, the CIOS addresses onboarding by enabling the team to provide the CIOS with configuration information via the checked-in code. In addition to this, the CIOS can automate other things, so it is an operation that is more important than the operation in previous implementations. In some cases, the CIOS addresses pre-deployment by providing the team with the ability to automatically deploy and test the code. In some cases, the CIOS can address the writing of the CM (Change Management) policy by enabling the team to automatically generate a plan for rolling out these artifacts (e.g., worldwide) when the team builds new artifacts. This can be done by inspecting the current state of each region and the current CIOS configuration (which can itself be an artifact). In addition to this, the team can inspect these plans and can iterate on them by changing the CIOS configuration and requesting that the CIOS re-make the plan. Once the team is satisfied with the plan, the team can create a "release" that references the plan. Then, the plan can be scored to determine approval or rejection. The team can continue to write CM, but these are just pointers to the CIOS plan. Thus, the team can reduce the time spent thinking about the plan. Since the plan is machine-generated, the accuracy is improved. The plan is too detailed for humans to understand, but it can be presented via an advanced UI (User Interface).

[0049] In some examples, the CIOS can handle the execution of the CM by automatically executing the deployment plan. Once the release plan is created and approved, the engineer no longer participates in the CM unless the CIOS starts a rollback. This may, in some cases, require the team to automate currently manual tasks. In some examples, the CIOS can handle the rollback of CM (Change Management) by automatically generating a plan to return the Flock to its original (e.g., pre-release) state when it detects a degradation of service health during operation. In some examples, the CIOS can handle the deployment of sudden changes / strategic changes by receiving a release plan that is limited in scope to a subset of regions and / or a subset of resources managed by the CIOS and then executing the plan.

[0050] In addition to this, the CIOS can support the primitives necessary to define fully automated worldwide deployments. For example, the CIOS can measure service health by monitoring alarms and running integration tests. The CIOS can assist the team in quickly defining rollback behavior when the service degrades and then automatically execute it. The CIOS can automatically generate and display release plans and track approvals. In some cases, the language used by the team to describe the desired deployment behavior may be declarative. The CIOS can combine code deployment capabilities with infrastructure provisioning (e.g., provisioning) into one system. Also, the CIOS supports flexible ordering between regions and between components within a region. The team can indicate the ordering via checked-in settings. The team can programmatically call the CIOS's planning API and release API.

[0051] Figure 1 shows an architecture 100 for explaining a technique for implementing at least CIOS Central 102. In some examples, CIOS Central 102 may be a service that addresses "Flock" level operations. CIOS Central 102 has several responsibilities as shown below, but is not limited to these.

[0052] ● Assume the responsibility as an authentication gateway for Flock metadata change and release operation.

[0053] ● Store a reliable mapping of Flock metadata in deployment artifacts and the CIOS repository of Flock.

[0054] ● Coordinate world-scale releases between phases and targets. ● Synchronization to enforce policies such as "Ongoing releases to Flock are limited to one at a time".

[0055] ● Detect changes to Flock settings and artifacts and trigger the generation of releases in response to the changes.

[0056] In some examples, the SCVMS (Source Code Version Control / Management Service) 104 can be configured to store reliable Flock settings, and CIOS Central 102 can subscribe to the ANS (Artifact Notification Service) 106, and CIOS Central 102 will be notified about the construction of new artifacts. Then, CIOS Central 102 can map the changes that reach the affected Flock and, if necessary, initiate release planning. In addition to this, in some examples, the APS (Artifact Push Service) can be called by CIOS Central 102 prior to release to the target, ensuring that all the artifacts required for a normal release are presented in the target region prior to the release.

[0057] In some examples, a customer (e.g., an engineer) 108 can call CIOS Central 102 for a Flock of CRUD and / or a release to view the status of ongoing CIOS activities. The Flock management service 110 can include one or more APIs for operating on the Flock, and the view / plan / approval service 112 can include CRUD APIs for creating and approving plans and for viewing a central copy of all the states of resources managed by CIOS. The change monitoring service 114 can watch the SCVMS 104 to confirm changes made to the Flock settings and can receive notifications about changes made to other artifacts from the ANS 106. The status Ingester service 116 can create a copy of the regional state in the CIOS Central DB (database) 118 so that the view / plan / approval 112 can publish it. In some examples, the CIOS Central DB 118 can be a DB for Flock, plans, and status. While Flock information can be reliable information, others can be old copies of data from the CIOS Regional 120. CIOS Central 102 can be configured to provide any suitable portion of a user interface (e.g., user interfaces 500 to 1300) and / or any number of such user interfaces for presenting any suitable data regarding Flock, releases, infrastructure components, artifacts, etc. In some embodiments, CIOS Central 102 can present data regarding one or more releases via any suitable interface. A release can include any suitable combination of tasks regarding one or more infrastructure components and / or one or more code changes to one or more applications (e.g., artifacts). Some examples of the user interfaces provided by CIOS Central 102 will be described later with reference to FIGS. 5 to 13.

[0058] In some examples, engineer 108 can execute API calls of Flock management service 110 (e.g., through ingress proxy fleet 122) to create a list of Flocks. The protocol for making such API calls may be, for example, HTTPS (Hypertext Transfer Protocol Secure). The relevant ACL (Access Control List) for this operation may include LAN (Local Area Network) 124 or other private connections. For example, instead of using the public Internet (e.g., dedicated connections, leased connections, and / or private connections) to connect the customer's on-premises data center or network to the CIOS, the CIOS can manage / control the network connectivity. In addition to this, authentication and authorization (e.g., of engineer 108) may be performed by a reservation system portal that enables a user to manage machine infrastructure (e.g., reservation services). In some cases, CIOS central 102 can store Flock metadata, plans, and status in central DB 118 using, for example, JDBC (Java Database Connectivity). In some examples, ANS 106 can be configured to notify change monitoring service 114 when a new artifact is published. ANS 106 may use HTTPS, and both authentication and authorization can be handled by mutual transport layer security services. In addition to this, in some cases, change monitoring service 114 can poll SCVMS 104 about changes to Flock settings. This polling can be performed using SSH (Secure Shell) or other protocols. Authentication of change monitoring service 114 may be handled by the CIOS system account, and authorization may be handled by SCVMS 104.

[0059] In some examples, engineer 108 can perform one or more of the following operations using the view / plan / approval service 112. Engineer 108 can perform a plan and / or approval by calling CIOS Central 102 to generate and approve a plan. Engineer 108 can view the status of ongoing worldwide CIOS operations by calling CIOS Central 102. In addition to this, engineer 108 can call CIOS Central 102 to view a snapshot of the status of worldwide resources managed by CIOS. These API calls (or similar) can be made via the HTTPS protocol or a similar protocol. In addition to this, the relevant ACLs can be controlled by LAN 124, and both authentication and authorization can be handled by the reservation service. In some examples, the view / plan / approval service 112 can request planning and drive plan approval for all regions of CIOS Regional 120 (using, for example, HTTPS, etc.). The relevant ACLs can be controlled using the security list managed by the WAN (Wide Area Network) gateway 126. Authentication can be handled by mutual transport layer security, and authorization can be handled by various ID policies. Furthermore, the state Ingester service 116 can watch CIOS Regional 120 to check for job status or state changes, and CIOS can provide these central views on demand (using, for example, also HTTPS, etc.). Also, the ACLs for this can be handled by the WAN gateway 126, and both authentication and authorization can be handled by the mutual transport layer security service.

[0060] Figure 2 is a diagram showing an architecture 200 for explaining techniques for implementing at least CIOS Regional 202. In some examples, in CIOS Regional 202, much of the declarative provisioning and planning, as well as the work of approved release applications, can occur. In some cases, each instance of CIOS Regional 202 may have a regional front end that can handle "execution target" level operations. It can be configured to perform the following operations.

[0061] ● Handle all CIOS authentications for operations received from CIOS Central 102.

[0062] ● Enforce the rule that for a given execution target, only one "execution" (planning / importing resources / applying a plan) may be in progress at a time.

[0063] ● Manage the storage of binary artifacts for declarative provisioning artifacts used in the input and output during the execution of declarative infrastructure provisioning. Examples of inputs include declarative infrastructure provisioning configuration files and input state files. The normal output is the final state file.

[0064] ● Request work from the CIOS executor for a given execution and poll for results from the CIOS executor.

[0065] In some cases, the CIOS front end may depend on a CIOS executor 206 (also referred to herein as a "scheduler") that can handle actual execution. The CIOS executor, in some examples, operates at the "execution" level and can perform the following operations.

[0066] ● Track a pool of available worker nodes. ●Query the received job requests and assign these requests when eligible workers become available.

[0067] ●Track the status of workers and the updates of execution for reporting to the client.

[0068] ●Detect dead nodes via the leasing protocol and, depending on the status of the tasks, be able to stop the functions of the tasks assigned to the dead nodes.

[0069] ●Provide facilities for canceling / force - terminating / pausing / resuming execution and map these to facilities to pass cancel / force - terminate / resume information to the worker nodes.

[0070] In some cases, the CIOS executor may depend on CIOS workers that can assign tasks to be executed by the workers and can provide facilities to enable the workers to update the progress of the job. The worker service operates at the granularity of "tasks". Each worker is an agent that executes the tasks assigned to that worker and reports the status and output of the tasks. Each worker can perform the following operations.

[0071] ●Poll the executor - worker API for the assigned work items and perform operations to match the assigned state to its local state.

[0072] ○Start a container for polling task items that do not exist locally. ○Force - terminate the container for locally executing a container that does not have the corresponding assigned task item.

[0073] ●Report the status of the job. ●Stage - in / out for the execution of the job container.

[0074] ● Start and monitor a declarative infrastructure provisioning container for performing the actual work of releasing for the execution target.

[0075] The CIOS worker may poll work from the worker endpoint of the CIOS executor and report results to the worker endpoint of the CIOS executor, depending on the CIOS executor. The worker may depend on the executor for all orchestration. Additionally, in addition to this, the CIOS worker may depend on CIOS Regional 202. In CIOS Regional 202, the worker service reads input from one or more APIs related to the regional front-end service and writes output to the API. Examples of input include configuration files, start state files, and import mappings. Examples of output include declarative provisioning processes, output of declarative provisioning state files, and import of result states.

[0076] In some examples, CIOS Regional 202 may be a regional service for managing regional instances / deployments of CIOS. CIOS Regional 202 covers the responsibility of storing and managing with authority the plans and the states associated with a particular region. Regional DB 204 may be a CIOS DB for the states and plans in a particular region. This is a reliable copy of a subset of the regions of Central DB 118 in FIG. 1. Scheduler 206 can take on the responsibility of managing the free capacity of the worker fleet, the responsibility of assigning tasks to the workers, and the responsibility of tracking the status of the tasks. In some cases, Task DB 208 may be another CIOS DB for the status of the tasks. The data in this DB is most often operational data. In addition to this, Worker 210 may be a fleet of JVM (Java (registered trademark) Virtual Machines) that manages the declarative provisioning images. These receive instructions from Scheduler 206 and communicate the results to both Scheduler 206 and CIOS Regional 202. CIOS Container 212 can execute the declarative provisioning operations in its own private Docker 214 container. This container does not need to contain secrets. In addition to this, in some examples, to avoid having secrets placed in the declarative provisioning images, Signature Proxy 216 can be configured to prevent secrets from leaking through the declarative provisioning tool. Instead, CIOS can perform request signing or start an mTLS (mutual Transport Layer Security) service with a proxy. Also, this facilitates the use of a FIPS-compliant cryptographic library.

[0077] In some examples, CIOS Central 102 can call CIOS Regional 202 to create a plan, drive approval, watch the job status (service principal), and extract the declarative provisioner state (service principal). The ingress proxy 218 can be configured as an ACL, and various ID policies can be used for authentication and authorization. Alternatively, in some examples, the ingress proxy 218 can be replaced with a load balancer configured to distribute the load of incoming requests, plans, etc. In some cases, CIOS Regional 202 may request the scheduler 206 to execute the declarative provisioner and perform the execution. The worker 210 can check with the scheduler 206 about what it should be doing and, upon completion, report the status to the scheduler 206. In some cases, mTLS can handle both the authentication and authorization of CIOS Regional 202 and the worker 210. In addition, when it is necessary to execute the declarative provisioner, the worker 210 performs the execution within the Docker container by interacting with the local Docker 214. Authentication at this stage can be handled by the local Unix® socket. Although the Docker protocol may be used in this last step, HTTPS can be used in the previous steps.

[0078] In some embodiments, CIOS Regional 202 may be configured to provide any suitable portion of a user interface (e.g., user interfaces 500-1300) and / or any number of such user interfaces for presenting any suitable data regarding Flock, releases, infrastructure components, artifacts, etc. In some embodiments, CIOS Regional 202 may present data regarding one or more releases via any suitable interface. A release may include any suitable combination of tasks regarding one or more infrastructure components and / or one or more code changes to one or more applications (e.g., artifacts). Some examples of user interfaces provided by CIOS Regional 202 are described below with reference to FIGS. 5-13.

[0079] In some examples, the CIOS container 212 enables the declarative provisioner to interact (via an API) with the signature proxy 216 while the signature proxy 216 is making calls to various CIOS services. The signature proxy 216 listens for one ephemeral port per call instance of the declarative provisioner that is known only to the declarative provisioner. The signature proxy 216 can initiate signature or mTLS requests and pass the declarative provisioner's calls to other CIOS services within the service enclave. Also, in some cases, the signature proxy 216 can communicate with one or more public CIOS services 220. For example, the signature proxy 216 uses the internal endpoint of the public service, if available. For services without an internal endpoint, an egress proxy 222 must be used to reach the external endpoint. This use of the signature proxy 216 may not be for inter-region communication. For example, the egress proxy whitelist in each region may be a whitelist only for the public IP range of that region. In some examples, the worker 210 can then hold the state and logs from the declarative provisioner in CIOS regional 202 so that they flow out to CIOS central 102.

[0080] When using CIOS, there are four representative phases of the customer experience: onboarding, pre-release, worldwide release, and strategic release. In the pre-release phase, examples of events that occur between the new artifacts being built and the artifacts being released to Release 1 (e.g., R1) are as follows. This will replace some or most of the current change management process. When relevant artifacts are built, CIOS can automatically generate releases using "all the latest versions in Flock". A release is a specific version of the Flock settings with specific inputs (e.g., artifact version, realm, region, and advertisement). A release includes one roll-forward plan per region and metadata describing the ordering of the regions. Each regional plan is a series of actions that a declarative provisioner can perform to realize the Flock settings in that region. Teams with a pre-release environment can use CIOS to automatically release and test software in that environment. Teams can configure CIOS to automatically test rollback plans. Teams will be able to inspect and approve releases through the CIOS UI. Teams can approve some, but not all, of the regional plans within a release. If the "all the latest versions" does not result in a suitable plan, the team can request CIOS to generate a plan for a carefully selected version of the artifact.

[0081] In the case of a worldwide release phase, an example of how the team would execute the next version of today's "normal CM" is as follows. When the release is approved, the CIOS drives the approved regional plan to each region. The CIOS operates independently within each region to apply the approved plan. The CIOS only performs a series of actions explicitly described in the plan for that region. It does not "think on its own" and stops functioning. The CIOS UI displays the progress of the execution to the team. If manual approval is required, the CIOS UI prompts the team. If the execution fails due to a service outage in the CIOS or downstream services, the CIOS can notify the team and display a prompt for the next step (e.g., abort, retry). The CIOS retries, but the service outage in the downstream system exceeds the willingness to retry. If the execution fails due to a degradation of service health or a test failure, the CIOS assists the team in rolling back Flock to its starting state. The CIOS notifies (e.g., calls) the team when it starts the automatic rollback. The team must approve this rollback plan, after which the CIOS executes it.

[0082] In the case of a strategic release phase, an example of how the team would execute the "new commercial" for tomorrow's version is as follows. When generating the plan, the team may request the CIOS to target the plan for specific resources in several ways (topologically (e.g., realm, region, advertisement, etc.), by resource type (e.g., "only metric setting" or "only deployment of deployment orchestration service", etc.), or in combination (e.g., separately) of the above). The team approves the strategic release in the same way as a worldwide release. The CIOS also orchestrates the strategic release. If the team needs to deploy a strategic release despite having a valid worldwide release, the CIOS will stop the execution of the worldwide release in the target region and then start the execution of the strategic release.

[0083] In some examples, the state of a declarative provisioner (e.g., usually a file) is a reliable record of the set of resources managed by the declarative provisioner. This includes the mapping of the logical identifier of each resource from the configuration file to the actual identifier of the resource. When the declarative provisioner creates a resource, the actual identifier may not be recorded in the state due to a particular type of failure. This occurs when the actual identifier is no longer that of the declarative provisioner. These may be referred to as "orphaned resources".

[0084] For most resources, orphaned resources represent garbage. A declarative provisioner starts an instance that it has forgotten (for example), and instead of starting that instance again the next time it runs, it starts a different instance. In the case of resources with uniqueness constraints or identifiers provided by the client, the declarative provisioner is unable to proceed without resolving the orphan. For example, if a declarative provisioner creates a user named "nglass" and that user is orphaned due to failure, the next time the declarative provisioner runs, it will attempt to create "nglass", but the creation will fail because a user with that username already exists. In some cases, orphaned resources are only a problem when adding new resources to the state. In some cases, the declarative provisioner's update operation can recover gracefully from failed update and delete records.

[0085] CIOS must be robust in the event of a downstream service outage or an outage of CIOS itself. Since CIOS can apply changes using declarative provisioners, it can be said to be robust with respect to running declarative provisioners and maintaining declarative provisioner state. The declarative provisioner provider performs "small-scale" retries sufficient to avoid outages that last for several minutes. For example, a cloud provider will retry for up to 30 minutes. If a downstream system outage lasts longer than 30 minutes, the declarative provisioner will stop functioning. When the declarative provisioner stops functioning, it records all the changes it made during normal operation and then terminates. For retries, CIOS must re-run the declarative provisioner. Also, re-running the declarative provisioner enables CIOS to be retried in the event of a CIOS outage. In some cases, CIOS can loop through the next operation.

[0086] ● Update - The declarative provisioner calls the GET API to retrieve the most recent snapshot of all resources described in the state.

[0087] ● Plan - Considering the current state updated most recently, the declarative provisioner generates a plan (a series of specific API calls) that can realize the desired state.

[0088] ● Apply - The declarative provisioner executes the series of steps included in the plan. CIOS can always execute all three of these steps when running the declarative provisioner. The update operation helps to assist with the recovery from an unrecorded update or deletion. CIOS inspects the results of implementing the plan and compares them with the approved release plan. If an operation not included in the approved release plan is included in the newly generated plan, CIOS may stop functioning and may notify the service team.

[0089] Figure 3 shows a DAG (Directed Acyclic Graph) 300 for explaining an exemplary Flock302. From the deployment of the first test environment to the deployment of the last production environment, the progress of the code / settings from check-in to production for one Flock setting in CIOS can be described. Internally, CIOS calls each element in the progress of the ET (Execution Target). This exists everywhere in the internal API but does not leak into the Flock setting. CIOS executes the ET based on the DAG200 defined in the Flock setting. Each ET (for example, ET-1, ET-2, ET-3, ET-4, ET-5, ET-6, and ET-7) is generally one copy of the service described by the Flock setting.

[0090] Figure 4 is a DAG 400 for explaining an exemplary Flock 402. In the Flock configuration, the CIOS is very arbitrary about how the team represents this progress. The team must model the progress using the tenancy and region of the cloud infrastructure. The team should not model the progress using realms. The CIOS permits the team to utilize many tenancies within a realm and many regions within a tenancy. However, the CIOS prohibits the team from using the same region twice within one tenancy (although it is possible to use the same region twice within one realm in different tenancies). DAG 400 shows a kind of DAG 300 of FIG. 3 represented using tenancy and region. This embodiment is an example of an overlay service where the ET of the pre-production environment is in the production environment region. Service Enclave Service Release 1 has unstable tenancies and stable tenancies. In DAG 400, IAD is the local airport code of Dallas Airport in Washington, D.C., YYZ is the local airport code of Toronto in Ontario, PHX, LHR, and FRA are the local airport codes of Phoenix, London, and Frankfurt respectively, and LUF and LFI are the airport codes of two different air force bases.

[0091] In one embodiment, the CIOS and / or other techniques described herein are improved versions of each of Terraform (a declarative provisioning tool), Tanden (a code generation tool), and ODO (Oracle Deployment Orchestrator). In addition to this, in some examples, the CIOS and / or other techniques described herein can be implemented using at least a part of the Terraform, Tanden, and ODO tools.

[0092] FIG. 5 is a UI (user interface) 500 that presents information regarding a release that includes a plurality of phases and a plurality of execution targets, according to at least one embodiment. The UI 500 may include a phase area 502 and an execution target area 504. The phase area 502 may show an ordered list of phases, such as phase 506 (e.g., "R_n"), phase 507 (e.g., "R_s"), phase 508 (e.g., "R_e"), and phase 509 (e.g., "R_w"). In some embodiments, the order of the phases from left to right may be displayed. Here, the leftmost phase is completed before the phases on the right are completed, and the phases on the right are completed before the phases on the right, and so on. In some embodiments, the ordered list of phases may be stored in any suitable data structure, such as a linked list. An exemplary linked list will be described in more detail with reference to FIG. 7 later.

[0093] Each phase may be associated with a plurality of tasks (e.g., a plurality of tasks including deploying one or more infrastructure resources of one or more execution targets). The list of phases shown in UI500 includes four phases, but the phase area 502 may include any suitable number of phases for deploying infrastructure resources at one or more execution targets. In some embodiments, the ordered list of phases presented within the phase area 502 may be scrollable horizontally. The phase area 502 may indicate the number of phases 510, the status 511, the number of completed execution targets and / or the number of execution targets 513, and the Flock configuration identifier 514. The number of phases 510 may indicate the number of phases included in the linked list, and the status 511 may indicate the status of the release. As shown in FIG. 5, the status 511 may indicate "applying", but the status 511 may be any suitable status indicator (e.g., "before start", "completed", "failed", etc.). As shown in FIG. 5, the number of completed execution targets and / or the number of execution targets 513 indicates that 24 out of a total of 57 deployments for the execution targets have been completed. The number of completed execution targets and / or the number of execution targets 512 may be presented on the UI500 in any suitable manner. The Flock configuration identifier 514 is presented as "13380fb2832" on the UI500, but the Flock configuration 514 may be presented on the UI500 in any suitable manner for conveying the unique identifier of the Flock configuration file corresponding to the release shown in FIG. 5.

[0094] Each sub - area corresponding to a phase may include a phase identifier (e.g., phase identifier 516), a total execution target number indicator (e.g., total execution target number indicator 518), timestamp information (e.g., timestamp information 520), an execution target tracker area (e.g., execution target tracker area 522), and other suitable information regarding the phase. For example, the identifier 516 (e.g., "R_s") may be any suitable alphanumeric string for uniquely identifying the phase. In some embodiments, the total execution target number indicator 518 may be presented adjacent to the identifier 516. The total execution target number indicator 518 indicates the total number of execution targets associated with a given phase (e.g., 14).

[0095] The timestamp information 520 may include a start time and / or an end time respectively related to the time when the phase started and / or completed. In some embodiments, the timestamp information 520 may include any suitable indicator (e.g., "before start", "failed", "completed", etc.) for indicating the status of the corresponding phase. In some embodiments, any suitable information regarding the phase may be shown in the phase area 502. Information regarding each phase may be presented similarly, and the presentation of the phase information may be in a different format and / or may include more or less information than the information shown in FIG. 5.

[0096] An execution target tracker area (e.g., the execution target tracker area 522 in phase 506) can be presented within each phase. The execution target tracker area for each phase may include one or more execution target indicators (e.g., execution target indicators 524, 526, and 528). Each execution target indicator may include a number indicating the number of tasks (e.g., deployment to the corresponding execution target) to be executed simultaneously. As an example, execution target indicator 524 indicates that a deployment to a specific execution target can be executed. That execution target indicator 526 is located to the right of execution target indicator 524 indicates that after completion of the first task corresponding to execution target indicator 524, another deployment to a different execution target is executed. That execution target indicator 528 is located to the right of execution target indicator 526 indicates that after completion of the second task corresponding to execution target indicator 524, twelve separate deployments to twelve different execution targets are executed. The combination of execution target indicators 524, 526, and 528 collectively shows a folded form of a directed acyclic graph (e.g., DAG 900 described later in FIG. 9).

[0097] In some embodiments, the execution target indicators 524-528 may correspond to a list of execution targets related to a phase and a data structure configured to maintain the order in which tasks (e.g., deployments to each execution target) are performed. Details of an exemplary data structure for maintaining this list and order will be described later in connection with FIG. 9. Each execution target indicator may include a ring (e.g., ring 530). The ring may be divided into any suitable number of segments corresponding to the number of execution targets associated with the execution target indicator. In the illustrated example, ring 530 may be divided into 12 equal segments. Each segment of ring 530 may correspond to a particular execution target and may be colored with a color corresponding to the status of the task corresponding to that particular execution target. As an example, ring 530 may indicate (e.g., by nine green segments) that nine deployments to the execution target have been completed. The remaining three segments of ring 530 (e.g., colored white) may indicate that the three tasks corresponding to the three execution targets have not yet been started. As another example, the remaining three segments of ring 530 may be colored with different colors (e.g., yellow) to indicate that the corresponding tasks have started but have not yet been completed. Thus, by looking at ring 530, the user can quickly and visually identify the current status of the tasks associated with execution target indicator 528. By looking at the execution target indicators within execution target tracker area 522, the user can ascertain that deployments to 11 execution targets have been completed and 3 are in progress. In some embodiments, each execution target indicator may correspond to a node of a DAG (directed acyclic graph) (e.g., DAG 900 of FIG. 9) associated with a given phase (e.g., "R_s").

[0098] The execution target area 504 may include an execution target column 532, a progress column 534, and an operation column 536. The execution target column 532 may include a list of targets (e.g., execution targets) corresponding to tasks (e.g., deployments) executed in relation to a given phase (e.g., phase 506 as shown). In some embodiments, the execution target column 532 may be stored in chronological order and may be stored in any other suitable manner for storing task executions. The progress column 534 may include a list of progress indicators indicating the status (e.g., "normal completion", "before start", "in progress", etc.) corresponding to each execution target. The operation column 536 may include a list of operations to be performed with respect to the execution targets indicated in the execution target column 532. For example, the list of operations included in the operation column 536 may include CRUD (create, read, update, and delete) operations and the like.

[0099] FIG. 6 is an exemplary code segment 600 for defining a list and order of phases according to at least one embodiment. As shown in FIG. 6, four phases are defined in code segment 600. Each phase is defined as a resource of the type "phase" and an identifier (e.g., "R_n", "R_s", "R_e", and "R_n") is assigned. As shown in code segment 600, resources 602, 604, 606, and 608 correspond to their respective phases. Each phase may include one or more variables, and one of the variables may include an indicator for indicating the execution order of each phase. In some embodiments, the indicator may indicate the presence and / or absence of a dependency on one or more other phases. As an example, indicator 618 (e.g., predecessors=[] on the fifth line) may indicate no relationship with other phases. This may be interpreted as defining that the first phase is to be executed by the system. Indicator 620 (e.g., predecessors=[phase.R_n.variable1]) may be used to indicate a dependency on another phase (e.g., phase "R_n") by including an assignment of a value corresponding to a variable associated with another phase (e.g., phase "R_n"). Similarly, indicator 622 may indicate a dependency on another phase (e.g., "R_s") (e.g., by including an assignment of a value corresponding to a variable associated with "R_s"), and indicator 624 may indicate a dependency on yet another phase (phase "R_e") (by including an assignment of a value corresponding to a variable associated with "R_e").

[0100] In some embodiments, the configuration file corresponding to the release may include the code segment 600. As part of the execution of the preprocessing procedure, the configuration file may be parsed / traversed one or more times. In one of these traversals, the code segment 600 may be analyzed to generate a data structure that can hold the execution IDs and order corresponding to one or more phases. FIG. 7 shows an example (e.g., a linked list) of a data structure that can be used to hold this information.

[0101] Indicators 618-624 may be used to construct the specific order in which the phases are executed. For example, as shown in FIG. 6, the phase "R_n" is executed first, followed by the phases "R_s", "R_e", and then "R_w". It will be appreciated that one or more other phases that complete before the tasks associated with a given phase begin may indicate dependencies. Although four phases are defined in FIG. 6, it will be appreciated that any suitable number of phases may be defined similarly.

[0102] FIG. 7 is an exemplary data structure (e.g., linked list 700) generated by a CIOS (Cloud Infrastructure Orchestration Service) to maintain a list and order associated with one or more phases. The linked list 700 can be generated to identify the phases and execution order defined in the code segment 600 of FIG. 6. As shown in FIG. 7, the linked list 700 includes four nodes 702, 704, 706, and 708. The nodes 702, 704, 706, and 708 can each correspond to one of the four phases defined in the code segment 600. Each node of the linked list can correspond to a data object configured to store any appropriate information corresponding to a given phase. As an example, a given node can store any appropriate number of variables, identifiers, data structures, pointers, references, etc. corresponding to a particular phase. As a non-limiting example, the node 702 corresponding to the phase "R_n" can store the three variables defined in the code segment 600 as corresponding to the phase "R_n". Each node of the linked list 700 may include any appropriate number of variables corresponding to a given phase.

[0103] Each node of the linked list may include a pointer / reference to another node in the linked list (if there are multiple phases). As an example, the node 702 may include a reference to the node 704. The node 704 may include a reference to the node 706. The node 706 may include a reference to the node 708. The node 708 may indicate (e.g., by a null pointer) that it is the end node of the linked list 700. In some embodiments, these pointers / references can be identified based on the indicators 618-624 of the code segment 600.

[0104] In some embodiments, at runtime, CIOS may identify the order in which specific phases are to be executed based at least in part on traversing the linked list 700 starting from node 702 (e.g., the start node). For the execution of these phases, any suitable combination of data stored within each corresponding node may be used. When the operations corresponding to a given phase are completed, CIOS may traverse to the next phase and repeat this process any suitable number of times until the operations corresponding to the end node (e.g., node 708) of the linked list 700 are completed. In some embodiments, if the operations of a given node are not performed successfully (e.g., an error occurs), CIOS may not need to traverse to the next node and instead may abort the deployment and return a notification alerting the user of this situation.

[0105] Each node of the linked list 700 may correspond to a data structure configured to identify and hold the execution order for one or more execution targets. In FIGS. 8 and 9, the definition and usage of such a data structure will be described in more detail. It will be appreciated that the linked list 700 may provide the information utilized by the UI 500 of FIG. 5 to present any suitable information related to the phase (e.g., identifier 516, timestamp information 520, execution target 518, etc.).

[0106] FIG. 8 is an exemplary code segment 800 for defining a list and order of execution targets (e.g., execution targets corresponding to specific phases) according to at least one embodiment. As shown in FIG. 8, four execution targets are defined in code segment 800. Each execution target is defined as a resource of the type "execution target" and is assigned an identifier (e.g., "us-la", "us-sj", "us-sf", and "us-sd"). As shown in code segment 800, resources 802-808 each correspond to a unique execution target. Each execution target may be associated with one or more variables, and one of the variables may include an indicator for indicating the execution order of each phase. In some embodiments, the indicator may indicate the presence and / or absence of one or more dependencies on one or more other execution targets. As an example, indicator 818 (e.g., predecessors=[] on the fifth line) may indicate no relationship with other phases. This may be interpreted as defining that the first execution target is to be executed by the system. Indicator 620 (e.g., predecessors=[execution_target.us-la.variable1]) may indicate a dependency on another execution (e.g., phase "us-la") by including an assignment of a value corresponding to a variable associated with another execution target (e.g., execution target "us-la"). Similarly, indicator 822 may indicate a dependency on another phase (e.g., execution target "us-sj") by including an assignment of a value corresponding to a variable associated with the execution target "us-sj", and indicator 624 may indicate a dependency on yet another phase (e.g., execution target "us-sf") by including an assignment of a value corresponding to a variable associated with the execution target "us-sf").

[0107] In some embodiments, the configuration file corresponding to the release (e.g., the same or different configuration file including code segment 800) may include code segment 800. As part of the execution of the preprocessing procedure, the configuration file may be parsed / traversed one or more times. In one of these traversals, code segment 800 may be analyzed to generate a data structure that can hold the ID and order of the execution targets corresponding to a given phase. FIG. 9 shows an example of a data structure (e.g., a DAG (Directed Acyclic Graph)) that can be used to hold this information.

[0108] Indicators 818-824 may be used to construct the specific order in which the execution targets are executed. For example, as shown in FIG. 8, the tasks associated with the execution target "us-la" are executed first, followed by the tasks associated with the execution target "us-sj", then the tasks associated with the execution target "us-sf", and finally the tasks associated with the execution target "us-sd". It will be appreciated that one or more other execution targets whose corresponding tasks are completed before the tasks associated with a given execution target start can be indicated by dependencies. Although four execution targets are defined in FIG. 8, it will be appreciated that any suitable number of phases can be defined similarly. In some embodiments, the execution targets may share a common dependency (e.g., the definition of the same predecessors). Using the common dependency, it can be shown that the tasks associated with the execution targets sharing the common dependency can be executed simultaneously.

[0109] FIG. 9 is an exemplary data structure (e.g., a directed acyclic graph (DAG) 900) that a CIOS (Cloud Infrastructure Orchestration Service) can generate to hold a list and order related to one or more execution targets related to a phase, according to at least one embodiment. DAG 900 may be one of a plurality of DAGs generated in response to a configuration file related to a release by CIOS being parsed one or more times. Each node of DAG 900 may correspond to one execution target. As shown in FIG. 9, DAG 900 includes six nodes (e.g., nodes 902, 904, 906, 908, 910, and 912). These nodes may each correspond to one of six execution targets defined in a manner similar to the method described in code segment 800 of FIG. 8. Each node of DAG 900 may correspond to a data object configured to store any appropriate information corresponding to a given execution target. As an example, a given node may store any appropriate number of variables, identifiers, data structures, pointers, references, etc. corresponding to a particular execution target.

[0110] Each node of DAG900 may include a pointer / reference to one or more nodes in DAG90. As an example, node 902 may include a reference to node 904. Node 904 may include references to nodes 906 - 910. Each of nodes 906 - 910 may include a reference to node 912. Node 912 may indicate (e.g., by a null pointer) that it is an end node of DAG900. In some embodiments, these pointers / references may be identified based on indicators similar to indicators 818 - 824 described above with respect to FIG. 8. In some embodiments, nodes 906, 908, and 910 may include a common dependency on node 904, so that tasks associated with nodes 906 - 910 may be executed at least partially simultaneously. In some embodiments, node 912 may correspond to an execution target that depends on nodes 906 - 910. Thus, tasks associated with the execution target corresponding to node 912 may be executed only after all tasks associated with the execution targets corresponding to nodes 906 - 910 are completed.

[0111] In some embodiments, DAG900 may be associated with the nodes of one linked list 700 described above with respect to FIG. 7. That is, one or more execution targets (e.g., identified and represented by nodes of DAG900) may be associated with a particular phase (e.g., one node) of linked list 700. In some embodiments, the nodes of linked list 700 may include a reference to DAG900, or one or more nodes of DAG900 may include a reference to the nodes of linked list 700.

[0112] In some embodiments, the CIOS may generate a DAG 900 across a configuration file (e.g., including code segment 800 of FIG. 8), and the generation of the DAG 900 may be completed as part of a preprocessing procedure executed before or during runtime. When the operations corresponding to a given execution target are completed, the CIOS may cross to the next execution target(s) and repeat this process any suitable number of times until the operations corresponding to one or more end nodes (e.g., node 912) of the DAG 900 are completed. In some embodiments, if the operations corresponding to a given node are not performed successfully (e.g., an error occurs), the CIOS may not cross to the next node and instead may return a notification alerting the user of this situation.

[0113] Each node of the DAG900 may correspond to a data structure configured to identify and hold an execution order corresponding to one or more resources (e.g., services, software modules, etc.). In FIGS. 10-12, the definition and usage of such a data structure (e.g., the DAG of capabilities) will be described in more detail. As an example, each node of the DAG900 may correspond to another DAG (e.g., DAG1200 in FIG. 12, which shows a list and order of resources and / or capabilities and identifies the order of task execution). It will be understood that the DAG900 may provide the information utilized by the UI500 of FIG. 5 to present any suitable information related to the execution target indicators 524-528. Thus, the information shown in the execution target tracking area 522 may represent a simplified version of the DAG900 (the DAG corresponding to a given phase such as phase "R_s"). In the simplified version of the DAG, the concurrently executable nodes of the DAG900 may be condensed into one node (see, for example, the execution target indicator 528 indicating that 12 nodes of the DAG are condensed into one node). In some embodiments, the CIOS may deploy infrastructure resources and / or release software artifacts based at least in part on traversing the DAG900. Specific tasks and the specific order of tasks are identified as described with respect to FIGS. 10-12.

[0114] FIG. 10 is an exemplary code segment 1000 for constructing explicit and implicit dependencies among resources of an execution target according to at least one embodiment. As shown in FIG. 10, the code segment 1000 includes two modules 1002 and 1004 and a resource 1006. The modules 1002 and 1004 each include names 1008 and 1010 shown as "apps_example1" and "apps_example2", respectively. The modules may include names of any appropriate length containing any appropriate alphanumeric characters (multiple allowed). The modules 1002 and 1004 may define the applications / services that the user wants to launch or provision. Using the modules 1002 and 1004, applications can be deployed to the available domain 1 and the available domain 2, respectively. The resource 1006 can include a multi-parameter list including a resource type 1012 shown as "type" in FIG. 10 and a resource name 1014 shown as "executor" in FIG. 10. The resource type 1012 may be any type suitable for deployment, and the resource name 1014 may be any name.

[0115] Resource 1006 may be an ability and may include implicit dependencies, explicit dependencies, or both. As shown in code segment 1000, Resource 1006 attempts to assign the variable "variable1" to a value equal to "module.apps_example1.variable1" and a value accessible via the module apps_example1 (e.g., module 1002). This is intended to indicate an implicit dependency relationship formed between Resource 1006 and Module 1002. The formed implicit dependency relationship may prevent Resource 1006 from operating until Module 1002 completes deployment. The process responsible for starting Resource 1006 may receive a notification that Module 1002 has completed deployment. This notification may be sent by a scheduler (e.g., scheduler 206 in FIG. 2) and may be received by the process responsible for starting Resource 1006. Since Resource 1006 shown in FIG. 10 does not directly define the dependency relationship between Module 1002 and Resource 1006, the formed implicit dependency relationship is considered to be implicit.

[0116] In contrast, resource 1006 includes an explicit dependency that defines the dependency between resource 1006 and the app_example2 module explicitly at line 29 of FIG. 10. As shown in line 29 of code segment 1000, resource 1006 includes "depends_on=apps_example2.variable2". Based on the code in line 29, an explicit dependency may be formed, and the explicit dependency may prevent resource 1006 from being deployed until apps_example2 is successfully deployed. When apps_example2 is successfully deployed, a notification may be sent by the scheduler and received by the process responsible for deploying resource 1006. Note that code segment 1000 in FIG. 10 includes one resource 1006 that contains one implicit dependency and one explicit dependency. However, those skilled in the art will understand that any combination of resource 1006, implicit dependencies, and explicit dependencies may be used to achieve the goals of CIOS users.

[0117] To analyze the configuration file containing code segment 1000, CIOS (or a declarative infrastructure provisioner such as the declarative provisioning tool for CIOS described above) may be utilized. Through this analysis, CIOS (or the declarative provisioning provisioner) can generate a DAG (Directed Acyclic Graph) for each resource, module, and / or capability that compiles and defines an ordered list of dependencies on other resources, modules, and / or capabilities. During an attempt to deploy a resource, CIOS can traverse the DAG to identify when a resource depends on another resource, module, and / or capability. The DAG for each resource may specify implicit dependencies, explicit dependencies, or a combination thereof, and may be used to start or deploy the corresponding resource having CIOS.

[0118] FIG. 11 is an exemplary code segment 1100 for constructing explicit and implicit dependencies among resources of an execution target according to at least one embodiment. The code segment 1100 shown in FIG. 11 includes four resources 1102, 1104, 1106, and 1108. Each of the resources 1102, 1104, 1106, and 1108 may correspond to an ability and may include implicit dependencies, explicit dependencies, or a combination thereof.

[0119] The resource 1102 shown in FIG. 11 includes a name 1110 shown as "object_storage", a type 1112 shown as "type1" (e.g., indicating that the resource is a capability), and a plurality of variables (e.g., variables 1 to 3. However, more or fewer variables may be used). The resource 1104 shown in FIG. 11 includes a name 1116 shown as "worker" and an explicit dependency 1118 shown as "depends_on=type1.object_storage". The parameter list of the resource 1102 includes an identifier 1115. The identifier 1115 can be used to reference that resource (or the name 1110 can be used as well). The statement 1118 forms an explicit dependency on the resource 1102 by reference to type1.object_storage. Due to this explicit dependency, the resource 1104 may not be deployed until the resource 1102 completes the deployment. The resource 1106 shown in FIG. 11 includes an identifier 1120 shown as "peacock", a type (e.g., "type1" indicating a capability), and a plurality of variables (e.g., variable1 and variable2). The resource 1108 shown in FIG. 11 includes a type (e.g., "type4"), a name 1124 shown as "LB", and a statement 1126 ("count=type1.peacock.exists."). By using the statement 1126, an implicit dependency on the resource 1106 can be formed. The resource 1108 does not use an explicit dependency construct (e.g., "depends_on"), but there is an implicit dependency because it attempts to assign a value equal to whether the capability "peacock" exists (determined from the statement type1.peacock.exists) to the variable "count". Therefore, since there is an assignment attempted on line 18, the resource 1108 may not be deployed until the resource 1106 "peacock" is deployed.The code segment 1100 of FIG. 11 includes four resources 1102, 1104, 1106, and 1108 that include one implicit dependency and one explicit dependency. However, those skilled in the art will understand that any combination of resources, implicit dependencies, and explicit dependencies may be used to achieve the goals of the CIOS user.

[0120] CIOS (or a declarative infrastructure provisioner such as the declarative provisioning tool of CIOS described above) may be used to parse a configuration file that includes the code segment 1100. Through this parsing, CIOS (or the declarative provisioning provisioner) may generate a DAG (directed acyclic graph) for each resource, module, and / or capability. The resources, modules, and / or capabilities compile and define an ordered list of dependencies on other resources, modules, and / or capabilities. While attempting to deploy a resource, CIOS may traverse the DAG to identify when a resource depends on another resource, module, and / or the capabilities of another resource. The DAG for each resource may specify an implicit dependency, an explicit dependency, or a combination thereof, and may be used to start or deploy the corresponding resource having CIOS.

[0121] FIG. 12 is an exemplary DAG (directed acyclic graph) 1200 corresponding to a resource (e.g., resource A) of a cloud computing system according to at least one embodiment. As shown in the figure, the DAG 1200 may be a finite directed graph that includes any suitable number of nodes (e.g., six nodes shown in FIG. 12) and edges (e.g., seven edges shown in FIG. 12). Each edge is directed from one node to another as shown in FIG. 12. The nodes and edges may be arranged so as not to form a directed cycle. That is, the DAG 1200 is defined such that after starting from any node, a consistent series of directed edges will not finally cycle back to the same node. The last node (e.g., node "6") may point to a null value or may indicate the end point of the DAG.

[0122] DAG 1200 shows six nodes and seven edges, although a DAG may include any suitable number of nodes and directed edges. In some embodiments, each node corresponds to a series of operations (e.g., operations for performing tasks such as deploying and / or starting a resource such as resource A) or a series of capabilities on which the next node that depends on a plurality of operations depends. Each directed edge of each DAG defines the order in which these operations are executed and / or the dependency between a subset of the operations associated with the node and a subset of the capabilities associated with the node immediately preceding that node.

[0123] As a simple example, nodes 1, 2, 5, and 6 of DAG 1200 represent nodes corresponding to four separate series of operations. Based on the edges shown in FIG. 1200, the operations of each node are executed in an order corresponding to the order of nodes 1, 2, 5, and then 6. Nodes 3 and 4 represent a plurality of nodes that individually match one or more dependencies. As an example, node 3 may correspond to a dependency of the operation corresponding to node 5 on a capability associated with a different resource (e.g., resource B). Similarly, node 4 may correspond to a dependency of the operation corresponding to node 5 on a capability associated with a different resource (e.g., resource C). In some embodiments, different capability nodes (e.g., nodes that identify the dependency of a particular resource on one or more capabilities) may be used for different resources, and one node may be used to specify all dependencies regardless of how many resources the dependency references. Thus, in some embodiments, the dependency corresponding to resource B (identified, for example, at node 3) and the dependency corresponding to resource C (identified, for example, at node 4) may be combined into one node.

[0124] DAG1200 can orchestrate the execution of operations to launch and / or deploy resources in a cloud computing environment, in a way that is more detailed with respect to FIGS. 10-12, in relation to one or more dependencies of a resource on the capabilities of other resources (or the other resources themselves).

[0125] FIG. 13 is a flowchart showing an exemplary process 1300 for orchestrating the execution of a task (e.g., deploying a resource) that includes a dependency on at least one capability (e.g., the capabilities of different resources) according to at least one embodiment. As shown in FIG. 13, the process flow 1300 includes a scheduler 1302 (e.g., scheduler 206 of FIG. 2), a worker 1304 (e.g., worker 210 of FIG. 2), and an IP process 1306 (e.g., CIOS container 212 of FIG. 2).

[0126] At 1308, the scheduler 1302 may receive a task for deploying infrastructure resources in a region, and the scheduler 1302 may send data related to the task to the worker 1304. In some embodiments, the scheduler 1302 may instantiate the worker 1304 to handle the deployment of resources (e.g., services).

[0127] At 1310, the worker 1304 may instantiate the IP process 1306. The IP process 1306 may be configured to execute an instance of a declarative infrastructure provisioner (e.g., the declarative provisioning tool, Terraform, described above).

[0128] At 1312, the IP process 1306 may analyze a configuration file related to deployment (e.g., a configuration file including code segments 1000 and / or 1100 of FIGS. 10 and 11) to generate a DAG (Directed Acyclic Graph) of specific resources. Through the analysis of the configuration, the IP process 1306 (declarative infrastructure provisioner) may identify any appropriate number of implicit and / or explicit dependencies of other resources on capabilities. Once identified, the IP process 1306 constructs a DAG (e.g., according to the implicit and / or explicit dependencies identified during the analysis) that specifies tasks for starting and / or deploying resources having one or more nodes corresponding to the capabilities on which the resources depend.

[0129] At 1314, the IP process 1306 may initiate traversal of the DAG and, upon reaching various nodes of the DAG, execute at least a portion of the deployment and / or startup of the specific resources. Any appropriate operations may be performed according to at least one node of the DAG to make available a portion of the functionality corresponding to the resources. Multiple portions of the functionality corresponding to the resources may become available. In some embodiments, the IP process 1306 may send a notification (not shown) to the scheduler 1302 indicating that one or more capabilities of the resources are currently available. At least one of the nodes of the DAG may correspond to the capabilities of one or more other resources. Upon reaching these types of nodes, the IP process 1306 may check whether the capabilities are available. If available, the IP process 1306 may proceed with the traversal of the DAG.

[0130] At 1316, the IP process 1306 may reach a node of the DAG corresponding to one or more capabilities of one or more other resources. In some embodiments, the IP process 1306 may determine that at least one capability associated with the node is not yet available.

[0131] In 1320, in response to determining that at least one capability associated with a node is unavailable, IP process 1306 may send data indicating that one or more capabilities on which the resources corresponding to IP process 1306 depend are unavailable, to scheduler 1302.

[0132] In 1322, optionally, after storing status information indicating which operations and / or which nodes of the DAG are complete and / or status information indicating which specific node of the DAG IP process 1306 last accessed, IP process 1306 may terminate. IP process 1306 terminates, is force-terminated, is paused once, or execution is aborted.

[0133] In 1324, scheduler 1302 may store information indicating that a particular resource was waiting for one or more specific capabilities required for the purpose of resuming startup and / or deployment.

[0134] In 1326, scheduler 1302 may receive one or more notifications informing that one or more capabilities that a resource was waiting for have become available. In some embodiments, scheduler 1302 may receive various notifications about various capabilities of the corresponding resource from other IP processes when these become available. Scheduler 1302 may maintain one or more records of these various available capabilities and / or one or more records of the various capabilities that a resource is currently waiting for. Scheduler 1302 may identify from these one or more records that a particular one / more capabilities that the resources corresponding to IP process 1306 are waiting for have become available. Thus, scheduler 1302 may proceed to 1328.

[0135] In 1328, in response to determining that the capabilities on which the resources corresponding to IP process 1306 depend have become available, scheduler 1302 may return to step 1308. In step 1308, scheduler 1302 sends data related to the original task (e.g., deploying a resource) to worker 1304. In some embodiments, scheduler 1302 may instantiate a new worker or, using the previous worker 1304 (shown), continue to handle tasks related to the resource. Worker 1304 may instantiate an IP process (not shown) configured to execute and parse a configuration file to generate a DAG for the resource. The IP process may access the stored state information to identify the node last accessed in the DAG (e.g., the node corresponding to one or more capabilities for which the resource was waiting for a response). Since one or more capabilities have become available, the IP process may proceed across the DAG in a manner similar to the method described above, performing the operations at each node, executing a portion of the task, or checking the capabilities on which the next portion of the task depends until the operation of the end node of the DAG is complete.

[0136] For all resources of a task, a process similar to the process described above may be executed. As an example, when deploying a system with multiple resources (e.g., multiple services), process 1300 may be executed for each resource in place of each resource to deploy each resource of the system.

[0137] Figure 14 is an exemplary process flow 1400 for performing a release (e.g., by a CIOS) according to at least one embodiment. At event number 1, a scheduler 1402 (e.g., scheduler 206 of FIG. 2) may send a task to a worker 1404 (e.g., worker 210 of FIG. 2). This task may include deploying a computing system or a subset thereof, such as deploying infrastructure resources to a set of execution targets. The task may need to traverse a linked list (e.g., an example of the linked list 700 of FIG. 7), a DAG (e.g., DAGs 900 and 1200 of FIGS. 9 and 12, respectively), a combination thereof, or other suitable tasks for deploying a computing system. Worker 1404 may receive this task from scheduler 206. Worker 1404 may be one of the fleet of worker nodes. The fleet of worker nodes may include any suitable number of worker nodes for deploying a computing system. Scheduler 1402 may select worker 1404 based at least in part on the capacity of worker 1404. For example, if the computing power of worker 1404 is the highest among the fleet of worker nodes, scheduler 1402 may select to send the task to worker 1404.

[0138] At event number 2, worker 1404 may parse / traverse the configuration file 1406 one or more times. The configuration file 1406 may include instructions for deploying a computing system, and by performing the parsing one or more times, resources or other capabilities that are desired to be launched or otherwise deployed for deploying the computing system may be identified. By way of non-limiting example, the configuration file 1406 may include the respective code segments 600, 800, 1000, and 1100 of FIGS. 6, 8, 10, and 11.

[0139] In event number 3, the information from the configuration file 1406 can be sent to the IP process 1408 (e.g., the IP process 212 in FIG. 2). The IP process 1408 can receive the information from the configuration file 1406 based on one or more analyzes / traversals performed by the worker 1404. This information may include a set of capabilities, execution targets, or other suitable resources for deploying a computing system.

[0140] In event number 4, in response to receiving this information from the configuration file 1406, the IP process 1408 can determine the order in which capabilities, or other suitable resources for deploying a computing system, are deployed. The IP process 1408 can generate a concatenated list 1410 of phases (e.g., an example of the concatenated list 700 in FIG. 7), a DAG 1412 of execution targets (e.g., an example of the DAG 900 in FIG. 9), a DAG 1414 of capabilities (e.g., an example of the DAG 1200 in FIG. 12), or other suitable lists, graphs, or data structures. The concatenated list 1410 of phases, the DAG 1412 of execution targets, and the DAG 1414 of capabilities (collectively referred to as the "release data structures") can be generated in any suitable order.

[0141] The release data structure can be used to identify and determine the order for performing the tasks of a release. For example, each node of the linked list 1410 of phases corresponds to a separate instance of the DAG of an execution target (e.g., an example of the DAG 1412 of an execution target). Here, each node of the DAG of an execution target corresponds to a DAG of capabilities (e.g., an example of the DAG 1414 of capabilities). The IP process 1408 can identify the DAG of the corresponding execution target starting from the first node of the linked list 1400. Using the first node of the DAG of an execution target, the corresponding DAG of capabilities can be identified. According to the DAG of capabilities, the tasks related to that DAG of capabilities may be executed, and upon completion, the IP process 1408 can traverse to the next node of the DAG of an execution target and identify the next corresponding DAG of capabilities. It may traverse each node of the DAG of an execution target, and when the tasks corresponding to these nodes are completed, the IP process 1408 can traverse to the next node of the linked list 1400 and identify the next phase. This process can be repeated any appropriate number of times until all the tasks related to each of the execution targets related to the last phase of the release are completed.

[0142] As an example, at event number 5, it reaches the first node of the linked list 1410 of phases. The IP process 1408 identifies the DAG of the execution target corresponding to the first node.

[0143] At event number 6, it reaches the first node of the DAG 1412 of an execution target. The DAG 1414 of capabilities can be identified based at least in part on being related to the first node of the DAG 1412 of an execution target.

[0144] At event number 7, based at least in part on traversing the DAG 1414 of capabilities, execute tasks for a given execution target. When these tasks are completed, the IP process 1408 may traverse the next node of the DAG of the execution target, determine the DAG of the corresponding capability, and execute tasks according to the traversal of that DAG of capabilities. This process may continue until the tasks associated with the last node of the DAG 1412 of the execution target are executed. Thereafter, the IP process 1408 may traverse the next node of the linked list 1410 of phases. The operations of event numbers 5 - 7 may be repeated any suitable number of times until all tasks associated with all execution targets associated with the last node of the linked list 1410 of phases are executed. When a task, execution target, and / or phase is completed, the IP process 1408 may update or cause the UI 500 of FIG. 5 to be updated to display the current state of the task, execution target, and / or phase.

[0145] At event number 8, the IP process 1408 sends a signal indicating that the traversal of the release has been completed to the scheduler 1402. The scheduler 1402 may receive the signal from the IP process 1408 and may broadcast a notification indicating that the computing system is ready for use.

[0146] FIG. 15 is a flowchart of a process 1500 for deploying a computing system using a DAG in a CIOS according to at least one embodiment. In block 1502, the CIOS may execute instructions for parsing the configuration data related to the deployment of the computing system one or more times. The configuration data may be included in a configuration file, and when parsing the configuration data one or more times, the CIOS may determine a series of tasks to be executed for deploying the computing system. The tasks included in the series of tasks may include deploying infrastructure resources at the execution target or other suitable tasks for deploying the computing system.

[0147] In block 1504, the CIOS causes a first DAG (e.g., DAG 1200 of FIG. 12) to be generated for deploying resources based on a configuration file. The first DAG may be a DAG of capabilities that outlines a particular order of capabilities to be launched or deployed. The first DAG may include any suitable number of tasks for deploying the capabilities and any suitable number of dependencies between the capabilities to be deployed.

[0148] In block 1506, the CIOS generates a second DAG (e.g., DAG 900 of FIG. 9) that defines dependencies between execution targets for deploying the execution targets based on a configuration file. The second DAG may be a DAG of execution targets that specifies the order in which the execution targets are to be deployed. The second DAG may include any suitable number of execution targets for deploying the computing system and any suitable number of dependencies for deploying the execution targets.

[0149] In block 1508, the CIOS generates a linked list (e.g., linked list 700 of FIG. 7) that specifies dependencies between phases based on a configuration file. These phases may include the first DAG, the second DAG, or a combination thereof, and the linked list may define the order in which the phases are to be executed. The phases of the linked list may be executed sequentially and may not include phases to be executed in parallel. The linked list may include any suitable number of phases for deploying the computing system.

[0150] In block 1510, the CIOS may deploy a computing system by traversing a first DAG, a second DAG, and a linked list. The CIOS may traverse the linked list, and the first DAG and the second DAG may be traversed simultaneously (i.e., the first DAG and the second DAG may be included in the linked list). When the linked list, the first DAG, and the second DAG are traversed successfully, the deployment of the computing system may be performed successfully.

[0151] System Example Figures 16 to 18 are diagrams for explaining aspects of an exemplary environment for realizing aspects of the present disclosure according to various embodiments. FIG. 16 is a schematic diagram of a distributed system 1600 for realizing an embodiment of the present disclosure. In the illustrated embodiment, the distributed system 1600 includes one or more client computing devices 1602, 1604, 1606, and 1608. The one or more client computing devices 1602, 1604, 1606, and 1608 are configured to execute and operate client applications such as web browsers, proprietary clients (e.g., Oracle Forms), etc. via one or more networks 1610. The server 1612 may be communicatively connected to the remote client computing devices 1602, 1604, 1606, and 1608 via the network 1610.

[0152] In various embodiments, server 1612 may be configured to execute one or more services or software applications, such as services and applications that provide an ID management service. Also, in certain embodiments, server 1612 may provide other services or software applications that may include a non-virtual environment and a virtual environment. In some embodiments, these services may be provided to users of client computing devices 1602, 1604, 1606, and / or 1608 as web-based services or cloud services, or may be provided under a Software as a Service (SaaS) model. And users operating client computing devices 1602, 1604, 1606, and / or 1608 can interact with server 1612 using one or more client applications to utilize the services provided by these components.

[0153] In the configuration shown in FIG. 16, software components 1618, 1620, and 1622 of system 1600 are shown as being implemented on server 1612. Also, in other embodiments, one or more of the components of system 1600 and / or one or more of the services provided by these components may be implemented by one or more of client computing devices 1602, 1604, 1606, and / or 1608. Next, a user operating a client computing device can use one or more client applications to use the services provided by these components. These components may be implemented in hardware, firmware, software, or a combination thereof. It should be understood that various different system configurations are possible and these may differ from distributed system 1600. Thus, the embodiment shown in FIG. 16 is an example of a distributed system for implementing an embodiment of the system and is not intended to be limiting.

[0154] Client computing devices 1602, 1604, 1606, and / or 1608 may include various types of computer systems. For example, a client computing device may be a palm-sized portable device (e.g., iPhone®, mobile phone, iPad®, computing tablet, personal digital assistant (PDA)) or a wearable device (e.g., Google Glass® head-mounted display) that runs software such as Microsoft Windows® Mobile and / or various mobile operating systems such as iOS, Windows Phone®, Android, BlackBerry 10, Palm OS. The device may support various applications such as various Internet-related applications, email, SMS (Short Message Service) applications, etc., and may use various other communication protocols. Also, as an example, a client computing device may include a general-purpose personal computer, including a personal computer and / or a laptop computer that runs various versions of Microsoft Windows, Apple Macintosh®, and / or Linux® operating systems. A client computing device may be a workstation computer that runs various UNIX® or UNIX-like operating systems, including various GNU / Linux® operating systems such as Google Chrome OS, for example, but not limited to these. Also, a client computing device may include electronic devices that can communicate on a network(s) 1610, such as a thin client computer, an Internet-enabled gaming system (e.g., a Microsoft Xbox gaming console with or without a Kinect® gesture input device), and / or a personal messaging device.

[0155] The distributed system 1600 of FIG. 16 is shown with four client computing devices, but any number of client computing devices can be supported. Other devices, such as sensor-equipped devices, may communicate with the server 1612.

[0156] The network(s) 1610 in the distributed system 1600 can be any type of network familiar to those skilled in the art that supports data communication using various available protocols, including but not limited to TCP / IP (Transmission Control Protocol / Internet Protocol), SNA (Systems Network Architecture), IPX (Internet packet exchange), AppleTalk, etc. By way of example only, the network(s) 1610 can be a LAN (Local Area Network), an Ethernet®-based network, token ring, wide area network, the Internet, virtual network, VPN (Virtual Private Network), intranet, extranet, PSTN (Public Switched Telephone Network), infrared network, wireless network (e.g., a network operating under any of the IEEE (Institute Of Electrical And Electronics) 1002.16 suite of protocols, Bluetooth®, and / or other wireless protocols), and / or any combination thereof and / or other networks.

[0157] Server 1612 can be composed of one or more general-purpose computers, dedicated server computers (including, for example, PC (Personal Computer) servers, UNIX servers, midrange servers, mainframe computers, rack-mounted servers, etc.), server farms, server clusters, or other suitable arrangements and / or combinations. Server 1612 can include one or more virtual machines running a virtual operating system, or other computing architectures with virtualization. One or more flexible pools of logical storage devices can be virtualized to maintain a virtual storage device for the server. A virtual network can be controlled by server 1612 using software-defined networking. In various embodiments, server 1612 can be configured to execute one or more of the services or software applications described in the above disclosure. For example, server 1612 can correspond to a server for executing the above-described processes according to embodiments of the present disclosure.

[0158] Server 1612 can execute an operating system including any of the above-described operating systems and any of the commercially available server operating systems. In addition, server 1612 can execute various additional server applications and / or mid-tier applications, including an HTTP (Hypertext Transfer Protocol) server, an FTP (File Transfer Protocol) server, a CGI (Common Gateway Interface) server, a JAVA (registered trademark) server, a database server, etc. Exemplary database servers include, but are not limited to, database servers commercially available from Oracle, Microsoft, Sybase, IBM (International Business Machines), etc.

[0159] In some implementations, server 1612 may include one or more applications for analyzing and consolidating data feeds and / or event updates received from users of client computing devices 1602, 1604, 1606, and 1608. By way of example, the data feeds and / or event updates may include, but are not limited to, Twitter® feeds, Facebook® updates or real-time updates received from one or more third-party information sources and continuous data streams, and the one or more third-party information sources and continuous data streams may include real-time events related to, but not limited to, sensor data applications, tickers, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, automotive traffic monitoring, etc. Further, server 1612 may include one or more applications for displaying data feeds and / or real-time events via one or more display devices of client computing devices 1602, 1604, 1606, and 1608.

[0160] Additionally, the distributed system 1600 may include one or more databases 1614 and 1616. These databases may provide a mechanism for storing information such as user identification information and other information used in embodiments of the present disclosure. Databases 1614 and 1616 may be located in various locations. As an example, one or more of databases 1614 and 1616 may be located on a non-transitory storage medium that is local to (and / or present on) server 1612. Alternatively, databases 1614 and 1616 may be located at a location remote from server 1612 and may communicate with server 1612 through a network-based or dedicated connection. In a set of embodiments, databases 1614 and 1616 may be present in a SAN (Storage Area Network). Similarly, any necessary files for performing functions attributable to server 1612 may be stored, as appropriate, in a local location on server 1612 and / or at a location remote from server 1612. In a set of embodiments, databases 1614 and 1616 may include relational databases, such as databases provided by Oracle that are adapted to store, update, and retrieve data in response to SQL-format commands.

[0161] FIG. 17 is a diagram showing an exemplary computer system 1700 that may be used to implement embodiments of the present disclosure. In some embodiments, computer system 1700 may be used to implement any of the various servers and computer systems described above. As shown in FIG. 17, computer system 1700 includes a processing subsystem 1704 that communicates with several peripheral subsystems via a bus subsystem 1702, and includes various subsystems. These peripheral subsystems may include a processing acceleration device 1706, an I / O subsystem 1708, a storage subsystem 1718, and a communication subsystem 1724. Storage subsystem 1718 may include a tangible computer-readable storage medium 1722 and a system memory 1710.

[0162] The bus subsystem 1702 provides a mechanism for enabling the various components and subsystems of the computer system 1700 to communicate with each other as intended. Although the bus subsystem 1702 is illustrated as a single bus, other embodiments of the bus subsystem may utilize multiple buses. The bus subsystem 1702 may be any of several types of bus structures, including a memory bus of a memory controller, a peripheral bus, and a local bus that uses various bus architectures. For example, such architectures may include an ISA (Industry Standard Architecture) bus, an MCA (Micro Channel Architecture) bus, an EISA (Enhanced ISA) bus, a VESA (Video Electronics Standards Association) local bus, and a PCI (Peripheral Component Interconnect) bus, which may be implemented as, for example, a Mezzanine bus manufactured in accordance with the IEEE P1386.1 standard specification.

[0163] The processing subsystem 1704 controls the operation of the computer system 1700 and may include one or more processing devices 1732, 1734, etc. The processing device may include one or more processors, including a single-core or multi-core processor, one or more cores of a processor, or a combination thereof. In some embodiments, the processing subsystem 1704 may include one or more dedicated coprocessors, such as a graphics processor, a digital signal processor (DSP), etc. In some embodiments, some or all of the processing devices of the processing subsystem 1704 may be implemented using custom circuits, such as application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs).

[0164] In some embodiments, the processing device included in the processing subsystem 1704 can execute instructions stored in the system memory 1710 or on a computer-readable storage medium 1722. In various embodiments, the processing device can execute various programs or code instructions and maintain multiple simultaneously-executing programs or processes. At any given time, some or all of the program code being executed can reside in the system memory 1710 and / or on a computer-readable storage medium 1710 that potentially includes one or more storage devices. Through appropriate programming, the processing subsystem 1704 can provide the various functions described above for dynamically changing a document (e.g., a web page) according to usage patterns.

[0165] In certain embodiments, a processing acceleration device 1706 can be provided to reduce some of the load of the processing executed by the processing subsystem 1704 for executing customized processing or to speed up the overall processing executed by the computer system 1700.

[0166] The I / O subsystem 1708 may include devices and mechanisms for inputting information into the computer system 1700 and / or for outputting information via or from the computer system 1700. Generally, the term "input device" is intended to include any type of device and mechanism for inputting information into the computer system 1700. User interface input devices may include, for example, a keyboard, a pointing device such as a mouse or trackball, a touchpad or a touch screen incorporated into a display, a scroll wheel, a click wheel, a dial, a button, a switch, a keypad, a voice input device having a voice command recognition system, a microphone, and other types of input devices. Also, the user interface input devices may include motion detection devices and / or gesture recognition devices such as the Microsoft Kinect (registered trademark) motion sensor that enable a user to control and interact with the input device, the Microsoft Xbox (registered trademark) 360 game controller, and devices that provide an interface for receiving input using gesture commands and voice commands. Additionally, the user interface input devices may include eye gesture recognition devices such as the Google Glass (registered trademark) blink detection device that detects a user's eye movement (e.g., a "blink" while taking a photo and / or making a menu selection) and transforms the eye behavior into an input to the input device (e.g., Google Glass (registered trademark)). In addition to this, the user interface input devices may include voice recognition detection devices that enable a user to interact with a voice recognition system (e.g., the Siri (registered trademark) navigator) by voice commands.

[0167] Examples of other user interface input devices include, but are not limited to, 3D mice, joysticks or pointing sticks, game pads and graphic tablets, and audio / visual devices such as speakers, digital cameras, digital camcorders, portable media players, webcams, image scanners, fingerprint scanners, barcode readers, 3D scanners, 3D printers, laser rangefinders, and eye tracking devices. In addition to these, the user interface input device may include, for example, medical image input devices such as computed tomography, magnetic resonance imaging, positron emission tomography, and ultrasonic examination devices. Also, the user interface input device may include, for example, audio input devices such as MIDI keyboards and digital musical instruments.

[0168] The user interface output device may include non-visual display devices such as a display subsystem, indicator lights, or an audio output device. The display subsystem may be a flat panel display device such as one using a cathode ray tube (CRT), a liquid crystal display (LCD), or a plasma display, a projection device, a touch screen, or the like. Generally, the use of the term "output device" is intended to include any type of device and mechanism for outputting information from the computer system 1700 to the user or another computer. For example, the user interface output device may include, but is not limited to, monitors, printers, speakers, headphones, automotive navigation systems, plotting devices, audio output devices, and modems, and various display devices for visually conveying text, graphics, and audio / video information.

[0169] Storage subsystem 1718 provides a repository or data store for storing information used by computer system 1700. Storage subsystem 1718 provides a tangible non-transitory computer-readable storage medium for storing basic programming structures and data structures that provide the functionality of some embodiments. Software (programs, code modules, instructions) that provides the above-described functionality when executed by processing subsystem 1704 may be stored in storage subsystem 1718. The software may be executed by one or more processing devices of processing subsystem 1704. Also, storage subsystem 1718 may provide a repository for storing data used in accordance with the present disclosure.

[0170] Storage subsystem 1718 may include one or more non-transitory memory elements, including volatile and non-volatile memory elements. As shown in FIG. 17, storage subsystem 1718 includes system memory 1710 and computer-readable storage medium 1722. System memory 1710 may include several memories, including volatile main RAM (Random Access Memory) for storing instructions and data during program execution, and non-volatile ROM (Read Only Memory) or flash memory in which fixed instructions are stored. In some implementations, the BIOS (Basic Input / Output system), which includes basic routines that help transfer information between elements within computer system 1700, such as during startup, may be stored in the ROM. The RAM may include data and / or program modules that processing subsystem 1704 is currently operating on and executing. In some implementations, system memory 1710 may include multiple different types of memory, such as SRAM (Static Random Access Memory) or DRAM (Dynamic Random Access Memory).

[0171] As an example, as shown in FIG. 17, the system memory 1710 may include, but is not limited to, application programs 1712 such as client applications, web browsers, mid-tier applications, relational database management systems (RDBMS), program data 1714, and an operating system 1716. As an example, the operating system 1716 may include various versions of Microsoft Windows (registered trademark), Apple Macintosh (registered trademark), and / or Linux operating systems, various distributed UNIX (registered trademark) or UNIX-like operating systems (including, but not limited to, various GNU / Linux operating systems, Google Chrome (registered trademark) OS, etc.), and / or mobile operating systems such as iOS, Windows (registered trademark) Phone, Android (registered trademark) OS, BlackBerry (registered trademark) 10 OS, and Palm (registered trademark) OS.

[0172] The computer-readable storage medium 1722 can store programming structures and data structures that provide the functionality of some embodiments. When executed by the processing subsystem 1704, software (programs, code modules, instructions) that provides the above-described functionality to the processor can be stored in the storage subsystem 1718. As an example, the computer-readable storage medium 1722 may include non-volatile memory such as a hard disk drive, a magnetic disk drive, an optical disk drive such as a CD ROM, a DVD, a Blu-Ray (registered trademark) disk, or other optical media. The computer-readable storage medium 1722 may include, but is not limited to, a Zip (registered trademark) drive, a flash memory card, a USB (Universal Serial Bus) flash drive, an SD (Secure Digital) card, a DVD disk, a digital video tape, etc. The computer-readable storage medium 1722 may include SSDs (Solid-State Drives) based on non-volatile memory such as flash memory-based SSDs, enterprise flash drives, solid-state ROMs, etc., SSDs based on volatile memory such as solid-state RAM, dynamic RAM, static RAM, etc., DRAM-based SSDs, magnetoresistive RAM (MRAM) SSDs, and hybrid SSDs that use a combination of DRAM-based SSDs and flash memory-based SSDs. The computer-readable medium 1722 can provide storage of computer-readable instructions, data structures, program modules, and other data for the computer system 1700.

[0173] In certain embodiments, storage subsystem 1700 may further include a computer-readable storage medium reader 1720 that may be further connected to a computer-readable storage medium 1722. In combination with system memory 1710, and optionally in combination with system memory 1710, computer-readable storage medium 1722 may comprehensively represent remote storage devices, local storage devices, fixed storage devices, and / or rim-bubble storage devices, as well as storage media for storing computer-readable information.

[0174] In certain embodiments, computer system 1700 may provide support for running one or more virtual machines. Computer system 1700 may execute a program, such as a hypervisor, to facilitate the configuration and management of the virtual machines. Memory, a computer (e.g., a processor, a core), I / O, and networking resources may be allocated to each virtual machine. Each virtual machine may execute its own operating system. This operating system may be the same as or different from the operating systems executed by other virtual machines executed by computer system 1700. Thus, multiple operating systems may be executed simultaneously by computer system 1700. Each virtual machine generally runs separately from other virtual machines.

[0175] The communication subsystem 1724 provides an interface to other computer systems and networks. The communication subsystem 1724 functions as an interface for receiving data from the computer system 1700 and transmitting data from the computer system 1700 to other systems. For example, the communication subsystem 1724 may enable the computer system 1700 to establish a communication channel to one or more client computing devices for sending and receiving information via the Internet. In addition to this, the communication subsystem 1724 can be used to convey a notification of successful login or a notification requesting the requester user to re-enter the password from the administrator of a privileged account.

[0176] The communication subsystem 1724 may support both wired communication protocols and / or wireless communication protocols. For example, in certain embodiments, the communication subsystem 1724 may include an RF (Radio Frequency) transmission component (using, for example, cellular phone technology, next-generation data network technologies such as 3G, 4G, or EDGE (Enhanced Data Rates For Global Evolution), WiFi (IEEE 802.11 family of standard specifications), other mobile object communication technologies, or any combination thereof), a GPS (Global Positioning System) receiving component, and / or other components for accessing a wireless voice network and / or a data network. In some embodiments, the communication subsystem 1724 can provide wired network connectivity (such as Ethernet) in addition to or instead of a wireless interface.

[0177] The communication subsystem 1724 can transmit and receive data in various forms. For example, in some embodiments, the communication subsystem 1724 can receive an input communication message in the form of a structured and / or unstructured data feed 1726, an event stream 1728, an event update 1730, etc. For example, the communication subsystem 1724 can be configured to receive (or send) the data feed 1726 in real time from users of social media networks and / or other communication services, such as Twitter (registered trademark) feeds, Facebook (registered trademark) updates, web feeds such as RSS (Rich Site Summary) feeds, and / or real-time updates from one or more third-party information sources.

[0178] In certain embodiments, the communication subsystem 1724 may be configured to receive data in the form of a continuous data stream, which may include an event stream 1728 of continuous or infinite real-time events and / or event updates 1730 that do not have a clearly defined end. Examples of applications that generate continuous data can include sensor data applications, tickers, network performance measurement tools (such as network monitoring and traffic management applications), clickstream analysis tools, and automotive traffic monitoring.

[0179] Also, the communication subsystem 1724 can be configured to output structured and / or unstructured data feeds 1726, event streams 1728, event updates 1730, etc. to one or more databases that can be in communication with one or more streaming data source computers connected to the computer system 1700.

[0180] Computer system 1700 can be of one of various types, including a palm-sized portable device (e.g., an iPhone (registered trademark) mobile phone, an iPad (registered trademark) computing tablet, a PDA), a wearable device (e.g., a Google Glass (registered trademark) head-mounted display), a personal computer, a workstation, a mainframe, a kiosk, a server rack, or other data processing systems.

[0181] Due to the ever-changing nature of computers and networks, the description of computer system 1700 shown in FIG. 17 is merely illustrative. Many other configurations are possible that have more or fewer components than the system shown in FIG. 17. Based on the disclosure and teachings described herein, those skilled in the art will appreciate other ways and / or methods for implementing various embodiments.

[0182] The systems shown in some of the drawings can be provided in various configurations. In some embodiments, these systems can be configured as distributed systems in which one or more components of the system are distributed among one or more networks in one or more cloud infrastructure systems.

[0183] A cloud infrastructure system is a collection of one or more server computing devices, network devices, and / or storage devices. These resources can be partitioned by a cloud service provider and allocated to its customers in some way. For example, a cloud service provider such as Oracle in Redwood Shores, California, can provide various types of cloud services, including but not limited to one or more services provided under the SaaS (Software as a Service) classification, services provided under the PaaS (Platform as a Service) classification, services provided under the IaaS (Infrastructure as a Service) classification, or services in other classifications such as hybrid services. As SaaS services, there are functions such as building and delivering a set of on-demand applications such as Oracle Fusion applications, but not limited to these. SaaS services enable customers to use applications running on a cloud infrastructure system without the need to purchase software for the applications. As PaaS services, there are services such as JCS (Oracle Java Cloud Service), DBCS (Oracle Database Cloud Service), etc., that enable an organization (such as Oracle) to consolidate existing applications onto a shared common architecture, and the ability to create new applications that utilize shared services provided by the platform, but not limited to these. IaaS services will facilitate the management and control of underlying computing resources, such as storage, network, and other basic computing resources, for customers who are using the services provided by the SaaS platform and the PaaS platform.

[0184] FIG. 18 is a simplified block diagram of one or more components of a system environment 1800 in which services provided by one or more components of the system according to an embodiment of the present disclosure can be provided as cloud services. In the illustrated embodiment, the system environment 1800 includes one or more client computing devices 1804, 1806, and 1808. The one or more client computing devices 1804, 1806, and 1808 may be used by a user to interact with a cloud infrastructure system 1802 that provides cloud services. The client computing device may be configured to operate a client application, such as a web browser, a proprietary client application (e.g., Oracle Forms), or other applications. The client application may be used by a user of the client computing device to interact with the cloud infrastructure system 1802 to utilize the services provided by the cloud infrastructure system 1802.

[0185] It should be understood that the cloud infrastructure system 1802 shown in the figure may have components other than those shown. Further, the embodiment shown in the figure is only an example of a cloud infrastructure system that may incorporate embodiments of the present disclosure. In some other embodiments, the cloud infrastructure system 1802 may have more or fewer components than those shown in the figure, may combine two or more components, or may have a different configuration or arrangement of components.

[0186] The client computing devices 1804, 1806, and 1808 may be similar devices to the client computing devices described above with respect to 1602, 1604, 1606, and 1608.

[0187] The exemplary system environment 1800 is shown with three client computing devices, but any number of client computing devices may be supported. Other devices, such as sensor - equipped devices, may interact with the cloud infrastructure system 1802.

[0188] The network(s) 1810 may facilitate data communication and interaction between clients 1804, 1806, and 1808 and the cloud infrastructure system 1802. Each network may be any type of network familiar to those skilled in the art that can support data communication using any of a variety of commercially available protocols, including the protocols described above with respect to network(s) 1810.

[0189] The cloud infrastructure system 1802 may be composed of one or more computers and / or servers that may include the computers and servers described above with respect to server 1812.

[0190] In certain embodiments, the services provided by a cloud infrastructure system may include hosting services that are available to users of the cloud infrastructure system upon request, such as online data storage and backup solutions, web-based email services, hosted office suite document collaboration services, database processing, managed technical support services, and the like. The services provided by a cloud infrastructure system can scale dynamically to meet the needs of its users. A specific instantiation of a service provided by a cloud infrastructure system is referred to herein as a "service instance." In general, any service from a cloud service provider's system that is made available to users via a communication network such as the Internet is referred to as a "cloud service." In a public cloud environment, the servers and systems that make up a cloud service provider's system are different from customer-owned on-premises servers and systems. For example, a cloud service provider's system may host applications, and users may order and use the applications on demand via a communication network such as the Internet.

[0191] In some examples, services in the cloud infrastructure of a computer network may include storage, hosted databases, hosted web servers, protected computer network access to software applications, or other services provided by a cloud vendor to a user, or other services other than the above well-known in the art. For example, the service may include password-protected access to remote storage on the cloud over the Internet. As another example, the service may include a web service-based hosted relational database and a middleware engine for a scripting language for private use by networked developers. As another example, the service may include access to an email software application hosted on a cloud vendor's website.

[0192] In certain embodiments, the cloud infrastructure system 1802 may include a suite of applications, middleware, and database service offerings that are delivered to a customer in a secure manner that is self-service, subscription-based, elastically scalable, reliable, and highly available. An example of such a cloud infrastructure system is the Oracle Public Cloud provided by the assignee of the present application.

[0193] In various embodiments, the cloud infrastructure system 1802 can be configured to automatically provision, manage, and track customer subscriptions to the services provided by the cloud infrastructure system 1802. The cloud infrastructure system 1802 can provide cloud services via different deployment models. For example, the services can be provided under a public cloud model. In the public cloud model, the cloud infrastructure system 1802 is owned by an organization that provides cloud services (e.g., owned by Oracle) and is made available to the general public or different industrial enterprises. As another example, the services can be provided under a private cloud model where the cloud infrastructure system 1802 is run only for one organization and can provide services for one or more entities within the organization. Also, the cloud services can be provided under a community cloud model. In the community cloud model, the cloud infrastructure system 1802 and the services provided by the cloud infrastructure system 1802 are shared by several organizations within the relevant community. The cloud services can be provided under a hybrid cloud model, which is a combination of two or more different models.

[0194] In some embodiments, the services provided by the cloud infrastructure system 1802 can include one or more services provided under other classifications, including SaaS (Software as a Service) classification, PaaS (Platform as a Service) classification, IaaS (Infrastructure as a Service) classification, or hybrid services. A customer can order one or more services provided by the cloud infrastructure system 1802 by placing a subscription order. The cloud infrastructure system 1802 then performs processing to provide the services ordered in the customer's subscription order.

[0195] In some embodiments, the services provided by the cloud infrastructure system 1802 may include, but are not limited to, application services, platform services, and infrastructure services. In some examples, the application services may be provided by the cloud infrastructure system via SaaS services. The SaaS platform may be configured to provide cloud services corresponding to the SaaS classification. For example, the SaaS platform may provide functions for building an on-demand set of applications and delivering them to an integrated development / deployment platform. The SaaS platform may manage and control the software and infrastructure underlying the provision of the SaaS services. By utilizing the services provided by the SaaS platform, customers can utilize applications that are executed on the cloud infrastructure system. Customers can obtain the application services without the need to purchase separate licenses and support. A variety of different SaaS services may be provided. Examples include, but are not limited to, services that provide solutions for sales performance management, enterprise integration, and business agility for large organizations.

[0196] In some embodiments, the platform service may be provided by a cloud infrastructure system via a PaaS platform. The PaaS platform may be configured to provide cloud services corresponding to the PaaS classification. Examples of platform services include, but are not limited to, services that enable an organization (such as Oracle) to consolidate existing applications onto a shared common architecture, and the ability to create new applications that utilize the shared services provided by the platform. The PaaS platform may manage and control the underlying software and infrastructure for providing PaaS services. Customers can obtain the services provided by the PaaS cloud infrastructure system without separately purchasing licenses and support. Examples of platform services include, but are not limited to, JCS (Oracle Java Cloud Service), DBCS (Oracle Database Cloud Service), and others.

[0197] By using the services provided by the PaaS platform, customers can adopt the programming languages and tools supported by the cloud infrastructure system and can also manage the deployed services. In some embodiments, the platform services provided by the cloud infrastructure system may include database cloud services, middleware cloud services (e.g., Oracle Fusion Middleware services), and Java cloud services. In one embodiment, the database cloud service may support a shared service deployment model. The shared service deployment model enables organizations to pool database resources and provide databases to customers as services in the form of a database cloud. The middleware cloud service may provide a platform for customers to develop and deploy various business applications, and the Java cloud service may provide a platform for customers to deploy Java applications in the cloud infrastructure system.

[0198] In a cloud infrastructure system, the IaaS platform can provide various different infrastructure services. The infrastructure services facilitate the management and control of underlying computing resources, such as storage, network, and other basic computing resources, for customers who are using the services provided by the SaaS platform and the PaaS platform.

[0199] In certain embodiments, the cloud infrastructure system 1802 may include infrastructure resources 1830 for providing resources used to provide various services to customers of the cloud infrastructure system. In one embodiment, the infrastructure resources 1830 may include an optimal pre-integrated combination of hardware such as servers, storage, and networking resources, and other resources for executing services provided by the PaaS platform and the SaaS platform, and other resources.

[0200] In some embodiments, the resources in the cloud infrastructure system 1802 are shared by multiple users and can be dynamically reallocated in response to requests. In addition, the resources can be allocated to users in different time zones. For example, the cloud infrastructure system 1830 enables a first set of users in a first time zone to utilize the resources of the cloud infrastructure system for a specified number of hours and then reallocate the same resources to another set of users located in a different time zone, maximizing the utilization of the resources.

[0201] In certain embodiments, some internal shared services 1832 can be provided that are shared by different components or modules of the cloud infrastructure system 1802 and by the services provided by the cloud infrastructure system 1802. These internal shared services can include, but are not limited to, services for enabling security / ID services, integration services, enterprise repository services, enterprise manager services, virus scan / whitelist services, highly available backup / recovery services, cloud support, email services, notification services, file transfer services, and the like.

[0202] In certain embodiments, the cloud infrastructure system 1802 may provide comprehensive management of cloud services (e.g., SaaS services, Paa services, and IaaS services) in the cloud infrastructure system. In one embodiment, the cloud management functionality may include functionality for provisioning, managing, and tracking customer subscriptions received by, for example, the cloud infrastructure system 1802.

[0203] In one embodiment, as shown in the figure, the cloud management functionality may be provided by one or more modules, such as an order management module 1820, an order orchestration module 1822, an order provisioning module 1824, an order management / monitoring module 1826, and an ID management module 1828. These modules may include, or may be provided using, one or more computers and / or servers. The one or more computers and / or servers may be general-purpose computers, dedicated server computers, server farms, server clusters, or other suitable arrangements and / or combinations.

[0204] In exemplary operation 1834, a customer using a client device such as client devices 1804, 1806, or 1808 may interact with cloud infrastructure system 1802 by requesting one or more services provided by cloud infrastructure system 1802 and placing an order for a subscription to one or more services provided by cloud infrastructure system 1802. In certain embodiments, the customer may access a cloud UI (user interface) such as cloud UI 1812, cloud UI 1814, and / or cloud UI 1816 and place an order for a subscription via these UIs. Order information received by cloud infrastructure system 1802 in response to the order placed by the customer may include information identifying the customer and one or more services provided by cloud infrastructure system 1802 for which the customer intends to subscribe.

[0205] After the order is placed by the customer, the order information is received via cloud UIs 1812, 1814, and / or 1816.

[0206] In operation 1836, this order is stored in order database 1818. Order database 1818 may be one of several databases operated by cloud infrastructure system 1818 and operated in conjunction with other system elements.

[0207] In operation 1838, the order information is transferred to order management module 1820. Optionally, order management module 1820 may be configured to perform billing and accounting functions related to the order, such as order confirmation and registration of the order after confirmation.

[0208] In operation 1840, information regarding an order is transmitted to the order orchestration module 1822. The order orchestration module 1822 may utilize this order information to orchestrate the provisioning of services and resources related to the order placed by the customer. In some cases, the order orchestration module 1822 may orchestrate the provisioning of resources to support the subscribed services using the services of the order provisioning module 1824.

[0209] In certain embodiments, the order orchestration module 1822 enables the management of the business flow associated with each order and applies business logic to determine whether the order should proceed to provisioning. In operation 1842, upon receiving an order for a new subscription, the order orchestration module 1822 sends a request to the order provisioning module 1824 to allocate and configure the resources required to fulfill the subscription order. The order provisioning module 1824 enables the allocation of resources for the services subscribed by the customer. The order provisioning module 1824 provides an abstraction level between the cloud services provided by the cloud infrastructure system 1800 and the physical implementation layer used to provision the resources for providing the requested services. Thus, the order orchestration module 1822 can be decoupled from the implementation details such as whether the services and resources are actually provisioned on-the-fly or are only allocated when provisioned in advance and requested.

[0210] In operation 1844, when services and resources are provisioned, a notification about the provided services can be sent by the order provisioning module 1824 of the cloud infrastructure system 1802 to the customers on the client devices 1804, 1806, and / or 1808. In operation 1846, the order management / monitoring module 1826 can manage and track the orders of the customer subscriptions. Optionally, the order management / monitoring module 1826 can be configured to collect usage statistic data regarding the services in the subscription orders, such as the amount of storage used, the amount of data transferred, the number of users, as well as the system uptime and system downtime, etc.

[0211] In certain embodiments, the cloud infrastructure system 1800 may include an ID management module 1828. The ID management module 1828 can be configured to provide ID services such as access management and authorization services in the cloud infrastructure system 1800. In some embodiments, the ID management module 1828 can manage information about customers who want to utilize the services provided by the cloud infrastructure system 1802. Such information can include information for authenticating the IDs of such customers, and information describing what operations they are authorized to perform on various system resources (such as files, directories, applications, communication ports, memory segments, etc.). Further, the ID management module 1828 may include management of the descriptive information for each customer, and the descriptive information about who can access and modify such descriptive information and how.

[0212] While specific embodiments of the present disclosure have been described, various modifications, alternatives, alternative configurations, and equivalents are also encompassed within the scope of the present disclosure. Embodiments of the present disclosure are not limited to operations within a specific particular data processing environment and can operate freely within multiple data processing environments. In addition to this, while specific sequences of transactions and steps have been used to describe embodiments of the present disclosure, it should be apparent to those skilled in the art that the scope of the present disclosure is not limited to the recited sequences of transactions and steps. The various features and aspects of the above-described embodiments can be used individually or jointly.

[0213] Furthermore, while specific combinations of hardware and software have been used to describe embodiments of the present disclosure, it should be recognized that other combinations of hardware and software are also within the scope of the present disclosure. Embodiments of the present disclosure may be implemented by hardware only, or by software only, or may be implemented using a combination thereof. The various processes described herein can be implemented on the same processor or on respective different processors of any combination. Thus, where a component or module is described as being configured to perform a particular operation, such a configuration can be achieved, for example, by designing an electronic circuit to perform this operation, by programming a programmable electronic circuit (such as a microprocessor) to perform this operation, or by any combination thereof. Processes can communicate using a variety of techniques including prior art for inter-process communication, but are not limited thereto, and different pairs of processes may use different techniques, and the same pair of processes may use different techniques at different times.

[0214] The specification and drawings should be regarded as illustrative rather than restrictive, as appropriate. However, it will be apparent that additions, subtractions, deletions, and other changes and modifications can be made to them without departing from the broader spirit and scope of the appended claims. Accordingly, while specific embodiments of the present disclosure have been described, these are not intended to be limiting. Various variations and equivalents are within the scope of the appended claims.

Claims

1. A method implemented by a computer, comprising: A computing device executes instructions for performing one or more analyses of configuration data related to the deployment of a computing system; The computing device generates a first DAG (Directed Acyclic Graph), wherein the first DAG is utilized to deploy a first resource based at least in part on the execution of the one or more analyses, and the method further comprises: The computing device generates a second DAG for deploying a plurality of execution targets based at least in part on the execution of the one or more analyses, wherein the second DAG specifies dependencies between the execution targets of the deployment, and the method further comprises: The computing device generates a linked list data structure based at least in part on the execution of the one or more analyses, wherein the linked list data structure specifies dependencies between a plurality of deployment phases, and the method further comprises: The computing device deploys the computing system based at least in part on traversing the linked list data structure, the second DAG, and the first DAG.

2. The method implemented by a computer according to claim 1, wherein the first DAG is generated by a declarative infrastructure provisioner.

3. The method implemented by a computer according to claim 1 or 2, wherein the first DAG specifies the dependency of a first resource of the computing system on the capabilities of a second resource of the computing system.

4. Each of the first resource or the second resource is one of a plurality of computing services, and the ability is a part of the function of the second resource. The method realized by the computer according to claim 3.

5. At least one node of the second DAG refers to a node of the first DAG. The method realized by the computer according to any one of claims 1 to 4.

6. A node of the linked list data structure refers to at least one node of the second DAG. The method realized by the computer according to any one of claims 1 to 5.

7. The instructions for performing one or more analyses of the configuration data are Detecting a first dependency through an explicit statement provided in the configuration data, or Detecting a second dependency based at least in part on identifying an implicit dependency provided in the configuration data. The method realized by the computer according to any one of claims 1 to 6.

8. The configuration data indicates an order for performing infrastructure deployment operations for deploying a plurality of resources through one or more dependencies. The method realized by the computer according to any one of claims 1 to 7.

9. The one or more dependencies are defined using declarative statements. The method realized by the computer according to any one of claims 1 to 8.

10. A system comprising One or more processors and One or more memories storing computer-executable instructions, which when executed by the one or more processors cause the one or more processors to A computing device executes instructions for performing one or more analyses of configuration data related to the deployment of a computing system. The computing device is configured to generate a first DAG, where the first DAG is utilized to deploy a first resource based at least in part on the execution of the one or more analyses. When the computer-executable instructions are executed by the one or more processors, the one or more processors are further The computing device is configured to generate a second DAG for deploying a plurality of execution targets based at least in part on the execution of the one or more analyses, where the second DAG specifies dependencies between the execution targets of the deployment. When the computer-executable instructions are executed by the one or more processors, the one or more processors are further The computing device is configured to generate a linked list data structure based at least in part on the execution of the one or more analyses, where the linked list data structure specifies dependencies between a plurality of deployment phases. When the computer-executable instructions are executed by the one or more processors, the one or more processors are further A system, wherein the computing device is configured to deploy the computing system based at least in part on traversing the linked list data structure, the second DAG, and the first DAG.

11. The system of claim 10, wherein the first DAG is generated by a declarative infrastructure provisioner, and the first DAG specifies a dependency of a first resource of the computing system on the capabilities of a second resource of the computing system.

12. The system of claim 11, wherein each of the first resource or the second resource is one of a plurality of computing services, and the capabilities are part of the functionality of the second resource.

13. The system according to any one of claims 10 to 12, wherein at least one node of the second DAG refers to a node of the first DAG, and a node of the linked list data structure refers to at least one node of the second DAG.

14. The configuration data indicates an order for performing infrastructure deployment operations for deploying a plurality of resources via one or more dependencies, the one or more dependencies being defined using declarative statements, and the instructions for performing one or more analyses of the configuration data are detecting a first dependency via an explicit statement provided in the configuration data, or detecting a second dependency based at least in part on identifying an implicit dependency provided in the configuration data. The system according to any one of claims 10 to 13.

15. An apparatus comprising means for performing the steps according to any one of claims 1 to 9.

16. A computer program comprising computer instructions which, when executed by a processor, perform the steps of the method according to any one of claims 1 to 9.

Images