Techniques for using directed acyclic graphs to deploy instructions

Abstract

A technique for using directed acyclic graphs (DAGs) to deploy instructions is disclosed. The computer-implemented method can include various operations. Instructions can be executed by a computing device to perform the parsing of configuration data associated with the deployment. The computing device can generate a first DAG, which is used to deploy a first resource based on the parsing. A second DAG can be generated to deploy execution targets based on the parsing, the second DAG specifying dependencies between the execution targets of the deployment. The computing device can generate a linked list data structure based on the parsing, and the computing system can be deployed by traversing the linked list data structure.

Description

[0001] This application is a non-provisional application of U.S. Provisional Application No. 62 / 963,477, filed January 20, 2020, entitled “TECHNIQUES FOR UTILIZING DIRECTEDACYCLIC GRAPHS FOR DEPLOYMENT INSTRUCTIONS”, and U.S. Non-Provisional Application No. 16 / 953,262, filed November 19, 2020, entitled “TECHNIQUES FOR UTILIZING DIRECTED ACYCLIC GRAPHS FOR DEPLOYMENT INSTRUCTIONS”, and claims the benefit and priority of both applications pursuant to 35 U.SC 119(e), the contents of which are incorporated herein by reference in their entirety for all purposes. Background Technology

[0002] Today, cloud infrastructure services utilize numerous separate services to provision and deploy code and configuration across many regions of the cloud infrastructure service (separately). These tools require a significant amount of manual work to use, especially given that provisioning is typically declarative and deployment code is imperative. Additionally, cloud infrastructure services will need to continue to grow as the number of service teams and regions increases. Some cloud infrastructure service deployment strategies to a large number of smaller regions involve per-region spending, which may not scale well. Summary of the Invention

[0003] Techniques for using directed acyclic graphs (DAGs) for deployment instructions are disclosed. In some embodiments, the computer-implemented method may include various operations. Instructions may be executed by a computing device to perform the parsing of configuration data associated with the deployment. The computing device may generate a first DAG, which is used to deploy a first resource (e.g., a software service) based on the parsing. A second DAG may be generated for deploying execution targets based on the parsing, the second DAG specifying dependencies between the execution targets of the deployment. The computing device may generate a linked list data structure based on the parsing, and the computing system may be deployed by traversing the linked list data structure.

[0004] In other embodiments, a system for using a Directed Acyclic Graph (DAG) for deployment instructions is disclosed. The system may include one or more processors and one or more memories storing computer-executable instructions, which, when executed by the one or more processors, configure the one or more processors to perform various operations. A computing device may execute instructions to perform one or more parses of configuration data associated with the deployment of the computing system. The computing device may generate a first DAG used at least partially based on the execution of one or more parses to deploy a first resource. The computing device may generate a second DAG for deploying multiple execution targets at least partially based on the execution of one or more parses, the second DAG specifying dependencies between the deployed execution targets. The computing device may generate a linked list data structure at least partially based on the execution of one or more parses, the linked list data structure specifying dependencies between multiple deployment stages. Furthermore, the computing device may deploy the computing system at least partially based on traversing the linked list data structure, the second DAG, and the first DAG.

[0005] In other embodiments, a computer-readable storage medium is disclosed for using a DAG for deployment instructions. This computer-readable storage medium can store computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform various operations. A computing device can execute the instructions to perform one or more parsings of configuration data associated with the deployment of a computing system. The computing device can generate a first DAG used at least partially based on the execution of one or more parsings to deploy a first resource. The computing device can generate a second DAG for deploying multiple execution targets at least partially based on the execution of one or more parsings, the second DAG specifying dependencies between the deployed execution targets. The computing device can generate a linked list data structure at least partially based on the execution of one or more parsings, the linked list data structure specifying dependencies between multiple deployment stages. Furthermore, the computing device can deploy the computing system at least partially based on traversing the linked list data structure, the second DAG, and the first DAG.

[0006] In other embodiments, an apparatus is disclosed. This apparatus may include components for performing steps of any of the methods disclosed herein.

[0007] In other embodiments, a computer program product is disclosed. The computer program product may include computer instructions that, when executed by a processor, implement the steps of any of the methods disclosed herein. Attached Figure Description

[0008] To facilitate the identification of any particular element or action in discussion, one or more of the most significant digits in the figure references refer to the figure number in which the element was first introduced.

[0009] Figure 1 It is a block diagram of an architecture for implementing cloud infrastructure orchestration services according to at least one embodiment.

[0010] Figure 2 It is a block diagram of an architecture for implementing cloud infrastructure orchestration services according to at least one embodiment.

[0011] Figure 3 It is a flowchart used to illustrate an example group (flock) according to at least one embodiment.

[0012] Figure 4 This is a flowchart used to illustrate an example group according to at least one embodiment.

[0013] Figure 5 It is a user interface that presents information related to the release of information including multiple stages and multiple execution objectives, according to at least one embodiment.

[0014] Figure 6 This is an example code snippet for defining a list and order of stages, according to at least one embodiment.

[0015] Figure 7 It is an example data structure that can be generated by Cloud Infrastructure Orchestration Service (CIOS) according to at least one embodiment to maintain a list and order associated with one or more phases.

[0016] Figure 8 This is an example code segment according to at least one embodiment for defining a list and order of execution targets.

[0017] Figure 9 It is an example data structure that can be generated by Cloud Infrastructure Orchestration Service (CIOS) according to at least one embodiment to maintain a list and order associated with one or more execution targets associated with stages.

[0018] Figure 10 This is an example code segment according to at least one embodiment for establishing explicit and implicit dependencies between resources of an execution target.

[0019] Figure 11 This is an example code segment according to at least one embodiment for establishing explicit and implicit dependencies between resources of an execution target.

[0020] Figure 12 It is an example directed acyclic graph corresponding to resources of a cloud computing system according to at least one embodiment.

[0021] Figure 13This is a flowchart illustrating an example process for orchestrating the execution of a task that includes dependencies on at least one capability, according to at least one embodiment.

[0022] Figure 14 This is an example processing flow for a computer infrastructure orchestration service that traverses linked lists, executes target directed acyclic graphs, and capability directed acyclic graphs according to at least one embodiment.

[0023] Figure 15 This is a flowchart of a process for deploying a computing system using a directed acyclic graph in a computer infrastructure orchestration service, according to at least one embodiment.

[0024] Figure 16 It is a block diagram used to illustrate a distributed system according to at least one embodiment.

[0025] Figure 17 It is a block diagram illustrating one or more components of a system environment according to at least one embodiment, through which services provided by one or more components of the system of the embodiment can be provided as cloud services.

[0026] Figure 18 This is a block diagram of an example computer system in which various embodiments of the present disclosure may be implemented. Detailed Implementation

[0027] In some examples, Infrastructure as a Service (IaaS) is a specific type of cloud computing. IaaS can be configured to provide virtualized computing resources over a public network (e.g., the Internet). In some examples, IaaS is one of the three main categories (or subcategories) of cloud computing services. Most consider the other main categories to be Software as a Service (SaaS) and Platform as a Service (PaaS), and sometimes SaaS may be considered a broader category that includes both PaaS and IaaS, and some even consider IaaS to be a subcategory of PaaS.

[0028] In the IaaS model, cloud computing providers can host infrastructure components (e.g., servers, storage devices, network nodes (e.g., hardware), deployment software, platform virtualization (e.g., hypervisor layer), etc.).

[0029] In some cases, IaaS providers can also offer a variety of services to accompany those infrastructure components (e.g., billing, monitoring, logging, security, load balancing, and clustering). Therefore, because these services may be policy-driven, IaaS users may be able to implement policies to drive load balancing to maintain application availability and performance.

[0030] In some cases, IaaS customers can access resources and services over a wide area network (WAN) such as the Internet and can use the cloud provider's services to install the rest of the application stack. For example, a user can log in to an IaaS platform to create virtual machines (VMs), install an operating system (OS) on each VM, deploy middleware such as databases, create buckets for workloads and backups, and even install enterprise software onto the VM. The customer can then use the provider's services to perform various functions, including balancing network traffic, troubleshooting application issues, monitoring performance, and managing disaster recovery.

[0031] In most cases, cloud computing models will require the involvement of cloud providers. Cloud providers can, but are not necessarily, third-party services that exclusively offer (e.g., sell) IaaS. Entities can also choose to deploy private clouds, thus becoming their own infrastructure service providers.

[0032] In some examples, IaaS deployment is the process of placing a new application or version onto a prepared application server, etc. It may also include the process of preparing the server (e.g., installing libraries, daemons, etc.). This is typically managed by the cloud provider below the hypervisor layer (e.g., servers, storage, network hardware, and virtualization). Therefore, the customer can be responsible for (e.g., on self-service virtual machines that can be spun up on demand) the processing (OS), middleware, and / or application deployment.

[0033] In some examples, IaaS provisioning can refer to acquiring computers or virtual hosts for use, and even installing the necessary libraries or services on them. In most cases, deployment does not include provisioning, and provisioning may need to be performed first.

[0034] In some cases, there are two distinct challenges with IaaS provisioning. First, there's the initial challenge of provisioning the initial set of infrastructure before anything is operational. Second, there's the challenge of evolving the existing infrastructure (e.g., adding new services, changing services, removing services, etc.) once everything is provisioned. In some cases, these challenges can be addressed by enabling the configuration of the infrastructure to be declaratively defined. In other words, the infrastructure (e.g., what components are needed and how they interact) can be defined by one or more configuration files. Therefore, the overall topology of the infrastructure (e.g., what resources depend on which and how they work together) can be declaratively described. In some cases, once the topology is defined, workflows for creating and / or managing the different components described in the configuration files can be generated.

[0035] In some embodiments, IaaS provisioning may include generating a directed acyclic graph (DAG). A DAG can be a finite directed graph comprising any suitable number of nodes and edges, where each edge points from one node to another. Nodes and edges can be arranged to avoid directed cycles. That is, the DAG is arranged such that there is no path that starts from any node and follows a consistent sequence of directed edges that eventually cycle back to the same node. IaaS provisioning may include resolving profiles corresponding to one or more resources of the system (e.g., services, software resources, etc.). Individual DAGs may be generated for each resource. The DAG for each resource can define the resource's dependencies on the capabilities of one or more other resources of the system (e.g., services, software resources, etc.). For example, a first DAG may be generated and used to deploy a first resource (e.g., a software service, also referred to as a "computing service," such as an email service configured to manage electronic messaging for one or more users). The first DAG may indicate the first resource's dependencies on capabilities provided by a second resource (e.g., an additional software service different from the first resource, such as an identity service configured to verify / authenticate user identities based on previously acquired credentials). The first and / or second resources can each be one of a plurality of services provided by the computing system. A “capability” can be intended to refer to a portion of the functionality of a given resource (e.g., the ability of the second resource to verify / authenticate a user). Processes can be instantiated to traverse a Directed Acyclic Graph (DAG). When a node in the DAG corresponding to a currently unavailable capability is reached, the process can announce to the scheduling service that the process has reached a dependency on the capability and is therefore waiting for that particular capability to become available before it can continue. As various resources of the system are deployed and / or booted, these resources can announce the availability of various capabilities to the scheduling service when capabilities become available. As used herein, the terms “boot,” “booting,” and “booted” refer to a sequence of processes that perform operations corresponding to a specific resource (e.g., software / computing service, computing device, etc.). Deploying a resource (e.g., a software service) may include booting and / or otherwise making at least a portion (e.g., one or more capabilities) of the functionality provided by the resource available. When the scheduling service determines that a particular capability has become available, it can restart the process from its last point of exit (e.g., immediately after publishing the demand for the capability). The process can regenerate the DAG and restart the traversal (e.g., starting from the last visited node). By utilizing the DAG for each resource, the system can manage dependencies between the capabilities of different resources, eliminating the need for human operators to manually start complex systems.

[0036] In some examples, the infrastructure can have many interconnected elements. For example, there may be one or more Virtual Private Clouds (VPCs) (e.g., potentially on-demand, configurable, and / or shared pools of compute resources), also known as the core network. In some examples, there may also be one or more security group rules, provided to define how the network's security will be configured, and one or more virtual machines (VMs). Other infrastructure elements such as load balancers, databases, etc., may also be provided. As more infrastructure elements are expected and / or added, the infrastructure can evolve gradually.

[0037] As mentioned above, one way to supply infrastructure is to describe it declaratively. Therefore, a configuration file can be a declarative document that simply describes each of the aforementioned infrastructure components and how they interact. The configuration file can describe the resources and associated fields required to create an element, and then other elements referencing the previously described elements can be described. In some examples, the supply tool can then generate workflows for creating and managing the elements described in the configuration file.

[0038] In some cases, the workflow of a provisioning tool can be configured to execute various commands. One executable function is view reconciliation, where the provisioning tool compares a view of the current infrastructure (e.g., the expected state of the infrastructure) with how the infrastructure is actually operating. In some cases, performing view reconciliation may include querying various resource providers or infrastructure resources to identify which resources are actually running. Another function that a provisioning tool can perform is plan generation, where the provisioning tool compares the actually running infrastructure components with what the provisioning tool hopes the state will look like (e.g., the desired configuration). In other words, plan generation can determine what changes need to be made to bring resources to the latest expected state. In some cases, a third function is execution (e.g., application) functionality, where the provisioning tool can execute plans generated by the plan generation function.

[0039] Typically, provisioning tools can be configured to obtain a configuration file, parse the declarative information contained therein, and programmatically / automatically determine the order in which resources need to be provisioned to execute the plan. For example, if a VPC needs to be started before security group rules and VMs are started, the provisioning tool will be able to make that decision and start them in that order without user intervention and / or without requiring that information to be included in the configuration file.

[0040] In some cases, continuous deployment techniques can be used to deploy infrastructure code across various virtual computing environments. Additionally, the described techniques enable infrastructure management within these environments. In some examples, service teams may write code that is expected to be deployed to one or more, but often many, different production environments (e.g., across various geographical locations, sometimes spanning the entire world). However, in some examples, the infrastructure on which the code will be deployed must first be set up. In some cases, provisioning can be done manually, provisioning tools can be used to provision resources, and / or deployment tools can be used to deploy the code once the infrastructure is provisioned.

[0041] As mentioned above, there are typically two different tools for controlling infrastructure resources, one for handling the provisioning of infrastructure resources and the other for deploying code, with orchestration between these two tools being performed manually. However, at a certain scale, manual implementation always leads to deviations. Therefore, automated tools for provisioning and deploying virtual infrastructure enable more efficient and reliable technologies for realizing virtual cloud environments.

[0042] In some examples, problems can arise when users manually make changes to code between the provisioning and deployment phases, especially when two tools are used. As described in this article, techniques that use a single tool for both provisioning and deployment can mitigate this by automating the process, eliminating the need for manual code changes. There may be situations where a slight change to how a user codes something can create significant problems during deployment. In some examples, when an operator first performs an action in a new area (e.g., a typo in the code), the object coded with the typo may remain unchanged forever. If the application is deployed with that typo, and the application is insensitive to the typo (e.g., it still works), it is possible that at some point in the future, additional code changes might become sensitive to the typo and crash the entire system. Therefore, the techniques presented in this article can remove the gap between provisioning and deployment, which can often lead to problems.

[0043] Typically, deployment modeling is declarative, making configuration files available for declarative purposes regarding infrastructure resources. For example, create, read, update, delete (CRUD) instructions are often used to generate deployment files using general representational state transition (REST) ​​concepts (e.g., REST application programming interfaces (APIs)). However, deployment itself typically does not follow the same concept. Additionally, while infrastructure provisioning tools tend to be very powerful and / or expressive, deployment tools tend to be more restrictive about the operations they can perform (e.g., they are imperative rather than declarative). Therefore, there has long been a need for a tool capable of handling both functional requirements within a cloud environment (e.g., provisioning and deployment of infrastructure elements).

[0044] In some examples, this document describes technologies used to implement Cloud Infrastructure Orchestration Services (CIOS). As briefly described above, such technologies can be configured to manage the provisioning and deployment of infrastructure assets within a cloud environment. In some cases, CIOS may include two types of services: central and regional components (e.g., CIOS central and CIOS regions). The following terms will be used throughout the document:

[0045] • Infrastructure components – Long-lived infrastructure components that support running code.

[0046] Examples: Deploying applications, load balancers, Domain Name System (DNS) entries, object buckets, etc.

[0047] • Artifacts – Code that is being deployed to a deployment application or Kubernetes engine cluster, or configuration information (hereinafter, “config”) that is being applied to infrastructure components. These may be read-only resources.

[0048] • Deployment tasks – Short-lived tasks typically associated with deploying or testing code. Additionally, deployment tasks are modeled as having a lifespan that does not exceed the length of the resource that created them.

[0049] oExamples: "deploy$artifact to$environment", "watch$alarm for 10minutes", "execute$testSuite" or "wait for$manualApproval"

[0050] For example, CIOS can model deployment orchestrator deployments as creating resources that are transformed into an available state when they are completed.

[0051] Because CIOS maintains the state of its cloud infrastructure service declarative providers, CIOS can control the lifecycle of these short-lived resources as it is release-dependent.

[0052] • Resources – Resources that can be CRUD

[0053] CIOS models each of the constructs listed above as a resource. This modeling is discussed in detail in the next section.

[0054] • Groups – CIOS encapsulates the control region and a model of all its components. They exist primarily to model ownership of infrastructure components and point to those components.

[0055] · Group config – describes the collection of all infrastructure components, artifacts, and deployment tasks associated with a single service.

[0056] Each group has only one group config. The group config check-in source is controlled.

[0057] The group configuration is declarative. They expect CIOS to provide the domain, region, ad, and artifact version as input.

[0058] oGroups are fine-grained—groups consist of individual services and supporting infrastructure.

[0059] • Status — A snapshot of the status of each resource in the group at a specific point in time.

[0060] • Release — A tuple of the group config for a specific version and the specific version of each artifact it references.

[0061] o views a post as describing a state that may not yet exist.

[0062] • Release Plan – The set of steps CIOS will take to transition all regions from their current state to the state described in the release.

[0063] The release plan has a limited number of steps and clearly defined start and end times.

[0064] • Application – This is a noun. A single attempt to execute a release plan. An action that changes the current state of a group.

[0065] CIOS can be described as an orchestration layer that applies configurations to downstream systems (e.g., worldwide). It is designed to allow worldwide infrastructure provisioning and code deployment without manual work from service teams (e.g., beyond initial approval in some cases). The advanced responsibilities of CIOS include, but are not limited to:

[0066] • Provide the team with a view of the current status of resources managed by CIOS, including any ongoing change activities.

[0067] • Help the team plan and release new changes.

[0068] • Coordinate activities across various downstream systems within the region to execute approved release plans without human intervention.

[0069] • Coordinate activities across regions / sectors to implement approved release plans worldwide.

[0070] In some examples, CIOS handles onboarding by enabling teams to provide configuration information to CIOS via check-in code. Additionally, CIOS can automate much more, making this a more substantial practice compared to previous implementations. In some cases, CIOS handles pre-deployment by providing teams with the ability to automatically deploy and test code. In others, CIOS handles change management (CM) policy writing by enabling the automatic generation of plans to roll out new artifacts (e.g., worldwide) as teams build them. It does this by checking the current state of each region and the current CIOS config (which itself can be an artifact). Additionally, teams can review these plans and iterate on them by changing the CIOS config and requesting CIOS to re-plan. Once the team is satisfied with the plan, they can create a "release" referencing the plan. The plan can then be marked as approved or rejected. While teams can still write CMs, they are merely pointers to CIOS plans. Therefore, teams can spend less time reasoning about plans. Plans are more accurate because they are machine-generated. The plan is far too detailed for human processing; however, it can be displayed through a sophisticated user interface (UI).

[0071] In some examples, CIOS can handle the execution of Change Management (CM) by automating the execution of deployment plans. Once a release plan is created and approved, engineers will no longer be involved in CM unless CIOS initiates a rollback. In some cases, this may require teams to automate tasks that are currently performed manually. In some examples, when CIOS detects a decline in service health during execution, it can handle rollback change management (CM) by automatically generating a plan to return the group to its original (e.g., pre-release) state. In some examples, CIOS can handle urgent / tactical deployment changes by receiving release plans that cover a subset of regions and / or resources managed by CIOS and then executing those plans.

[0072] Additionally, CIOS supports defining the primitives necessary for fully automated, worldwide deployments. For example, CIOS can measure service health by monitoring alerts and executing integration tests. In the event of service degradation, CIOS can help teams quickly define rollback behavior, which can then be executed automatically. CIOS can automatically generate and display release plans and track approvals. In some cases, the language teams use to describe desired deployment behavior can be declarative. Within a system, CIOS can combine code deployment and infrastructure config (e.g., provisioning) capabilities. CIOS also supports flexible ordering of components across regions and across regions. Teams can express ordering via check-in configs. Teams can programmatically invoke CIOS's planning and release APIs.

[0073] Figure 1 An architecture 100 is depicted, illustrating the technologies used to at least implement a CIOS center 102. In some examples, the CIOS center 102 may be a service that processes operations at the "group" level. The CIOS center 102 has several responsibilities, including but not limited to:

[0074] • Serves as an authentication gateway for group metadata changes and publishing operations.

[0075] • Store group metadata for groups and provide an authoritative mapping to deployment artifacts and CIOS repositories.

[0076] • Global release with coordinated phases and objectives.

[0077] • Synchronize to implement strategies such as "no more than one ongoing group post at a time".

[0078] • Detect changes to group configurations and artifacts, and trigger the generation of releases for such changes.

[0079] In some examples, the Source Code Version Control Management Service (SCVMS) 104 can be configured to store authority group configurations, and the Artifact Notification Service (ANS) 106 can be subscribed to by the CIOS Center 102, allowing the CIOS Center 102 to be notified of new artifact builds. The CIOS Center 102 can then map incoming changes to the affected groups and initiate release plans where desired. Additionally, in some examples, the Artifact Push Service (APS) can be invoked by the CIOS Center 102 prior to release to the target region to ensure that any artifacts required for successful release before the target region is present.

[0080] In some examples, a customer (e.g., an engineer) 108 can invoke CIOS Central 102 to CRUD groups and / or publish, and view the status of ongoing CIOS activities. Group Management Service 110 may include one or more APIs to manipulate groups, View / Schedule / Approve Service 112 may include a CRUD API to create and approve plans, and view a central copy of the status of all CIOS-managed resources, Change Monitoring Service 114 can monitor SCVMS 104 for changes to group configurations and can receive notifications from ANS 106 regarding changes to other artifacts, and Status Ingestor Service 116 can create copies of region statuses in CIOS Central Database (DB) 118, allowing View / Schedule / Approve 112 to expose them. In some examples, CIOS Central DB 118 may be a DB for groups, plans, and statuses. Group information can be authoritative; everything else can be stale copies of data from CIOS Region 120. CIOS Center 102 can be configured to provide any suitable portion and / or number of user interfaces (e.g., user interfaces 500-1300) for presenting any suitable data related to groups, releases, infrastructure components, artifacts, etc. In some embodiments, CIOS Center 102 can present data via any suitable interface data related to one or more releases. A release can include any suitable combination of tasks related to one or more infrastructure components and / or one or more code changes to one or more applications (e.g., artifacts). Some examples of the user interfaces provided by CIOS Center 102 are described below. Figures 5-13 Describe it.

[0081] In some examples, Engineer 108 can (e.g., via Ingress Agent Cluster 122) make API calls to Group Management Service 110 to create a list of groups. The protocol for making such API calls could be Secure Hypertext Transfer Protocol (HTTPS), etc. The relevant Access Control Lists (ACLs) used for this operation could include a Local Area Network (LAN) 124 or other private connection. For example, CIOS can manage / control alternative network connections (e.g., dedicated, leased, and / or private connections) that use the public internet to connect a customer's local data center or network to CIOS. Additionally, authentication and authorization (e.g., for Engineer 108) can be performed by a reservation system portal (e.g., a reservation service) that allows users to manage machine infrastructure. In some cases, CIOS Center 102 can use Java Database Connectivity (JDBC), etc., to store group metadata, schedules, and status in a central DB 118. In some examples, ANS 106 can be configured to notify Change Monitoring Service 114 when a new artifact has been published. ANS 106 can use HTTPS, and both authentication and authorization can be handled by mutual transport layer security services. Additionally, in some cases, Change Monitoring Service 114 can poll SCVMS 104 for group configuration changes. This polling can be performed using Secure Shell (SSH) or other protocols. Authentication for Change Monitoring Service 114 can be handled by the CIOS system account, and authorization can be handled by SCVMS 104.

[0082] In some examples, Engineer 108 can use the View / Schedule / Approve service 112 to perform one or more of the following operations: Engineer 108 can schedule and / or approve plans by calling CIOS Center 102 to generate and approve plans. Engineer 108 can view the status of ongoing CIOS activities worldwide by calling CIOS Center 102. Additionally, Engineer 108 can view a copy of the status of CIOS-managed resources worldwide through CIOS Center 102. These API calls (etc.) can be performed via HTTPS or similar protocols. Additionally, the relevant ACLs can be controlled by LAN 124, and both authentication and authorization can be handled by the scheduling service. In some examples, the View / Schedule / Approve service 112 can request a plan and (e.g., using HTTPS, etc.) push plan approval to all areas of CIOS Zone 120. The relevant ACLs can be controlled using a security list managed by Wide Area Network (WAN) Gateway 126. Authentication can be handled through mutual transport layer security, and authorization can be handled through various identity policies. Furthermore, the State Ingestor Service 116 can monitor the CIOS Zone 120 to obtain job status or status changes, allowing the CIOS to provide a central view of themselves upon request (e.g., also using HTTPS). ACLS for this purpose can also be handled by the WAN Gateway 126, and both authentication and authorization can be handled by mutual transport layer security services.

[0083] Figure 2 An architecture 200 is depicted illustrating the technologies used to at least implement CIOS Zone 202. In some examples, CIOS Zone 202 is where most of the work on declarative provisioning and planning, as well as the approved release of applications, can occur. In some cases, each instance of CIOS Zone 202 can have a zone frontend that can handle operations at the "execution target" level. It can be configured to perform the following:

[0084] • Process all CIOS authentications from incoming operations originating from CIOS Center 102.

[0085] • Implement a rule that only one "execution" (plan / import resources / apply plan) can be in progress at a time for a given execution goal.

[0086] • Manages a binary artifact store for declarative infrastructure provisioning artifacts used as inputs and outputs during declarative infrastructure provisioning execution. Examples of inputs are declarative infrastructure provisioning configuration files and input state files. Typical output is a final state file.

[0087] • For any given execution, request work from the CIO executor and poll for the results from the CIO executor.

[0088] In some cases, the CIOS frontend can rely on the CIOS executor 206 (also referred to as the "scheduler" in this document), which handles the actual execution. In some examples, the CIOS executor runs at the "execution" level, and it can:

[0089] • Track the pool of available worker nodes

[0090] • Query incoming job requests and assign them to eligible workers when available.

[0091] • Track worker status and perform updates for reporting to clients.

[0092] • Dead nodes can be detected via a lease protocol, and tasks assigned to dead nodes can be made to fail, depending on the task status.

[0093] • Provides facilities for canceling / terminating / pausing / resuming execution, and can map those executions to facilities to pass cancellation / termination / resumption information to worker nodes.

[0094] In some cases, a CIOS executor can rely on CIOS workers, which can assign tasks for execution to workers and provide facilities for workers to update job progress. Worker services operate at the "task" granularity. Each worker is an agent that executes the tasks assigned to it and reports task status and output. Each worker can:

[0095] • Poll the executor worker API for the assigned work item and take action.

[0096] To make the allocation state match its local state:

[0097] o Start a container for polling for tasks that do not exist locally.

[0098] o Terminates containers that are running locally and do not have corresponding assigned task items.

[0099] • Report work status

[0100] • Causes the inputs and outputs of the job container to occur

[0101] • Launch and monitor the declarative infrastructure provisioning container for the actual work of releasing against the execution objectives.

[0102] CIOS workers can rely on CIOS executors to poll for work from worker endpoints of the CIOS executors and report results to them. For all coordination, workers can rely on the executor. Additionally, CIOS workers can also rely on CIOS Zone 202, where worker services read input from one or more APIs associated with the zone frontend service and write output to it. Examples of input are configuration and startup state files and import mappings. Examples of output are declarative provisioning procedures, output declarative provisioning state files, and import result states.

[0103] In some examples, CIOS Zone 202 can be a zone service used to manage zone instances / deployments of CIOS. CIOS Zone 202 is responsible for authoritatively storing and managing plans and states related to a specific zone. Zone DB 204 can be a CIOS DB used for the state and plans within a specific zone. This is Figure 1 The central DB 208 is an authoritative copy of a subset of the CIOS regions. Scheduler 206 can be responsible for managing worker cluster capacity, assigning tasks to workers, and keeping track of task status. In some cases, Task DB 208 is another CIOS DB used for task status. The data in this DB is primarily for operational purposes. Additionally, workers 210 can be a cluster of Java Virtual Machines (JVMs) that manage declarative provisioning images. These receive instructions from scheduler 206 and transmit results to both scheduler 206 and CIOS regions 202. CIOS container 212 can run declarative provisioning actions in its own private Docker container 214. This container does not need to contain secrets. Additionally, in some examples, signing broker 216 can be configured to prevent secret leakage via declarative provisioning tools to avoid including secrets in the declarative provisioning image. Instead, CIOS can perform request signing or initiate mutual transport layer security (mTLS) services within the broker. This also makes it easier to use FIPS-compliant cryptographic libraries.

[0104] In some examples, CIOS center 102 can invoke CIOS zone 202 to create schedules, push approvals, monitor job status (service principal), and retrieve declarative provider status (service principal). Ingress proxy 218 can be configured as an ACL, and various identity policies can be used for both authentication and authorization. Alternatively, in some examples, ingress proxy 218 can be replaced by a load balancer configured to balance the load of incoming requests, schedules, etc. In some cases, CIOS zone 202 can run a declarative provider by requesting scheduler 206 to run the declarative provider. Worker 210 can query scheduler 206 what it should run and can report the status to scheduler 206 upon completion. In some cases, mTLS can handle both authentication and authorization for CIOS zone 202 and worker 210. Additionally, when worker 210 needs to run a declarative provider, it does so within a Docker container by interacting with local Docker 214. Authentication at this stage can be handled by local Unix sockets. The Docker protocol can be used for this final step; however, HTTPS can be used for the preceding steps.

[0105] In some embodiments, CIOS area 202 may be configured to provide any suitable portion and / or number of user interfaces (e.g., user interfaces 500-1300) for presenting any suitable data related to groups, releases, infrastructure components, artifacts, etc. In some embodiments, CIOS area 202 may be presented via any suitable interface data related to one or more releases. A release may include any suitable combination of tasks related to one or more infrastructure components and / or one or more code changes to one or more applications (e.g., artifacts). Some examples of user interfaces provided by CIOS area 202 are described below. Figures 5-13 Describe it.

[0106] In some examples, CIOS container 212 enables declarative providers to interact (via API) with signing agent 216, which perceives itself as invoking various CIOS services. Signing agent 216 listens on a temporary port known only to the declarative provider for each invocation instance. Signing agent 216 can initiate request signing or mTLS and can pass the declarative provider's invocation to other CIOS services within the service enclave. In some cases, signing agent 216 may also communicate with one or more public CIOS services 220. For example, signing agent 216 will use internal endpoints of public services where possible. For services without internal endpoints, it must use egress agent 222 to reach external endpoints. This use of signing agent 216 may not be for cross-region communication; for example, the egress agent whitelist in each region may only be for public IP ranges within that region. In some examples, worker 210 can then persist the state and logs from declarative providers in CIOS region 202, allowing them to leak to CIOS hub 102.

[0107] Using CIOS, there are several phases that a representative customer experiences: Login, Pre-release, Worldwide Release, and Tactical Release. For the Pre-release phase, the following is an example of what happens between when a new artifact is being built and released, and Release 1 (e.g., R1). This should replace some or most of the current change management process. As the relevant artifact is built, CIOS can automatically generate a release using "Latest Version of Everything in the Group." A release is a specific version of a group configuration with specific inputs such as artifact version, domain, region, and ad. A release contains a roll-forward plan for each region and metadata describing the region ordering. Each region plan is a set of actions that a declarative provider will take to implement the group configuration in that region. Teams with a pre-release environment can use CIOS to automatically release and test software in that environment. Teams can configure CIOS to automatically test rollback plans. Teams will be able to inspect and approve releases via the CIOS UI. Teams can approve some, but not all, of the region plans within a release. If "Latest Version of Everything" does not produce a suitable plan, the team can request CIOS to generate a plan for a carefully selected artifact version.

[0108] For the global release phase, here's an example of how a team executes a tomorrow's version of today's "normal CM". Once the release is approved, CIOS pushes each approved regional plan to the corresponding region. CIOS acts independently within each region to apply the approved plan. CIOS will only execute the set of actions explicitly described in that regional plan. It will fail, not "think independently". The CIOS UI shows the team the execution progress. The CIOS UI prompts the team when manual approval is required. If execution fails due to CIOS or downstream service outages, CIOS can notify the team and prompt them for the following steps (e.g., abort, retry). CIOS does perform retries, but some downstream system outages will exceed its retrying intentions. If execution fails due to a decline in service health or a failed test, CIOS will assist the team in rolling the group back to its starting state. CIOS will notify (e.g., page) the team when it initiates an automatic rollback. The team must approve the rollback plan, and then CIOS will execute it.

[0109] For the tactical release phase, here's an example of how a team can execute a tomorrow's version of an "urgent CM." When generating a plan, the team might ask CIOS to target the plan to specific resources in several ways: by topology (e.g., domain, region, AD, etc.), by resource type (e.g., "metrics config only" or "deploy orchestration service deployment only"), or a combination of these (e.g., in a separate manner). The team approves tactical releases just like worldwide releases. CIOS orchestrates them in a similar way. If the team needs to deploy a tactical release while an active worldwide release exists, CIOS will stop executing the worldwide release in the target region and then begin executing the tactical release.

[0110] In some examples, the state of a declarative provider (e.g., traditionally a file) is the authoritative record of the resource group managed by the declarative provider. It contains a mapping between the logical identifier of each resource from the configuration file and the actual identifier of the resource. While the declarative provider is creating a resource, certain kinds of failures can prevent the actual identifier from being recorded in the state. When this happens, the actual identifier is lost for the declarative provider. These can be referred to as "orphaned resources."

[0111] For most resources, orphanage represents waste—a declarative provider starts (for example) an instance it forgot, but instead starts another instance on its next run. For resources with unique constraints or client-provided identifiers, orphanage prevents the declarative provider from making progress. For example, if a declarative provider creates user "nglass" and fails to orphanize it, then the next run of the declarative provider will attempt to create "nglass" and fail because a user with that username already exists. In some cases, orphanage is simply a problem when adding new resources to the state. In some cases, the declarative provider's refresh behavior can naturally recover from failures to record updates and deletions.

[0112] CIOS needs to be robust in the event of downstream service outages or CIOS itself failures. Because CIOS can utilize declarative providers to apply changes, robustness must exist around running and maintaining declarative providers' state. Declarative providers perform "small-scale" retries—sufficient to avoid outages lasting a few minutes. For example, a cloud provider might retry for up to 30 minutes. A downstream system outage lasting longer than 30 minutes will cause the declarative provider to fail. When a declarative provider fails, it logs all changes it successfully made in its state and then exits. To retry, CIOS must re-execute the declarative provider. Re-executing the declarative provider also allows CIOS to retry in the event of a CIOS failure. In some cases, CIOS can cycle through the following operations:

[0113] • Refresh — A declarative provider calls the GET API to retrieve a new snapshot of each resource described in its state.

[0114] • Planning – Taking into account the most recently refreshed current state, the declarative provider generates a plan (a set of specific API calls) that will achieve the desired state.

[0115] • Application – A set of steps in a declarative provider execution plan.

[0116] When executing a declarative provider, CIOS can always run all three steps. The refresh operation helps recover from any updates or deletions that were never recorded. CIOS checks the results of the planned operations and compares them to the approved release plan. If the newly generated plan contains operations not in the approved release plan, CIOS may fail and can notify the service team.

[0117] Figure 3A directed acyclic graph (DAG) 300 is depicted for the example group 302. For a single group config in CIOS, the process of code / config checking in to production can be described from the initial test deployment to the final production (prod) deployment. Internally, CIOS calls each element in the process, executing an objective (ET)—this is permeated throughout our internal API and does not leak into the group config. CIOS executes ETs based on the DAG 200 defined in the group config. Each ET (e.g., ET-1, ET-2, ET-3, ET-4, ET-5, ET-6, and ET-7) is roughly a copy of the service described in the group config.

[0118] Figure 4 The diagram illustrates DAG 400 for example group 402. In the group configuration, CIOS is very strict about how teams represent this process—they must model it using cloud infrastructure leases and regions. Teams should not use domains to model processes. CIOS allows teams to use multiple leases within a domain and multiple regions within a lease. However, CIOS does not allow teams to use the same region twice within a lease (although they can use the same region twice within a domain—in different leases). DAG 400 illustrates the representation of processes using leases and regions. Figure 3 This is a version of DAG 300. This example is for coverage services where the pre-production ET is located in the production area. Service enclave services will have unstable and stable leases in Release 1. In DAG 400, IAD is the area airport code for Dulles Airport in Washington, D.C., YYZ is the area airport code for Toronto, Ontario, PHX, LHR, and FRA are the area airport codes for Phoenix, London, and Frankfurt, respectively, and LUF and LFI are used for two different air force bases.

[0119] In one embodiment, the CIOS and / or other techniques described herein are improvements on each of Terraform (a declarative provisioning tool), Tanden (a code generation tool), and Oracle Deployment Orchestrator (ODO). Additionally, in some examples, the CIOS and / or other techniques described herein may be implemented using at least a portion of the Terraform, Tanden, and ODO tools.

[0120] Figure 5This is a user interface (UI) 500, according to at least one embodiment, presenting information related to a release including multiple stages and multiple execution targets. UI 500 may include a stage area 502 and an execution target area 504. Stage area 502 may indicate an ordered list of stages, such as stage 506 (e.g., "R_n"), stage 507 (e.g., "R_s"), stage 508 (e.g., "R_e"), and stage 509 (e.g., "R_w"). In some embodiments, the stage order may be depicted from left to right, where the leftmost stage completes before its rightmost stage, which in turn completes before its rightmost stage, and so on. In some embodiments, the ordered list of stages may be stored in any suitable data structure such as a linked list. An example linked list is provided below. Figure 7 It will be discussed in more detail.

[0121] Each phase can be associated with multiple tasks (e.g., tasks that include deploying one or more infrastructure resources to one or more execution targets). The list of phases shown in UI 500 includes four phases, but any suitable number of phases can be included in the phase area 502 for deploying infrastructure resources at one or more execution targets. In some embodiments, the ordered list of phases presented within the phase area 502 can be horizontally scrollable. The phase area 502 may indicate the total number of phases 510, the status 511, the number and / or total number of execution targets completed 513, and a group configuration identifier 514. The total number of phases 510 may indicate the total number of phases included in the linked list, and the status 511 may indicate the status of the release (e.g., release includes). Figure 5 As shown, status 511 is "Applying", but status 511 can be any suitable status indicator (e.g., "Not Started", "Completed", "Failed", etc.). Figure 5 As shown, the number and / or total number of completed execution targets 513 indicates that 24 deployments of execution targets out of a total of 57 execution targets have been completed. The number and / or total number of completed execution targets 512 can be presented on UI 500 in any suitable manner. The group configuration identifier 514 is presented on UI 500 as "13380fb2832", but the group configuration 514 can be presented on UI 500 in any suitable manner to convey the corresponding Figure 5 The unique identifier of the published group profile as described in the document.

[0122] Each sub-region corresponding to a stage may include a stage identifier (e.g., stage identifier 516), a total execution target indicator (e.g., total execution target indicator 518), timestamp information (e.g., timestamp information 520), an execution target tracker region (e.g., execution target tracker region 522), and any other suitable information associated with the stage. For example, identifier 516 (e.g., "R_s") may be any suitable alphanumeric string used to uniquely identify the stage. In some embodiments, the total execution target indicator 518 may be presented adjacent to identifier 516. The total execution target indicator 518 indicates the number of total execution targets associated with a given stage (e.g., 14).

[0123] The timestamp information 520 may include a start time and / or end time, respectively, associated with the start and / or completion times of the phase. In some embodiments, the timestamp information 520 may include any suitable indicators for indicating the status of the corresponding phase (e.g., "not started," "failed," "completed," etc.). In some embodiments, any suitable information associated with a phase may be depicted in the phase area 502. Information associated with each phase may be presented similarly, or the presentation of phase information may be formatted differently and / or may include more... Figure 5 More or less information is described in the text.

[0124] An execution target tracker area (e.g., execution target tracker area 522 of stage 506) may be presented within each stage. The execution target tracker area of ​​each stage may include one or more execution target indicators (e.g., execution target indicators 524, 526, and 528). Each execution target indicator may include a number indicating the number of tasks to be executed concurrently (e.g., deployments to a corresponding execution target). For example, execution target indicator 524 indicates that a deployment to a specific execution target can be executed. Execution target indicator 526 and its placement to the right of execution target indicator 524 indicate that another deployment to a different execution target will be executed after the first task corresponding to execution target indicator 524 is completed. Execution target indicator 528 and its placement to the right of execution target indicator 526 indicate that 12 separate deployments to 12 different execution targets will be executed after the second task corresponding to execution target indicator 524 is completed. The combination of execution target indicators 524, 526, and 528 collectively depicts a folded form of a directed acyclic graph (e.g., hereinafter combined with...). Figure 9 The DAG 900 under discussion.

[0125] In some embodiments, execution target indicators 524-528 may correspond to a data structure configured to maintain a list of execution targets associated with a phase and the order in which tasks (e.g., deployments to each execution target) are to be executed. An example data structure for maintaining this list and order is provided. Figure 9 This is described in more detail. Each execution target indicator may include a ring (e.g., ring 530). The ring may be divided into any suitable number of sections corresponding to the number of execution targets associated with the execution target indicator. In the depicted example, ring 530 may be divided into 12 equal sections. Each section of ring 530 may correspond to a specific execution target and may be colored with a color corresponding to the task status corresponding to that specific execution target. For example, ring 530 may (e.g., via nine green sections) indicate that nine deployments to the execution targets have been completed. The remaining three sections of ring 530 (e.g., colored white) may indicate that three tasks corresponding to three execution targets have not yet started. As another example, the remaining three sections of ring 530 may be colored with a different color (e.g., yellow) to indicate that the corresponding tasks have started but have not yet been completed. Thus, by observing ring 530, the user can quickly and visually identify the current status of the tasks associated with execution target indicator 528. By viewing the execution target indicators within execution target tracker area 522, the user can determine that 11 execution targets have been deployed and 3 are in progress. In some embodiments, each execution target indicator may correspond to a directed acyclic graph (DAG) associated with a given stage (e.g., "R_s"). Figure 9 The nodes of the DAG (900).

[0126] The execution target area 504 may include an execution target bar 532, a progress bar 534, and an action bar 536. The execution target bar 532 may include a list of targets (e.g., execution targets) corresponding to tasks (e.g., deployments) performed in conjunction with a given phase (e.g., phase 506 as shown). In some embodiments, the execution target bar 532 may be ordered chronologically or by any other suitable method for ordering task execution. The progress bar 534 may include a list of progress indicators indicating the status corresponding to each execution target (e.g., "successful," "not started," "in progress," etc.). The action bar 536 may include a list of operations to be performed relative to a given execution target indicated in the execution target bar 532. For example, the list of operations included in the action bar 536 may include create, read, update, and delete (CRUD) operations, etc.

[0127] Figure 6 This is an example code segment 600 for defining a list and order of stages according to at least one embodiment. For example... Figure 6As shown, four phases are defined in code segment 600. Each phase is defined as a resource of type "phase" and assigned an identifier (e.g., "R_n", "R_s", "R_e", and "R_n"). As shown in code segment 600, resources 602, 604, 606, and 608 correspond to the respective phases. Each phase may include one or more variables, one of which may include an indicator used to indicate the execution order of each phase. In some embodiments, the indicator may indicate dependencies on one or more other phases and / or the absence of dependencies. For example, indicator 618 (e.g., `predecessors=[]` in line 5) may indicate the absence of dependencies on any other phase. This can be interpreted by the system as defining the first phase to be executed. Indicator 620 (e.g., `predecessors=[phase.R_n.variable 1]`) may be used to indicate a dependency on another phase (e.g., phase "R_n") by including an assignment corresponding to the value of a variable associated with another phase (e.g., phase "R_n"). Similarly, indicator 622 can indicate a dependency on another stage (e.g., stage "R_s", by including the assignment of the value corresponding to the variable associated with "R_s"), and indicator 624 can indicate a dependency on yet another stage (e.g., stage "R_e", by including the assignment of the value corresponding to the variable associated with "R_e").

[0128] In some embodiments, code segment 600 may be included in a configuration file corresponding to the release. As part of the execution preprocessing, the configuration file may be parsed / traversed once or multiple times. During one of these traversals, code segment 600 may be analyzed to generate a data structure that can be used to maintain the identity and execution order corresponding to one or more stages. Figure 7 An example data structure (e.g., a linked list) that can be used to maintain this information is described.

[0129] Indicators 618-624 can be used to establish the specific order in which the phases are to be executed. For example, ... Figure 6 The description shows that the execution sequence is as follows: first stage "R_n", then stage "R_s", then stage "R_e", and finally stage "R_w". It should be understood that dependencies can indicate one or more other stages to be completed before the task associated with a given stage begins. Although in Figure 6 Four phases are defined, but it should be understood that any appropriate number of phases can be defined in a similar manner.

[0130] Figure 7This is an example data structure (e.g., linked list 700) generated by the Cloud Infrastructure Orchestration Service (CIOS) to maintain lists and sequences associated with one or more stages. Linked list 700 can be generated to identify, for example,... Figure 6 The stages and execution order are defined in code segment 600. For example... Figure 7 As shown, linked list 700 includes four nodes 702, 704, 706, and 708, each of which can correspond to one of the four phases defined in code segment 600. Each node of the linked list can correspond to a data object configured to store any suitable information corresponding to a given phase. For example, a given node can store any suitable number of variables, identifiers, data structures, pointers, references, etc., corresponding to a particular phase. As a non-limiting example, node 702 corresponding to phase "R_n" can store three variables defined in code segment 600 corresponding to phase "R_n". Each node of linked list 700 can include any suitable number of variables corresponding to a given phase.

[0131] Each node in the linked list may include a pointer / reference to another node in the list (in scenarios with multiple stages). For example, node 702 may include a reference to node 704, node 704 may include a reference to node 706, node 706 may include a reference to node 708, and node 708 may (e.g., via a null pointer) indicate that it is the end node of linked list 700. In some embodiments, these pointers / references may be identified based on indicators 618-624 of code segment 600.

[0132] In some embodiments, during execution, CIOS may identify the order in which specific stages to be executed based at least in part on traversing linked list 700 starting at node 702 (e.g., the starting node). The execution of these stages may utilize any suitable combination of data stored within each corresponding node. Once the operation corresponding to a given stage is completed, CIOS may traverse to the next stage, repeating this process as many times as appropriate until the operation corresponding to the end node of linked list 700 (e.g., node 708) has been completed. In some embodiments, if the operation at a given node is unsuccessful (e.g., an error occurs), CIOS may not traverse to the next node, but may instead halt deployment and return a notification to alert the user to the situation.

[0133] Each node of linked list 700 can correspond to a data structure configured to identify and maintain the execution order corresponding to one or more execution targets. Figure 8 and Figure 9 The definition and use of such a data structure are discussed in more detail. It should be understood that linked list 700 can provide... Figure 5UI 500 is used to present any appropriate information associated with the stage (e.g., identifier 516, timestamp information 520, execution target 518, etc.).

[0134] Figure 8 This is an example code segment 800, according to at least one embodiment, for defining a list and order of execution targets (e.g., execution targets corresponding to a specific stage). Figure 8 As shown, four execution targets are defined in code segment 800. Each execution target is defined as a resource of type "execution target" and assigned an identifier (e.g., "us-la", "us-sj", "us-sf", and "us-sd"). As shown in code segment 800, resources 802-808 each correspond to a unique execution target. Each execution target may be associated with one or more variables, one of which may include an indicator for indicating the execution order of each stage. In some embodiments, the indicator may indicate one or more dependencies and / or lack of dependencies on one or more other execution targets. For example, indicator 818 (e.g., predecessors = [] in line 5) may indicate the lack of dependencies on any other stage. This can be interpreted by the system as defining the first execution target. Indicator 620 (e.g., predecessors = [execution_target.us-la.variable 1]) may be used to indicate a dependency on another execution target (e.g., stage "us-la") by including an assignment corresponding to the value of a variable associated with another execution target (e.g., execution target "us-la"). Similarly, indicator 822 can indicate a dependency on another stage (e.g., execution target "us-sj", by including the assignment of values ​​corresponding to variables associated with execution target "us-sj"), and indicator 624 can indicate a dependency on yet another stage (e.g., execution target "us-sf", by including the assignment of values ​​corresponding to variables associated with execution target "us-sf").

[0135] In some embodiments, code segment 800 may be included in a configuration file corresponding to the release (e.g., the same or a different configuration file including code segment 800). As part of the execution preprocessing, the configuration file may be parsed / traversed once or multiple times. During one of these traversals, code segment 800 may be analyzed to generate a data structure that can be used to maintain the order of identities and execution objectives corresponding to a given stage. Figure 9 An example data structure (e.g., a directed acyclic graph (DAG)) that can be used to maintain this information is described.

[0136] Instructions 818-824 can be used to define the specific order in which the execution target should be performed. For example, ... Figure 8 The description proceeds first with the task associated with execution target "us-la", then with the task associated with execution target "us-sj", then with the task associated with execution target "us-sf", and finally with the task associated with execution target "us-sd". It should be understood that dependencies can instruct one or more other execution targets to complete corresponding tasks for that one or more other execution targets before the tasks associated with a given execution target begin. Although in Figure 8 Four execution goals are defined, but it should be understood that any suitable number of stages can be defined in a similar manner. In some embodiments, execution goals may share common dependencies (e.g., the same predecessor definition). Common dependencies can be used to indicate that tasks associated with execution goals that share common dependencies can be executed concurrently.

[0137] Figure 9 This is an example data structure (e.g., a directed acyclic graph (DAG) 900) that can be generated by a cloud infrastructure orchestration service (CIOS) to maintain a list and order associated with one or more execution targets associated with a phase, according to at least one embodiment. According to at least one embodiment, DAG 900 may be one of multiple DAGs generated in response to the CIOS parsing one or more configuration files associated with a release. Each node of DAG 900 may correspond to a single execution target. Figure 9 As shown, DAG 900 includes six nodes (e.g., nodes 902, 904, 906, 908, 910, and 912), each of which can correspond to a node with respect to… Figure 8 One of six execution targets defined in a similar manner as described in code segment 800. Each node of DAG900 can correspond to a data object configured to store any suitable information corresponding to a given execution target. For example, a given node can store any suitable number of variables, identifiers, data structures, pointers, references, etc., corresponding to a particular execution target.

[0138] Each node of DAG 900 may include pointers / references to one or more nodes in DAG 900. For example, node 902 may include a reference to node 904, which in turn may include references to nodes 906-910, each of which may include a reference to node 912, which may (e.g., via a null pointer) indicate that it is the end node of DAG 900. In some embodiments, these pointers / references may be based on a similar combination as described above. Figure 8The indicators 818-824 discussed are used for identification. In some embodiments, nodes 906, 908, and 910 may share a common dependency on node 904, so tasks associated with nodes 906-910 can be executed concurrently at least partially. In some embodiments, node 912 may correspond to an execution target that depends on nodes 906-910. Therefore, tasks associated with an execution target corresponding to node 912 can only be executed after all tasks associated with execution targets corresponding to nodes 906-910 have been completed.

[0139] In some embodiments, the DAG 900 can be combined with the above. Figure 7 A single node of the linked list 700 discussed is associated with it. That is, one or more execution targets (e.g., identified and represented by nodes of DAG 900) may be associated with a specific stage (e.g., a single node) of linked list 700. In some embodiments, a node of linked list 700 may include a reference to DAG 900, or one or more nodes of DAG 900 may include references to nodes of linked list 700.

[0140] In some embodiments, CIOS can traverse configuration files (e.g., including...) Figure 8 The code segment 800 is used to generate DAG 900. The generation of DAG 900 can be done as part of a preprocessing procedure performed before runtime or at runtime. Once the operation corresponding to a given execution target is completed, CIOS can traverse to the next (or more) execution targets, repeating this process as many times as appropriate until the operation corresponding to one or more end nodes of DAG 900 (e.g., node 912) has been completed. In some embodiments, if the operation corresponding to a given node is unsuccessful (e.g., an error occurs), CIOS may not traverse to the next node but may instead return a notification to alert the user to the situation.

[0141] Each node of DAG 900 can correspond to a data structure configured to identify and maintain the execution order of one or more resources (e.g., services, software modules, etc.). Figures 10-12 The definition and use of such data structures (e.g., DAGs of capabilities) are discussed in more detail. For example, each node of DAG 900 can correspond to another DAG (e.g., Figure 12 The DAG 1200 indicates a list and order of resources and / or capabilities, identifying the order in which tasks are executed. It should be understood that the DAG 900 can provide information from... Figure 5The UI 500 is used to present any suitable information associated with the execution target indicators 524-528. Therefore, the information depicted in the execution target trace area 522 can depict a simplified version of the DAG 900 (corresponding to a DAG for a given stage, such as stage "R_s"). The simplified version of the DAG can compress the concurrent executable nodes of the DAG 900 into a single node (e.g., see the execution target indicator 528 depicting the compression of 12 nodes of the DAG into a single node). In some embodiments, the CIOS can deploy infrastructure resources and / or release software artifacts based at least in part on traversing the DAG 900. Specific tasks and task sequences are combined as follows. Figures 10-12 The description was identified.

[0142] Figure 10 This is an example code segment 1000 according to at least one embodiment for establishing explicit and implicit dependencies between resources of an execution target. Code segment 1000, as... Figure 10 As shown, the application includes two modules 1002 and 1004 and a resource 1006. Modules 1002 and 1004 each include names 1008 and 1010, respectively shown as "apps_example1" and "apps_example2". Modules can include names of any suitable length, including any suitable alphanumeric characters(s). Modules 1002 and 1004 can define the application / service that the user wishes to bootstrap or otherwise provision. Modules 1002 and 1004 can be used to deploy the application to availability domain 1 and availability domain 2, respectively. Resource 1006 can include a multi-parameter list, which includes... Figure 10 Resource type 1012, which is shown as "Type", and in Figure 10 The resource name 1014 is shown as "Executor". Resource type 1012 can be any type suitable for deployment, and resource name 1014 can be any name.

[0143] Resource 1006 can be a capability and can include implicit dependencies, explicit dependencies, or both. As depicted in code snippet 1000, resource 1006 attempts to assign the variable "variable1" to a value equal to "module.apps_example1.variable1", a value accessible via the apps_example1 module (e.g., module 1002). This is intended to depict the implicit dependency formed between resource 1006 and module 1002. The formed implicit dependency prevents resource 1006 from executing before module 1002 has been deployed. The process responsible for bootstrapping resource 1006 can receive notification that module 1002 has been deployed. The notification can be sent by a scheduler (e.g., ...). Figure 2The scheduler 206 sends the information and it can be received by the process responsible for bootstrapping resource 1006. The implicit dependencies formed are considered implicit because, as... Figure 10 The resource 1006 shown does not directly define the dependency relationship between module 1002 and resource 1006.

[0144] Conversely, Resource 1006 is Figure 10 Line 29 includes an explicit dependency, which explicitly defines the dependency between resource 1006 and the app_example2 module. As depicted in code snippet 1000 on line 29, resource 1006 includes "depends_on = apps_example2.variable2". Based on the code on line 29, the explicit dependency can be formed, and the explicit dependency prevents resource 1006 from being deployed until app_example2 has been successfully deployed. Once apps_example2 is successfully deployed, a notification can be sent by the scheduler and received by the process responsible for deploying resource 1006. Although Figure 10 Code segment 1000 includes a resource 1006 containing an implicit dependency and an explicit dependency; however, those skilled in the art should understand that any combination of resource 1006, the implicit dependency, and the explicit dependency can be used to achieve the goals of CIOS users.

[0145] CIOS (or a declarative infrastructure provider such as the declarative provisioning tool discussed above) can be used to parse configuration files including code snippets 1000. Through this parsing, CIOS (or the declarative provisioning provider) can generate a directed acyclic graph (DAG) for each resource, module, and / or capability, compiling and defining an ordered list of dependencies on other resources, modules, and / or capabilities. When attempting to deploy a resource, CIOS can traverse the DAG to identify when a resource depends on another resource, module, and / or capability. Each resource's DAG can specify implicit dependencies, explicit dependencies, or a combination thereof, and can be used to bootstrap or otherwise deploy the corresponding resource using CIOS.

[0146] Figure 11 This is an example code segment 1100 according to at least one embodiment for establishing explicit and implicit dependencies between resources of an execution target. For example... Figure 11 The code segment 1100 shown includes four resources 1102, 1104, 1106, and 1108. Each of the resources 1102, 1104, 1106, and 1108 may correspond to a capability and may include implicit dependencies, explicit dependencies, or a combination thereof.

[0147] like Figure 11The resource 1102 shown includes a name 1110 shown as "object_storage", a type 1112 shown as "type1" (e.g., indicating that the resource is a capability), and multiple variables (e.g., variables 1-3, although more or fewer variables may be used). Figure 11 Resource 1104, as shown, includes a name 1116 shown as "worker" and an explicit dependency 1118 shown as "depends_on = type1.object_storage". The parameter list for resource 1102 includes an identifier 1115 (or name 1110, which could be used similarly) that can be used to refer to this resource. Statement 1118 forms an explicit dependency on resource 1102 due to the reference to type1.object_storage. This explicit dependency prevents resource 1104 from being deployed until resource 1102 has been deployed. Figure 11 Resource 1106 shown includes an identifier 1120 labeled "peacock", a type (e.g., "type1" indicating capability), and multiple variables (e.g., variable1 and variable2). Figure 11 Resource 1108 shown includes a type (e.g., "type4"), a name 1124 shown as "LB", and a statement 1126 ("count = type1.peacock.exists"). The use of statement 1126 can form an implicit dependency on resource 1106. Although resource 1108 does not use explicit dependency construction (e.g., "depends_on"), the implicit dependency still exists because of the attempt to assign a value to the variable "count" equal to the existence of the capability "peacock" (as determined from the statement type1.peacock.exists). Therefore, due to the assignment attempted in line 18, resource 1108 may not be deployed until resource 1106 "peacock" is deployed. Although Figure 11 Code segment 1100 includes four resources 1102, 1104, 1106, and 1108, each containing one implicit dependency and one explicit dependency. However, those skilled in the art should understand that any combination of resources, implicit dependencies, and explicit dependencies can be used to achieve the goals of CIOS users.

[0148] CIOS (or a declarative infrastructure provider such as the declarative provisioning tool discussed above) can be used to parse configuration files including code snippet 1100. Through this parsing, CIOS (or the declarative provisioning provider) can generate a directed acyclic graph (DAG) for each resource, module, and / or capability, compiling and defining an ordered list of dependencies on other resources, modules, and / or capabilities. When attempting to deploy a resource, CIOS can traverse the DAG to identify when a resource depends on another resource, module, and / or capability of another resource. Each resource's DAG can specify implicit dependencies, explicit dependencies, or a combination thereof, and can be used to bootstrap or otherwise deploy the corresponding resource using CIOS.

[0149] Figure 12 This is an example directed acyclic graph (DAG) 1200 corresponding to a resource (e.g., resource A) in a cloud computing system, according to at least one embodiment. As shown, DAG 1200 can be a finite directed graph that includes any suitable number of nodes (e.g., such as...). Figure 12 The six nodes shown) and edges (e.g., such as Figure 12 The seven sides shown), where... Figure 12 Each edge shown points from one node to another. Nodes and edges can be arranged to avoid directed cycles. That is, DAG 1200 is arranged such that there is no path that starts from any node and follows a consistent sequence of edges that eventually returns to the same node. The last node (e.g., node "6") can point to a null value or otherwise indicate the end of the DAG.

[0150] Although DAG 1200 depicts six nodes and seven edges, a DAG can include any suitable number of nodes and directed edges. In some embodiments, each node corresponds to a set of operations (e.g., operations for performing tasks such as deploying and / or bootstrapping resources (such as resource A)) or a set of capabilities that the next node of an operation depends on. The directed edges of each DAG define the order in which these operations are performed and / or the dependencies between a subset of operations associated with a node and a subset of capabilities associated with the node immediately preceding that node.

[0151] As a simple example, nodes 1, 2, 5, and 6 of DAG 1200 are intended to depict nodes corresponding to four separate sets of operations. Based on the edges depicted in Figure 1200, the operations of each node will be executed in the order corresponding to the order of nodes 1, 2, 5, and 6. Nodes 3 and 4 are intended to depict nodes that individually correspond to one or more dependencies. For example, node 3 may correspond to the dependency of the operation pair corresponding to node 5 on the capability associated with a different resource (e.g., resource B). Similarly, node 4 may correspond to the dependency of the operation pair corresponding to node 5 on the capability associated with a different resource (e.g., resource C). In some embodiments, different capability nodes (e.g., nodes that identify the capability / multiple capability dependencies for a specific resource) may be used for different resources, or a single node may be used to specify all dependencies, regardless of how many resources the dependency references. Therefore, in some embodiments, the dependency corresponding to resource B (e.g., identified in node 3) and the dependency corresponding to resource C (e.g., identified in node 4) may be combined in a single node.

[0152] DAG 1200 can be about Figures 10-12 A more detailed description of the traversal is used to orchestrate the execution of operations for bootstrapping and / or deploying resources in a cloud computing environment, based on one or more dependencies on the capabilities of other resources (or other resources themselves).

[0153] Figure 13 This is a flowchart illustrating an example process 1300 for orchestrating the execution of a task (e.g., deploying resources) that includes dependencies on at least one capability (e.g., capabilities of different resources) according to at least one embodiment. Figure 13 As shown, process flow 1300 includes scheduler 1302 (e.g., Figure 2 Scheduler 206), worker 1304 (e.g.) Figure 2 Workers 210) and IP process 1306 (e.g. Figure 2 CIOS container 212).

[0154] At 1308, scheduler 1302 can receive tasks for deploying infrastructure resources in a region, and scheduler 1302 can send task-related data to worker 1304. In some embodiments, scheduler 1302 can instantiate worker 1304 to handle the deployment of resources (e.g., services).

[0155] At 1310, worker 1304 can instantiate IP process 1306, which can be configured to execute an instance of a declarative infrastructure provider (e.g., the declarative provisioning tool Terraform discussed above).

[0156] At 1312, IP procedure 1306 can resolve configuration files associated with the deployment (e.g., including...). Figure 10 and Figure 11 The configuration files (code segments 1000 and / or 1100) generate a directed acyclic graph (DAG) for a specific resource. By resolving the configuration, IP process 1306 (the declarative infrastructure provider) can identify any appropriate number of implicit and / or explicit dependencies on capabilities of other resources. Once identified, IP process 1306 constructs a DAG specifying tasks for bootstrapping and / or deploying resources that may have one or more nodes (e.g., based on the implicit and / or explicit dependencies identified during resolution) corresponding to capabilities the resource depends on.

[0157] At 1314, IP process 1306 may begin traversing the DAG, performing at least a portion of the deployment and / or bootstrapping of a specific resource as various nodes of the DAG are reached. Depending on at least one node of the DAG, any suitable operation may be performed to make a portion of the functionality corresponding to the resource available. It is possible that multiple portions of the functionality corresponding to the resource become available. In some embodiments, IP process 1306 may send a notification (not depicted) to scheduler 1302 indicating that one or more capabilities of the resource are now available. At least one node of the DAG may correspond to the capabilities of one or more other resources. When these types of nodes are reached, IP process 1306 may check to see if the capability is available. If so, IP process 1306 may continue its traversal of the DAG.

[0158] At 1316, IP process 1306 may reach a node in a DAG corresponding to one or more capabilities of one or more other resources. In some embodiments, IP process 1306 may determine that at least one capability associated with a node is not yet available.

[0159] At 1320, in response to determining that at least one capability associated with a node is unavailable, IP process 1306 may send data to scheduler 1302 indicating one or more capabilities that the resource corresponding to IP process 1306 depends on and that have been determined to be unavailable.

[0160] At 1322, IP procedure 1306 may exit after potentially storing state information indicating which operations and / or nodes of the DAG have been completed and / or which specific node of the DAG IP procedure 1306 last visited. IP procedure 1306 may exit, be terminated, suspended, or otherwise cease execution.

[0161] At 1324, scheduler 1302 may store information indicating that a particular resource is waiting for resource recovery boot and / or one or more specific capabilities required for deployment purposes.

[0162] At 1326, scheduler 1302 may receive one or more notifications that one or more capabilities that a resource is waiting for have become available. In some embodiments, when various capabilities corresponding to a resource become available, scheduler 1302 may receive various notifications indicating those capabilities from other IP procedures. Scheduler 1302 may maintain one or more records of various available capabilities and / or various capabilities that a resource is currently waiting for. Scheduler 1302 may identify from one or more records that a specific capability / multiple capabilities corresponding to the resource in IP procedure 1306 is waiting for have become available. Therefore, scheduler 1302 may proceed to 1328.

[0163] At 1328, in response to determining that the capability corresponding to the resource of IP process 1306 has become available, scheduler 1302 may return to step 1308, in which it sends data related to the original task (e.g., deploying the resource) to worker 1304. In some embodiments, scheduler 1302 may instantiate a new worker or utilize a previous worker 1304 (as depicted) to continue processing the task associated with the resource. Worker 1304 may instantiate an IP process (not depicted), which may be configured to perform a parsing profile to generate a DAG for the resource. The IP process may access stored state information to identify the last visited node in the DAG (e.g., a node corresponding to one or more capabilities the resource is waiting for). Since one or more capabilities are now available, the IP process may continue its traversal of the DAG in a similar manner to that discussed above, performing operations at each node to execute a part of the task or check the capabilities upon which the next part of the task depends, until the operations at the end node of the DAG have been completed.

[0164] A process similar to that discussed above can be executed for each resource of a task. For example, when deploying a system with multiple resources (e.g., multiple services), process 1300 can be executed on behalf of each resource to deploy each resource in the system.

[0165] Figure 14 This is an example process flow 1400 for executing a release according to at least one embodiment (e.g., via CIOS). At event number 1, scheduler 1402 (e.g. Figure 2 The scheduler 206 can direct workers 1404 (e.g., Figure 2Workers 210 send tasks. Tasks may include deploying computing systems or subsets thereof, such as deploying infrastructure resources to a set of execution targets. Tasks may involve traversing a linked list (e.g., Figure 7 (Example of linked list 700), DAG (e.g., Figure 9 and Figure 12 The tasks can be assigned to worker 1404 (either individual DAGs 900 and 1200, combinations thereof), or any other suitable task for deploying the computing system. Worker 1404 can receive tasks from scheduler 206. Worker 1404 can be a worker node in a worker node cluster. The worker node cluster can include any suitable number of worker nodes for deploying the computing system. Worker 1404 can be selected by scheduler 1402 based at least in part on the capabilities of worker 1404. For example, if worker 1404 has the largest amount of computing power in the worker node cluster, scheduler 1402 may choose to send tasks to worker 1404.

[0166] At event number 2, worker 1404 may execute one or more parsing / traversals of configuration file 1406. Configuration file 1406 may include instructions for deploying a computing system, and executing one or more parsings may result in the identification of resources or other capabilities expected to be guided or otherwise deployed for deploying the computing system. As a non-limiting example, configuration file 1406 may include Figure 6 , Figure 8 , Figure 10 and Figure 11 The respective code segments are 600, 800, 1000, and 1100.

[0167] At event number 3, information from configuration file 1406 can be sent to IP procedure 1408 (e.g., Figure 2 IP process 1408 may receive information from configuration file 1406 based on one or more parsing / traversal operations performed by worker 1404. The information may include a set of capabilities, execution objectives, or any other suitable resources for deploying the computing system.

[0168] At event number 4, in response to receiving information from configuration file 1406, IP process 1408 can determine the order in which to deploy the capabilities or any other suitable resources used to deploy the computing system. IP process 1408 can generate a linked list 1410 of stages (e.g., Figure 7 Example of linked list 700), DAG 1412 of the execution target (e.g.) Figure 9 Example of DAG 900), capability of DAG 1414 (e.g.) Figure 12(Example of DAG 1200), or any other suitable list, graph, or data structure. The linked list 1410 of stages, the DAG 1412 of execution objectives, and the DAG 1414 of capabilities (collectively referred to as the “publishing data structure”) can be generated in any suitable order.

[0169] The release data structure can be used to identify and determine the order in which tasks are executed in the release. For example, each node of the linked list 1410 of stages corresponds to a separate instance of the DAG of the execution target (e.g., an example of DAG 1412 of the execution target), where each node of the DAG of the execution target corresponds to the DAG of the capability (e.g., an example of DAG 1414 of the capability). The IP process 1408 can start from the first node of the linked list 1400 to identify the corresponding DAG of the execution target. The first node of the DAG of the execution target can be used to identify the corresponding DAG of the capability. Tasks associated with the DAG of the capability can be executed according to the DAG of the capability, and once completed, the IP process 1408 can traverse to the next node of the DAG of the execution target to identify the next corresponding DAG of the capability. Each node of the DAG of the execution target can be traversed, and when the tasks corresponding to these nodes are completed, the IP process 1408 can then traverse to the next node of the linked list 1400 to identify the next stage. This process can be repeated any suitable number of times until all tasks associated with each execution target associated with the final stage of the release have been completed.

[0170] As an example, at event number 5, the first node of the linked list 1410 of the stage is reached. IP process 1408 identifies the DAG corresponding to the execution target of the first node.

[0171] At event number 6, the first node of DAG 1412 that achieves the execution target is reached. The capability of DAG 1414 can be identified, at least in part, based on its association with the first node of DAG 1412 that achieves the execution target.

[0172] At event number 7, tasks for a given execution target are executed, at least in part, based on traversing the DAG 1414 of the capability. Once those tasks are completed, IP process 1408 can traverse to the next node of the execution target's DAG, determine the corresponding capability's DAG, and execute tasks based on traversing that capability's DAG. This process can continue until the tasks associated with the last node of the execution target's DAG 1412 have been executed. IP process 1408 can then traverse to the next node of the stage's linked list 1410. The operations for events 5-7 can be repeated any suitable number of times until all tasks associated with all execution targets associated with the last node of the stage's linked list 1410 have been executed. Once tasks, execution targets, and / or stages are completed, IP process 1408 can update... Figure 5 UI 500 or causing concern Figure 5 The UI 500 has been updated to depict the current state of tasks, execution goals, and / or phases.

[0173] At event number 8, IP procedure 1408 sends a signal to scheduler 1402 indicating that the published traversal is complete. Scheduler 1402 can receive the signal from IP procedure 1408 and can broadcast a notification that the computing system is ready for use.

[0174] Figure 15 This is a flowchart of process 1500 for deploying a computing system using a DAG in a CIOS, according to at least one embodiment. At block 1502, the CIOS may execute instructions for performing one or more parsing of configuration data associated with the deployment of the computing system. The configuration data may be included in a configuration file, and when performing one or more parsings of the configuration data, the CIOS may determine a set of tasks to be performed to deploy the computing system. The tasks included in the set of tasks may include deploying infrastructure resources at the execution target or any other suitable tasks for deploying the computing system.

[0175] At box 1504, CIOS makes the first DAG (e.g.) Figure 12 A first DAG (1200) is generated for deploying resources based on a configuration file. The first DAG can be a capability DAG, which can outline a specific order of capabilities to be bootstrapped or otherwise deployed. The first DAG can include any suitable number of tasks for deploying capabilities and can include any suitable number of dependencies between the capabilities to be deployed.

[0176] At box 1506, CIOS generates a second DAG (e.g., Figure 9 The second DAG (900) defines the dependencies between execution targets for deployment based on a configuration file. The second DAG can be a DAG of execution targets specifying the order in which they are deployed. The second DAG can include any suitable number of execution targets for deploying the computing system, and can include any suitable number of dependencies for deploying the execution targets.

[0177] At box 1508, CIOS generates a linked list based on the dependencies between stages specified in the configuration file (e.g., Figure 7 (A linked list 700). A stage may include a first DAG, a second DAG, or a combination thereof, and the linked list may define the order in which the stages are executed. Stages in the linked list may be executed sequentially, or may not include any stages to be executed in parallel. The linked list may include any suitable number of stages for deploying the computing system.

[0178] At box 1510, CIOS deploys the computing system by traversing the first DAG, the second DAG, and the linked list. CIOS can traverse the linked list, and the first and second DAGs can be traversed simultaneously (i.e., the first and second DAGs can be included in the linked list). Successful traversal of the linked list, the first DAG, and the second DAG leads to the successful deployment of the computing system.

[0179] Explanatory System

[0180] Figures 16-18 The illustrations depict aspects of an example environment for implementing aspects of this disclosure, according to various embodiments. Figure 16 A simplified diagram of a distributed system 1600 for implementing embodiments of the present disclosure is depicted. In the illustrated embodiment, the distributed system 1600 includes one or more client computing devices 1602, 1604, 1606, and 1608, which are configured to execute and operate client applications, such as web browsers, proprietary clients (e.g., Oracle Forms), etc., via one or more networks 1610. A server 1612 may be communicatively coupled to remote client computing devices 1602, 1604, 1606, and 1608 via network 1610.

[0181] In various embodiments, server 1612 may be adapted to run one or more services or software applications, such as services and applications providing identity management services. In some embodiments, server 1612 may also provide other services or software applications, which may include non-virtual and virtual environments. In some embodiments, these services may be provided as web-based services or cloud services or under a Software as a Service (SaaS) model to users of client computing devices 1602, 1604, 1606, and / or 1608. Users operating client computing devices 1602, 1604, 1606, and / or 1608 may then interact with server 1612 using one or more client applications to utilize the services provided by these components.

[0182] exist Figure 16 In the configuration depicted, software components 1618, 1620, and 1622 of system 1600 are shown as being implemented on server 1612. In other embodiments, one or more components of system 1600 and / or the services provided by these components may also be implemented by one or more of client computing devices 1602, 1604, 1606, and / or 1608. A user operating the client computing device can then utilize one or more client applications to use the services provided by these components. These components may be implemented in hardware, firmware, software, or a combination thereof. It should be understood that various different system configurations, different from those of the distributed system 1600, are possible. Therefore, Figure 16 The embodiments shown are an example of a distributed system for implementing the system of the embodiments and are not intended to be limiting.

[0183] Client computing devices 1602, 1604, 1606, and / or 1608 may include various types of computing systems. For example, client computing devices may include systems running programs such as Microsoft Windows. Software like this, and / or portable handheld devices with multiple mobile operating systems such as iOS, Windows Phone, Android, BlackBerry 10, Palm OS, etc. (e.g., Cellular phone Computing tablets, personal digital assistants (PDAs), or wearable devices (e.g., Google) (Head-mounted display). The device can support a variety of applications, such as various internet-related applications, email, short message service (SMS) applications, and can use various other communication protocols. Client computing devices can also include general-purpose personal computers, including, for example, those running various versions of Microsoft... Apple Personal computers and / or laptops running Linux operating systems. Client computing devices can be running various commercially viable operating systems. Workstation computers running any of the following operating systems: or UNIX-like operating systems (including, but not limited to, various GNU / Linux operating systems, such as, for example, Google Chrome OS). Client computing devices may also include electronic devices capable of communicating via one or more networks, such as thin client computers, internet-enabled gaming systems (e.g., with or without...). A gesture input device (Microsoft Xbox game console) and / or a personal messaging device.

[0184] although Figure 16 The distributed system 1600 is shown as having four client computing devices, but any number of client computing devices can be supported. Other devices (such as devices with sensors) can interact with the server 1612.

[0185] The network(s) 1610 in the distributed system 1600 can be any type of network(s) familiar to those skilled in the art that can support data communication using any of a variety of available protocols, including but not limited to TCP / IP (Transmission Control Protocol / Internet Protocol), SNA (System Network Architecture), IPX (Internet Packet Switching), AppleTalk, etc. For example only, the network(s) 1610 can be a local area network (LAN), an Ethernet-based network, a token ring-based network, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a public switched telephone network (PSTN), an infrared network, or a wireless network (e.g., in the IEEE 1002.16 protocol suite). (and / or any other wireless protocol under which a network operates), and / or any combination of these and / or other networks.

[0186] Server 1612 may be composed of one or more general-purpose computers, special-purpose server computers (for example, including PC (personal computer) servers, Server 1612 may consist of a server, a mid-range server, a mainframe computer, a rack server, etc., a server farm, a server cluster, or any other suitable arrangement and / or combination. Server 1612 may include one or more virtual machines running a virtual operating system or other computing architectures involving virtualization. One or more flexible logical storage device pools may be virtualized to maintain the server's virtual storage devices. The virtual network may be controlled by server 1612 using software-defined networking. In various embodiments, server 1612 may be adapted to run one or more services or software applications described in the foregoing disclosure. For example, server 1612 may correspond to a server according to embodiments of this disclosure for performing the processes described above.

[0187] Server 1612 can run operating systems, including any of those discussed above, as well as any commercially available server operating system. Server 1612 can also run any of a variety of additional server applications and / or middleware applications, including HTTP (Hypertext Transfer Protocol) servers, FTP (File Transfer Protocol) servers, CGI (Common Gateway Interface) servers, etc. Servers, database servers, etc. Exemplary database servers include, but are not limited to, commercially available database servers from Oracle, Microsoft, Sybase, IBM, etc.

[0188] In some implementations, server 1612 may include one or more applications to analyze and integrate data feeds and / or event updates received from users of client computing devices 1602, 1604, 1606, and 1608. As an example, data feeds and / or event updates may include, but are not limited to, those provided in the original text. feed, The server 1612 may receive real-time updates from one or more third-party information sources and continuous data streams, which may include real-time events related to sensor data applications, financial quote machines, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, vehicle traffic monitoring, and the like. The server 1612 may also include one or more applications to display data feeds and / or real-time events via one or more display devices of client computing devices 1602, 1604, 1606, and 1608.

[0189] Distributed system 1600 may also include one or more databases 1614 and 1616. These databases may provide a mechanism for storing information such as user identity information and other information used in embodiments of this disclosure. Databases 1614 and 1616 may reside in a variety of locations. For example, one or more of databases 1614 and 1616 may reside on a non-transitory storage medium local to server 1612 (and / or within server 1612). Alternatively, databases 1614 and 1616 may be located remotely from server 1612 and communicate with server 1612 via a network-based connection or a dedicated connection. In one set of embodiments, databases 1614 and 1616 may reside in a storage area network (SAN). Similarly, any necessary files for performing functions belonging to server 1612 may be stored locally on server 1612 and / or remotely, as appropriate. In one set of embodiments, databases 1614 and 1616 may include relational databases, such as those provided by Oracle, adapted to store, update, and retrieve data in response to commands in SQL format.

[0190] Figure 17 An example computer system 1700 that can be used to implement embodiments of the present disclosure is illustrated. In some embodiments, computer system 1700 can be used to implement any of the various servers and computer systems described above. Figure 17 As shown, the computer system 1700 includes various subsystems, including a processing subsystem 1704 that communicates with multiple peripheral subsystems via a bus subsystem 1702. These peripheral subsystems may include a processing acceleration unit 1706, an I / O subsystem 1708, a storage subsystem 1718, and a communication subsystem 1724. The storage subsystem 1718 may include a tangible computer-readable storage medium 1722 and system memory 1710.

[0191] Bus subsystem 1702 provides a mechanism for enabling various components and subsystems of computer system 1700 to communicate with each other as intended. Although bus subsystem 1702 is schematically shown as a single bus, alternative embodiments of the bus subsystem may utilize multiple buses. Bus subsystem 1702 may be any of several types of bus structures using any of a variety of bus architectures, including memory buses or memory controllers, peripheral buses, and local buses. For example, such architectures may include Industry Standard Architecture (ISA) buses, Micro Channel Architecture (MCA) buses, Enhanced ISA (EISA) buses, Video Electronics Standards Association (VESA) local buses, and Peripheral Component Interconnect (PCI) buses that may be implemented as mezzanine buses manufactured according to the IEEE P1386.1 standard.

[0192] The processing subsystem 1704 controls the operation of the computer system 1700 and may include one or more processing units 1732, 1734, etc. A processing unit may include one or more processors, including single-core or multi-core processors, one or more cores of a processor, or a combination thereof. In some embodiments, the processing subsystem 1704 may include one or more dedicated coprocessors, such as a graphics processor, a digital signal processor (DSP), etc. In some embodiments, some or all of the processing units of the processing subsystem 1704 may be implemented using custom circuitry such as an application-specific integrated circuit (ASIC) or a field-programmable gate array (FPGA).

[0193] In some embodiments, the processing unit in processing subsystem 1704 may execute instructions stored in system memory 1710 or computer-readable storage medium 1722. In various embodiments, the processing unit may execute multiple program or code instructions and may maintain multiple concurrently executing programs or processes. At any given time, some or all of the program code to be executed may reside on system memory 1710 and / or computer-readable storage medium 1710, including potentially on one or more storage devices. With appropriate programming, processing subsystem 1704 can provide the various functions described above for dynamically modifying documents (e.g., web pages) in response to usage patterns.

[0194] In some embodiments, the processing acceleration unit 1706 may be provided for performing custom processing or for offloading some processing performed by the processing subsystem 1704 in order to accelerate the overall processing performed by the computer system 1700.

[0195] I / O subsystem 1708 may include devices and mechanisms for inputting information to and / or outputting information from or via computer system 1700. Generally, the term "input device" is used to encompass all possible types of devices and mechanisms for inputting information into computer system 1700. User interface input devices may include, for example, keyboards, pointing devices such as mice or trackballs, touchpads or touchscreens incorporated into a display, scroll wheels, click wheels, dial pads, buttons, switches, keypads, audio input devices with voice command recognition systems, microphones, and other types of input devices. User interface input devices may also include motion sensing and / or gesture recognition devices, such as those enabling users to control the input device (Microsoft...). The 360 ​​Game Controller (a device that provides an interface for receiving input using gestures and verbal commands) and interacting with input devices is a Microsoft product. Motion sensors. User interface input devices may also include eye gesture recognition devices, such as Google's... A blink detector detects eye movements from the user (e.g., blinks when taking a photo and / or making menu selections) and translates these eye gestures into the input device (e.g., Google). Input from a voice recognition system (e.g., voice commands). Additionally, the user interface input device may include features that enable the user to interact with a voice recognition system via voice commands. A voice recognition sensing device that interacts with a navigator.

[0196] Other examples of user interface input devices include, but are not limited to, 3D mice, joysticks or pointing sticks, game controllers, and graphics tablets, as well as audio / visual devices such as speakers, digital cameras, digital camcorders, portable media players, webcams, image scanners, fingerprint scanners, barcode readers, 3D scanners, 3D printers, laser rangefinders, and eye-tracking devices. Additionally, user interface input devices may include, for example, medical imaging input devices such as computed tomography (CT), magnetic resonance imaging (MRI), positron emission tomography (PET), and medical ultrasound equipment. User interface input devices may also include, for example, audio input devices such as MIDI keyboards, digital musical instruments, etc.

[0197] User interface output devices may include display subsystems, indicator lights, or non-visual displays such as audio output devices. Display subsystems may be cathode ray tubes (CRTs), flat panel devices such as those using liquid crystal displays (LCDs) or plasma displays, projection devices, touchscreens, etc. Generally, the term "output device" is used to encompass all possible types of devices and mechanisms for outputting information from computer system 1700 to the user or other computers. For example, user interface output devices may include, but are not limited to, various display devices that visually convey text, graphics, and audio / video information, such as monitors, printers, speakers, headphones, car navigation systems, plotters, voice output devices, and modems.

[0198] Storage subsystem 1718 provides a repository or data repository for storing information used by computer system 1700. Storage subsystem 1718 provides a tangible, non-transitory computer-readable storage medium for storing basic programming and data constructs that provide the functionality of some embodiments. Software (programs, code modules, instructions) that provides the functionality described above, when executed by processing subsystem 1704, can be stored in storage subsystem 1718. This software can be executed by one or more processing units of processing subsystem 1704. Storage subsystem 1718 may also provide a repository for storing data used according to this disclosure.

[0199] The storage subsystem 1718 may include one or more non-transitory memory devices, including volatile and non-volatile memory devices. For example... Figure 17 As shown, the storage subsystem 1718 includes system memory 1710 and computer-readable storage medium 1722. System memory 1710 may include multiple memories, including volatile main random access memory (RAM) for storing instructions and data during program execution and non-volatile read-only memory (ROM) or flash memory for storing fixed instructions. In some implementations, a basic input / output system (BIOS) containing basic routines that facilitate the transfer of information between elements within computer system 1700, such as during startup, may be stored in ROM. RAM may contain data and / or program modules currently operated and executed by processing subsystem 1704. In some implementations, system memory 1710 may include various types of memory, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0200] For example, rather than limiting, such as Figure 17As shown, system memory 1710 can store applications 1712, program data 1714, and operating system 1716, which may include client applications, web browsers, middleware applications, relational database management systems (RDBMS), etc. For example, operating system 1716 may include various versions of Microsoft... Apple and / or Linux operating system, multiple commercially available options Or a UNIX-like operating system (including but not limited to various GNU / Linux operating systems, Google...) OS (and / or mobile operating systems, such as iOS) and / or mobile operating systems Phone OS 10OS and OS operating system.

[0201] Computer-readable storage medium 1722 may store programming and data constructs that provide the functionality of some embodiments. Software (programs, code modules, instructions) that provides the functionality described above when executed by processing subsystem 1704 may be stored in storage subsystem 1718. For example, computer-readable storage medium 1722 may include non-volatile memory such as hard disk drives, disk drives, CD-ROMs, DVDs, etc. Optical disc drives, such as disks or other optical media. Computer-readable storage media 1722 may include, but are not limited to, those... Drives, flash memory cards, Universal Serial Bus (USB) flash drives, Secure Digital (SD) cards, DVDs, digital video tapes, etc. Computer-readable storage media 1722 may also include solid-state drives (SSDs) based on non-volatile memory (such as flash-based SSDs, enterprise flash drives, solid-state ROMs, etc.), SSDs based on volatile memory (such as SSDs based on solid-state RAM, dynamic RAM, static RAM, DRAM), magnetoresistive RAM (MRAM) SSDs, and hybrid SSDs using a combination of DRAM-based and flash-based SSDs. Computer-readable media 1722 can provide storage for computer-readable instructions, data structures, program modules, and other data for computer system 1700.

[0202] In some embodiments, the storage subsystem 1700 may further include a computer-readable storage medium reader 1720 that may be further connected to the computer-readable storage medium 1722. The computer-readable storage medium 1722, together with and optionally combined with the system memory 1710, may collectively represent remote, local, fixed and / or removable storage devices and storage media for storing computer-readable information.

[0203] In some embodiments, computer system 1700 can support the execution of one or more virtual machines. Computer system 1700 can execute programs such as hypervisors to facilitate the configuration and management of the virtual machines. Each virtual machine can be allocated memory resources, computing resources (e.g., processor, cores), I / O resources, and networking resources. Each virtual machine can run its own operating system, which may be the same as or different from the operating systems executed by other virtual machines executed by computer system 1700. Therefore, multiple operating systems can potentially be run concurrently by computer system 1700. Each virtual machine typically runs independently of other virtual machines.

[0204] The communication subsystem 1724 provides an interface to other computer systems and networks. The communication subsystem 1724 serves as an interface for receiving data from and sending data to other systems from computer system 1700. For example, the communication subsystem 1724 enables computer system 1700 to establish a communication channel via the Internet to one or more client devices for receiving and sending information to the client devices. Additionally, the communication subsystem 1724 can be used to transmit notifications of successful login or password re-entry from the privileged account manager to the requesting user.

[0205] The communication subsystem 1724 may support both wired and / or wireless communication protocols. For example, in some embodiments, the communication subsystem 1724 may include radio frequency (RF) transceiver components, global positioning system (GPS) receiver components, and / or other components for accessing wireless voice and / or data networks, such as using cellular telephone technology, advanced data network technologies like 3G, 4G, or EDGE (Global Evolution Enhanced Data Rate), WiFi (IEEE 802.11 series standards, or other mobile communication technologies, or any combination thereof). In some embodiments, in addition to or as an alternative to a wireless interface, the communication subsystem 1724 may provide a wired network connection (e.g., Ethernet).

[0206] The communication subsystem 1724 can receive and transmit data in various forms. For example, in some embodiments, the communication subsystem 1724 can receive input communications in the form of structured and / or unstructured data feeds 1726, event streams 1728, event updates 1730, etc. For example, the communication subsystem 1724 can be configured to receive (or transmit) data feeds 1726 in real time from users of social media networks and / or other communication services, such as... feed, Updates, web feeds such as Rich Site Summary (RSS) feeds, and / or real-time updates from one or more third-party information sources.

[0207] In some embodiments, the communication subsystem 1724 may be configured to receive data in the form of a continuous data stream, which may include an event stream 1728 of real-time events and / or event updates 1730, which may be continuous or unbounded in nature without a definite end. Examples of applications that generate continuous data may include, for example, sensor data applications, financial quote machines, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, vehicle traffic monitoring, etc.

[0208] The communication subsystem 1724 can also be configured to output structured and / or unstructured data feeds 1726, event streams 1728, event updates 1730, etc., to one or more databases, which can communicate with one or more streaming data source computers coupled to the computer system 1700.

[0209] The computer system 1700 can be of a variety of types, including handheld portable devices (e.g., Cellular phone Computing tablets, PDAs), and wearable devices (e.g., Google). Head-mounted displays, personal computers, workstations, mainframes, kiosks, server racks, or any other data processing systems.

[0210] Due to the constantly changing nature of computers and networks, Figure 17 The description of the computer system 1700 depicted in the text is intended only as a concrete example. It has more than Figure 17 Many other configurations of the system depicted, with more or fewer components, are possible. Based on the disclosure and teachings provided herein, those skilled in the art will understand other ways and / or methods for implementing the various embodiments.

[0211] Some of the systems depicted in the diagram can be provided in various configurations. In some embodiments, the system can be configured as a distributed system, wherein one or more components of the system are distributed across one or more networks in one or more cloud infrastructure systems.

[0212] A cloud infrastructure system is a collection of one or more server computing devices, network devices, and / or storage devices. These resources can be partitioned and allocated to their customers by cloud service providers in several ways. For example, cloud service providers (such as Oracle Corporation of Redwood Coast, California) can provide various types of cloud services, including but not limited to one or more services offered under the Software as a Service (SaaS) category, services offered under the Platform as a Service (PaaS) category, services offered under the Infrastructure as a Service (IaaS) category, or other categories of services that include hybrid services. Examples of SaaS services include, but are not limited to, the ability to build and deliver a suite of on-demand applications (such as Oracle Fusion applications). SaaS services enable customers to leverage applications that run on a cloud infrastructure system without requiring customers to purchase software for the applications. Examples of PaaS services include, but are not limited to, services that enable organizations (such as Oracle) to consolidate existing applications on a shared, common architecture, and the ability to build new applications that utilize shared services provided by the platform (such as Oracle Java Cloud Service (JCS), Oracle Database Cloud Service (DBCS), and other services). IaaS services can facilitate customer management and control of underlying computing resources, such as storage, networking, and other basic computing resources, when leveraging services provided by SaaS and PaaS platforms.

[0213] Figure 18 This is a simplified block diagram of one or more components of a system environment 1800 according to an embodiment of the present disclosure, through which services provided by one or more components of the embodiment system can be provided as cloud services. In the illustrated embodiment, system environment 1800 includes one or more client computing devices 1804, 1806, and 1808, which users can use to interact with a cloud infrastructure system 1802 providing cloud services. The client computing devices can be configured to operate client applications, such as web browsers, proprietary client applications (e.g., OracleForms), or other applications, which users of the client computing devices can use to interact with the cloud infrastructure system 1802 to use the services provided by the cloud infrastructure system 1802.

[0214] It should be understood that the cloud infrastructure system 1802 depicted in the figures may have components other than those depicted. Furthermore, the embodiments shown in the figures are merely one example of a cloud infrastructure system that can be incorporated into embodiments of this disclosure. In some other embodiments, the cloud infrastructure system 1802 may have more or fewer components than shown in the figures, may combine two or more components, or may have different component configurations or arrangements.

[0215] Client computing devices 1804, 1806, and 1808 can be devices similar to those described above for 1602, 1604, 1606, and 1608.

[0216] Although the exemplary system environment 1800 is shown to have three client computing devices, any number of client computing devices can be supported. Other devices (such as devices with sensors) can interact with the cloud infrastructure system 1802.

[0217] One or more networks 1810 may facilitate communication and data exchange between clients 1804, 1806, and 1808 and cloud infrastructure system 1802. Each network may be any type of network familiar to those skilled in the art that supports data communication using any of a variety of commercially available protocols, including those described above for one or more networks 1810.

[0218] The cloud infrastructure system 1802 may include one or more computers and / or servers, which may include those described above for server 1812.

[0219] In some embodiments, the services provided by a cloud infrastructure system may include a wide range of services offered on demand to users of the cloud infrastructure system, such as online data storage and backup solutions, web-based email services, hosted office suites and document collaboration services, database processing, managed technical support services, and so on. The services provided by the cloud infrastructure system can be dynamically scaled to meet the needs of its users. A specific instantiation of a service provided by the cloud infrastructure system is referred to herein as a “service instance.” Generally, any service available to users from a cloud service provider’s system via a communication network (such as the Internet) is referred to as a “cloud service.” In a public cloud environment, the servers and systems that make up the cloud service provider’s system differ from the on-premises servers and systems of a customer’s own. For example, the cloud service provider’s system may host applications, and users may subscribe to and use these applications on demand via a communication network such as the Internet.

[0220] In some examples, services within a computer network cloud infrastructure may include protected computer network access to storage devices, hosted databases, hosted web servers, software applications, or other services provided to users by a cloud provider or as otherwise known in the art. For example, services may include password-protected access to remote storage devices in the cloud via the Internet. As another example, services may include hosted relational databases based on web services and scripting language middleware engines for web developers. As yet another example, services may include access to email software applications hosted on a cloud provider's website.

[0221] In some embodiments, cloud infrastructure system 1802 may include a suite of application, middleware, and database service offerings delivered to customers in a self-service, subscription-based, elastically scalable, reliable, highly available, and secure manner. An example of such a cloud infrastructure system is the Oracle public cloud provided by this assignee.

[0222] In various embodiments, cloud infrastructure system 1802 can be adapted to automatically provision, manage, and track customer subscriptions to services provided by cloud infrastructure system 1802. Cloud infrastructure system 1802 can provide cloud services via different deployment models. For example, services can be provided under a public cloud model, in which cloud infrastructure system 1802 is owned by an organization selling cloud services (e.g., owned by Oracle) and the services are available to the general public or businesses in different industries. As another example, services can be provided under a private cloud model, in which cloud infrastructure system 1802 is operated for only a single organization and can provide services to one or more entities within that organization. Cloud services can also be provided under a community cloud model, in which cloud infrastructure system 1802 and the services provided by cloud infrastructure system 1802 are shared by several organizations in the relevant community. Cloud services can also be provided under a hybrid cloud model, which is a combination of two or more different models.

[0223] In some embodiments, the services provided by the cloud infrastructure system 1802 may include one or more services offered under the Software as a Service (SaaS) category, the Platform as a Service (PaaS) category, the Infrastructure as a Service (IaaS) category, or other categories of services that include hybrid services. A customer may subscribe to one or more services provided by the cloud infrastructure system 1802 via a subscription order. The cloud infrastructure system 1802 then performs processing to provide the services in the customer's subscription order.

[0224] In some embodiments, the services provided by the cloud infrastructure system 1802 may include, but are not limited to, application services, platform services, and infrastructure services. In some examples, application services may be provided by the cloud infrastructure system via a SaaS platform. The SaaS platform may be configured to provide cloud services belonging to the SaaS category. For example, a SaaS platform may provide the ability to build and deliver a suite of on-demand applications on an integrated development and deployment platform. The SaaS platform can manage and control the underlying software and infrastructure used to provide the SaaS services. By utilizing the services provided by the SaaS platform, customers can leverage applications running on the cloud infrastructure system. Customers can obtain application services without needing to purchase separate licenses and support. A variety of different SaaS services may be provided. Examples include, but are not limited to, services providing solutions for sales performance management, enterprise integration, and business agility for large organizations.

[0225] In some embodiments, platform services may be provided by a cloud infrastructure system via a PaaS platform. The PaaS platform can be configured to provide cloud services that fall under the PaaS category. Examples of platform services may include, but are not limited to, services that enable organizations (such as Oracle) to integrate existing applications on a shared, public architecture, and the ability to build new applications that leverage the shared services provided by the platform. The PaaS platform can manage and control the underlying software and infrastructure used to provide PaaS services. Customers can obtain PaaS services provided by the cloud infrastructure system without having to purchase separate licenses and support. Examples of platform services include, but are not limited to, Oracle Java Cloud Service (JCS), Oracle Database Cloud Service (DBCS), and others.

[0226] By leveraging the services provided by a PaaS platform, customers can employ programming languages ​​and tools supported by the cloud infrastructure system and also control the deployed services. In some embodiments, the platform services provided by the cloud infrastructure system may include database cloud services, middleware cloud services (e.g., Oracle Fusion Middleware Service), and Java cloud services. In one embodiment, the database cloud service may support a shared services deployment model, which enables organizations to aggregate database resources and offer Database as a Service to customers in the form of a database cloud. Within the cloud infrastructure system, the middleware cloud service can provide customers with a platform to develop and deploy various business applications, and the Java cloud service can provide customers with a platform to deploy Java applications.

[0227] Various infrastructure services can be provided by IaaS platforms within cloud infrastructure systems. Infrastructure services facilitate the management and control of underlying computing resources (such as storage, networking, and other basic computing resources) for customers utilizing services provided by SaaS and PaaS platforms.

[0228] In some embodiments, the cloud infrastructure system 1802 may also include infrastructure resources 1830 for providing resources for offering various services to customers of the cloud infrastructure system. In one embodiment, infrastructure resources 1830 may include a pre-integrated and optimized combination of hardware (such as servers, storage devices, and networking resources) to perform services provided by PaaS platforms and SaaS platforms.

[0229] In some embodiments, resources in cloud infrastructure system 1802 can be shared by multiple users and dynamically reallocated on demand. Additionally, resources can be allocated to users in different time zones. For example, cloud infrastructure system 1830 can enable a first group of users in a first time zone to utilize the resources of the cloud infrastructure system for a specified number of hours, and then enable the same resources to be reallocated to another group of users located in a different time zone, thereby maximizing resource utilization.

[0230] In some embodiments, multiple internal shared services 1832 may be provided, shared by different components or modules of the cloud infrastructure system 1802 and by services provided by the cloud infrastructure system 1802. These internal shared services may include, but are not limited to, security and identity services, integration services, enterprise repository services, enterprise manager services, virus scanning and whitelisting services, high availability, backup and recovery services, cloud-enabled services, email services, notification services, file transfer services, etc.

[0231] In some embodiments, cloud infrastructure system 1802 can provide comprehensive management of cloud services (e.g., SaaS, PaaS, and IaaS services) within the cloud infrastructure system. In one embodiment, cloud management functionality may include the ability to provision, manage, and track customer subscriptions received by cloud infrastructure system 1802.

[0232] In one embodiment, as depicted in the figure, cloud management functionality may be provided by one or more modules, such as order management module 1820, order orchestration module 1822, order supply module 1824, order management and monitoring module 1826, and identity management module 1828. These modules may include or be provided using one or more computers and / or servers, which may be general-purpose computers, dedicated server computers, server farms, server clusters, or any other suitable arrangement and / or combination.

[0233] In exemplary operation 1834, a customer using a client device such as client device 1804, 1806, or 1808 can interact with cloud infrastructure system 1802 by requesting one or more services provided by cloud infrastructure system 1802 and placing an order to subscribe to one or more services provided by cloud infrastructure system 1802. In some embodiments, the customer can access cloud user interfaces (UIs), cloud UI 1812, cloud UI 1814, and / or cloud UI 1816, and place subscription orders via these UIs. Order information received by cloud infrastructure system 1802 in response to a customer placing an order may include information identifying the customer and the one or more services provided by cloud infrastructure system 1802 that the customer intends to subscribe to.

[0234] After a customer has placed an order, the order information is received via cloud UI 1812, 1814 and / or 1816.

[0235] At operation 1836, the order is stored in order database 1818. Order database 1818 can be one of several databases operated by cloud infrastructure system 1818 in conjunction with other system elements.

[0236] At operation 1838, the order information is forwarded to the order management module 1820. In some cases, the order management module 1820 can be configured to perform order-related billing and accounting functions, such as verifying the order and, after verification, reserving the order.

[0237] At operation 1840, order information is transmitted to order orchestration module 1822. Order orchestration module 1822 can use the order information to orchestrate the supply of services and resources for orders placed with customers. In some cases, order orchestration module 1822 can orchestrate the supply of resources to support subscribed services using the services of order supply module 1824.

[0238] In some embodiments, the order orchestration module 1822 enables the management of business processes associated with each order and applies business logic to determine whether an order should be provisioned. At operation 1842, upon receiving a new subscription order, the order orchestration module 1822 sends a request to the order provisioning module 1824 to allocate resources and configure those resources required to satisfy the subscription order. The order provisioning module 1824 enables the allocation of resources for the services ordered by the customer. The order provisioning module 1824 provides a level of abstraction between the cloud services provided by the cloud infrastructure system 1800 and the physical implementation layer used to provision resources to provide the requested services. The order orchestration module 1822 can therefore be isolated from implementation details such as whether services and resources are actually provisioned on demand or pre-provisioned and allocated / assigned only upon request.

[0239] At operation 1844, once services and resources are provisioned, notifications of the provided services can be sent by the order provisioning module 1824 of the cloud infrastructure system 1802 to customers on client devices 1804, 1806, and / or 1808. At operation 1846, customer subscription orders can be managed and tracked by the order management and monitoring module 1826. In some cases, the order management and monitoring module 1826 can be configured to collect service usage statistics from subscription orders, such as the amount of storage used, the amount of data transferred, the number of users, and the amount of system uptime and downtime.

[0240] In some embodiments, the cloud infrastructure system 1800 may include an identity management module 1828. The identity management module 1828 may be configured to provide identity services, such as access management and authorization services within the cloud infrastructure system 1800. In some embodiments, the identity management module 1828 may control information about customers who wish to utilize the services provided by the cloud infrastructure system 1802. Such information may include information authenticating the identities of such customers and information describing which actions those customers are authorized to perform relative to various system resources (e.g., files, directories, applications, communication ports, storage segments, etc.). The identity management module 1828 may also include the management of descriptive information about each customer and about how and by whom that descriptive information can be accessed and modified.

[0241] Although specific embodiments of this disclosure have been described, various modifications, alterations, alternative constructions, and equivalents are also included within the scope of this disclosure. The embodiments of this disclosure are not limited to operation within certain specific data processing environments, but can be freely operated within multiple data processing environments. Additionally, although embodiments of this disclosure have been described using a specific series of transactions and steps, those skilled in the art will understand that the scope of this disclosure is not limited to the described series of transactions and steps. Various features and aspects of the above embodiments can be used individually or in combination.

[0242] Furthermore, while embodiments of this disclosure have been described using specific combinations of hardware and software, it should be recognized that other combinations of hardware and software are also within the scope of this disclosure. Embodiments of this disclosure may be implemented solely in hardware, solely in software, or using a combination thereof. The various processes described herein may be implemented on the same or different processors in any combination. Thus, where a component or module is described as being configured to perform certain operations, such a configuration may be implemented, for example, by designing electronic circuitry to perform operations, by programming programmable electronic circuitry (such as a microprocessor) to perform operations, or any combination thereof. Processes may communicate using a variety of technologies, including but not limited to conventional technologies for inter-process communication, and different pairs of processes may use different technologies, or the same pair of processes may use different technologies at different times.

[0243] Therefore, the specification and drawings are to be considered illustrative rather than restrictive. However, it will be apparent that additions, omissions, deletions, and other modifications and alterations may be made thereto without departing from the broader spirit and scope set forth in the claims. Thus, although specific disclosed embodiments have been described, they are not intended to be limiting. Various modifications and equivalents are within the scope of the following claims.

Claims

1. A computer-implemented method, comprising: Perform a cloud infrastructure orchestration service, which is configured to provision infrastructure resources and deploy services to the infrastructure resources, at least in part, based on a set of configuration files, the set of configuration files describing the infrastructure resources to be provisioned and the services to be deployed via declarative directives; The cloud infrastructure orchestration service executes instructions to perform one or more parsings of one or more configuration files associated with deploying multiple services to multiple execution targets; The cloud infrastructure orchestration service enables the generation of a first directed acyclic graph (DAG), which is used at least in part to deploy the plurality of services to an execution target based on the execution of the one or more resolutions. The first DAG indicates the dependencies between the tasks associated with deploying the plurality of services to the execution target. The cloud infrastructure orchestration service generates a second DAG for deploying the plurality of services to the plurality of execution targets, at least in part based on the execution of the one or more resolutions, the plurality of execution targets including the execution targets, the second DAG specifying dependencies associated with different deployments of the plurality of services to each of the plurality of execution targets; The cloud infrastructure orchestration service generates a linked list data structure, at least in part, based on the execution of the one or more parsing processes. The linked list data structure specifies dependencies between multiple deployment phases, each phase being associated with an execution objective of a different group to which the multiple services are to be deployed. as well as The cloud infrastructure orchestration service deploys the plurality of services to the plurality of execution targets based at least in part on traversing the linked list data structure, the second DAG, and the first DAG.

2. The computer-implemented method of claim 1, wherein the first DAG specifies the dependency relationship between the capabilities of the first resource and the second resource, and wherein each of the first resource and the second resource is a corresponding service among the plurality of services, and wherein the capability is part of the functionality of the second resource.

3. The computer-implemented method as described in any one of claims 1 or 2, wherein at least one node of the second DAG references a node of the first DAG.

4. The computer-implemented method as described in any one of claims 1 or 2, wherein a node of the linked list data structure references at least one node of the second DAG.

5. The computer-implemented method of any one of claims 1 or 2, wherein executing the instructions for executing the one or more parsing of the one or more configuration files comprises: The first dependency is detected via explicit statements provided in one or more configuration files; or The second dependency is detected at least in part based on recognizing implicit dependencies provided in the one or more configuration files.

6. The computer-implemented method of any one of claims 1 or 2, wherein the one or more configuration files indicate, via one or more dependencies, the order in which infrastructure deployment operations for deploying multiple resources are performed.

7. The computer-implemented method of claim 1 or 2, wherein the dependency is defined using declarative statements.

8. The computer-implemented method of claim 1, wherein traversing the linked list data structure, the second DAG, and the first DAG further comprises: Traverse to the first node of the linked list data structure; Identify the second DAG from the first node of the linked list data structure; Traverse to the first node of the second DAG; Identify the first DAG from the first node of the second DAG; Traverse to the first node of the first DAG; as well as At least a portion of the first service among the plurality of services is deployed based at least in part on traversing to the first node of the linked list data structure, traversing to the first node of the second DAG, and traversing to the first node of the first DAG.

9. The computer-implemented method of claim 8, further comprising: Based at least in part on traversing the second node of the first DAG, it is identified that a portion of the second service to be deployed depends on the capabilities of the available third service; The ability to identify third-party services is unavailable; as well as The deployment of the second service will be delayed until the capabilities of the third service become available.

10. The computer-implemented method of claim 9, wherein the deployment of said portion of the delayed second service further comprises: Indication of the capabilities upon which the portion of providing the second service to computing components associated with the cloud infrastructure orchestration service depends; as well as The traversal of the first DAG is recovered from the second node of the first DAG, based at least in part on data received by the computing component indicating the availability of the capability.

11. The computer-implemented method of claim 10, further comprising: Perform the operation corresponding to the end node of the first DAG; Traverse to the second node of the second DAG, where the first node of the second DAG corresponds to the first deployment of the plurality of services, and the second node of the second DAG corresponds to the second deployment of the plurality of services; The first DAG is identified as associated with the second node; as well as The multiple services are deployed as part of a second deployment, at least in part, based on traversing the first DAG.

12. The computer-implemented method of claim 11, further comprising: Perform the operation corresponding to the end node of the second DAG; Traversing to the second node of the linked list data structure, the first node of the linked list data structure corresponds to the first set of execution targets to which the multiple services are to be deployed, and the second node of the linked list data structure corresponds to the second set of execution targets to which the multiple services are to be deployed; The second DAG is identified as associated with the second node; as well as The multiple services are deployed to each execution target in the second set of execution targets, at least in part, based on traversing the second DAG.

13. A system comprising: One or more processors; as well as One or more memories storing computer-executable instructions that, when executed by the one or more processors, configure the one or more processors to: Perform a cloud infrastructure orchestration service, which is configured to provision infrastructure resources and deploy services to the infrastructure resources, at least in part, based on a set of configuration files, the set of configuration files describing the infrastructure resources to be provisioned and the services to be deployed via declarative directives; The cloud infrastructure orchestration service executes instructions to perform one or more parsings of one or more configuration files associated with deploying multiple services to multiple execution targets; The cloud infrastructure orchestration service enables the generation of a first directed acyclic graph (DAG), which is used at least in part to deploy the plurality of services to an execution target based on the execution of the one or more resolutions. The first DAG indicates the dependencies between the tasks associated with deploying the plurality of services to the execution target. The cloud infrastructure orchestration service generates a second DAG for deploying the plurality of services to the plurality of execution targets, at least in part based on the execution of the one or more resolutions, the plurality of execution targets including the execution targets, the second DAG specifying dependencies associated with different deployments of the plurality of services to each of the plurality of execution targets; The cloud infrastructure orchestration service generates a linked list data structure, at least in part, based on the execution of the one or more parsing processes. The linked list data structure specifies dependencies between multiple deployment phases, each phase being associated with an execution objective of a different group to which the multiple services are to be deployed. as well as The cloud infrastructure orchestration service deploys the plurality of services to the plurality of execution targets based at least in part on traversing the linked list data structure, the second DAG, and the first DAG.

14. The system of claim 13, wherein the first DAG specifies the dependency of the first resource on the capabilities of the second resource, and wherein each of the first resource and the second resource is a corresponding service among the plurality of services, and wherein the capability is part of the functionality of the second resource.

15. The system of claim 13, wherein at least one node of the second DAG references a first node of the first DAG, and wherein the second node of the linked list data structure references at least one node of the second DAG.

16. The system of any one of claims 13-15, wherein the one or more configuration files indicate the order of performing infrastructure deployment operations for deploying multiple resources via one or more dependencies, wherein the one or more dependencies are defined using declarative statements, and wherein performing the one or more instructions for executing the one or more parsed configuration files comprises: The first dependency is detected via explicit statements provided in one or more configuration files; or The second dependency is detected at least in part based on recognizing implicit dependencies provided in the one or more configuration files.

17. A computer-readable storage medium storing computer-executable instructions, which, when executed by one or more processors, cause the one or more processors to perform operations, the operations including: Perform a cloud infrastructure orchestration service, which is configured to provision infrastructure resources and deploy services to the infrastructure resources, at least in part, based on a set of configuration files, the set of configuration files describing the infrastructure resources to be provisioned and the services to be deployed via declarative directives; The cloud infrastructure orchestration service executes instructions to perform one or more parsings of one or more configuration files associated with deploying multiple services to multiple execution targets; The cloud infrastructure orchestration service enables the generation of a first directed acyclic graph (DAG), which is used at least in part to deploy the plurality of services to an execution target based on the execution of the one or more resolutions. The first DAG indicates the dependencies between the tasks associated with deploying the plurality of services to the execution target. The cloud infrastructure orchestration service generates a second DAG for deploying the plurality of services to the plurality of execution targets, at least in part based on the execution of the one or more resolutions, the plurality of execution targets including the execution targets, the second DAG specifying dependencies associated with different deployments of the plurality of services to each of the plurality of execution targets; The cloud infrastructure orchestration service generates a linked list data structure, at least in part, based on the execution of the one or more parsing processes. The linked list data structure specifies dependencies between multiple deployment phases, each phase being associated with an execution objective of a different group to which the multiple services are to be deployed. as well as The cloud infrastructure orchestration service deploys the plurality of services to the plurality of execution targets based at least in part on traversing the linked list data structure, the second DAG, and the first DAG.

18. The computer-readable storage medium of claim 17, wherein the first DAG specifies a dependency of the first resource on the capabilities of the second resource, and wherein each of the first resource and the second resource is a corresponding service among the plurality of services, and wherein the capability is part of the functionality of the second resource.

19. The computer-readable storage medium of any one of claims 17 or 18, wherein at least one node of the second DAG references a first node of the first DAG, and wherein a second node of the linked list data structure references at least one node of the second DAG.

20. The computer-readable storage medium of any one of claims 17 or 18, wherein executing the instructions for performing the one or more parsing of the one or more configuration files comprises: The first dependency is detected via explicit statements provided in one or more configuration files; or The second dependency is detected at least in part based on recognizing implicit dependencies provided in the one or more configuration files.

21. An apparatus comprising components for performing the steps of the method according to any one of claims 1-12.

22. A computer program product comprising computer instructions that, when executed by a processor, implement the steps of the method as described in any one of claims 1-12.

Images