Deployment control of cloud resources

Abstract

Techniques for controlling resource deployment in a cloud partition of a cloud environment are disclosed. A cloud service provider (CSP) operates a cloud environment in which its customers may specify constraints for deployment of their respective partitions (i.e., regions or domains). The partition-specific deployment constraints are rules that constrain changes / updates that may be made to one or more particular partitions. The partition-specific deployment constraints are applicable to at least one partition, but may be applicable to multiple partitions. For example, partition-specific deployment constraints may be applied to one or more regions in the domain. The partition-specific deployment constraints are evaluated when deployed using the latest state of at least one particular partition or a selected subset thereof. The global deployment orchestrator causes the deployment to depend at least in part on whether the deployment satisfies the partition-specific constraint (s) in the target partition.

Description

Technical Field

[0001] Each of the following applications is hereby incorporated by reference: Application No. 18 / 647,772, filed April 26, 2024; Application No. 63 / 462,875, filed April 28, 2023. The applicant hereby withdraws any disclaimer of the scope of the claims in one or more of the parent applications or their examination history, and informs the United States Patent and Trademark Office (USPTO) that the claims in this application may be more extensive than any of the claims in one or more of the parent applications. Background Technology

[0002] Cloud computing environments can be used to provide access to a range of complementary cloud-based components, such as software applications or services, enabling organizations or enterprise customers to operate their applications and services in a highly available managed environment.

[0003] The benefits of organizations moving their application and service needs to the cloud include reduced costs and complexity in designing, building, operating, and maintaining their own on-premises data centers, software application frameworks, or other IT infrastructure.

[0004] Deploying resources in a cloud environment can be complex due to the specific needs and preferences of individual customers. For example, one customer may require uninterrupted uptime on weekends, while another may require uninterrupted uptime on weekdays. Alternatively or additionally, different customers may have different security and / or compliance requirements, which influence when, how, and / or by whom resources are deployed. Generally, a deployment schedule suitable for one customer in a cloud environment may not be suitable for another. However, customers typically have no control over when resources are deployed to their respective cloud environment partitions.

[0005] The methods described in this section are possible methods, but not necessarily methods that have been previously conceived or adopted. Therefore, unless otherwise indicated, no method described in this section should be assumed to qualify as prior art simply because it is included in this section. Attached Figure Description

[0006] In the accompanying drawings, embodiments are illustrated by way of example rather than limitation. It should be noted that references to "a" or "an" embodiment in this disclosure do not necessarily refer to the same embodiment, and they mean at least one. In the drawings:

[0007] Figure 1 The illustration depicts a system for providing a cloud infrastructure environment according to an embodiment.

[0008] Figure 2The illustration further illustrates how a cloud infrastructure environment can be used to provide cloud-based applications or services according to embodiments.

[0009] Figure 3 The illustration shows an example cloud infrastructure architecture according to an embodiment.

[0010] Figure 4 The illustration shows another example of a cloud infrastructure architecture according to an embodiment.

[0011] Figure 5 The illustration shows another example of a cloud infrastructure architecture according to an embodiment.

[0012] Figure 6 The illustration shows another example of a cloud infrastructure architecture according to an embodiment.

[0013] Figure 7 The illustration shows how a system according to an embodiment can provide a dedicated or private tagged cloud environment for use by tenants or customers of a cloud infrastructure environment.

[0014] Figure 8 The illustration further illustrates the use of a private tag cloud domain for tenants or customers in a cloud infrastructure environment, according to an embodiment.

[0015] Figure 9 The illustration further illustrates the use of a private tag cloud domain for tenants or customers in a cloud infrastructure environment, according to an embodiment.

[0016] Figure 10 The illustration depicts a system according to an embodiment for providing access to software products or services in a cloud computing or other computing environment.

[0017] Figure 11 The illustration depicts a system for controlling the deployment of cloud resources according to one or more embodiments.

[0018] Figure 12 The illustration depicts a set of example operations for controlling the deployment of cloud resources according to one or more embodiments.

[0019] Figure 13 The illustration shows an example implementation of a graphical user interface (GUI) for a cloud environment. Detailed Implementation

[0020] In the following description, numerous specific details are set forth for purposes of explanation in order to provide a thorough understanding. One or more embodiments may be practiced without these specific details. Features described in one embodiment may be combined with features described in different embodiments. In some examples, well-known structures and devices are described in the form of block diagrams to avoid unnecessarily obscuring this disclosure.

[0021] 1. General Overview

[0022] 2. Dedicated or private tag cloud environment

[0023] 3. Deploy and orchestrate cloud system

[0024] 4. Control the deployment of cloud resources

[0025] 5. Example Implementation

[0026] 6. Other matters; extension

[0027] 1. General Overview

[0028] A cloud service provider (CSP) operates a cloud environment in which multiple tenants share a set of cloud partitions. One or more embodiments transmit a deployment request to a first cloud partition among the multiple cloud partitions to deploy resources to the first cloud partition. In response to receiving a message indicating approval of the deployment request from the first cloud partition, one or more embodiments deploy resources to the first cloud partition. Approval of the deployment request may depend on an evaluation of the deployment request based on the state of the cloud environment. Tenants of the first cloud partition can manage the resource deployment of their respective partitions by dynamically evaluating deployment requests based on partition-specific state information (e.g., and partition-specific deployment constraints).

[0029] The system can transmit a first deployment request to multiple cloud partitions, including a first cloud partition and a second cloud partition. In this way, the system can request the deployment of the same resource (or the same resource version) to both the first cloud partition and at least the second cloud partition. Based on the status information of the second cloud partition, one or more embodiments receive a message from the second cloud partition indicating that the deployment request is rejected. In response to receiving the rejection, one or more embodiments avoid deploying the resource in the second cloud partition. The rejection of the deployment request may depend on an evaluation of the deployment request based on the status of the cloud environment. Similar to tenants in the first cloud partition, tenants in the second cloud partition can manage resource deployments in their respective partitions by dynamically evaluating deployment requests based on status information (e.g., and partition-specific deployment constraints).

[0030] Instead of static rules governing the deployment of any updates / changes to a tenant's cloud partitions (e.g., specific regions within a domain), one or more embodiments enable adherence to tenant-defined rules for the CSP (referred to herein as partition-specific deployment constraints) before the CSP can perform any resource deployments. Since the entity managing a tenant's cloud partitions differs from the entity deploying updates / changes to those partitions, implementing deployment constraints provides the tenant with a degree of control over those updates / changes. While partition-specific deployment constraints provide the tenant with a degree of control over deployments within their cloud partitions, one or more embodiments can balance this degree of control with CSP-defined rules (referred herein as global deployment constraints). One or more embodiments can determine that a deployment request satisfies one or more global deployments before it is transmitted to the tenant's cloud partition.

[0031] One or more embodiments described in this specification and / or recited in the claims may not be included in this general overview section.

[0032] 2. Dedicated or private tag cloud environment

[0033] One or more embodiments provide features associated with a private or private tag cloud (PLC) environment for use by tenants of a cloud infrastructure environment when accessing software products, services, or other offers associated with that environment.

[0034] Cloud computing or cloud infrastructure environments can be used to provide access to a range of complementary cloud-based components, such as software applications or services, enabling organizations or enterprise customers to operate their applications and services in a highly available managed environment.

[0035] The benefits of organizations moving their application and service needs to a cloud infrastructure environment include reduced costs and complexity in designing, building, operating, and maintaining their own on-premises data centers, software application frameworks, or other IT infrastructure.

[0036] cloud infrastructure environment

[0037] Figure 1 and Figure 2 The illustration depicts a system for providing a cloud infrastructure environment according to an embodiment.

[0038] According to an embodiment, Figure 1 The components and processes shown herein, as well as those further described herein with respect to various embodiments, may be provided as software or program code executable by a computer system or other type of processing device (e.g., a cloud computing system).

[0039] The illustrated examples are provided to illustrate computing environments that can be used to provide dedicated or privately tagged cloud environments for tenants of cloud infrastructure to use when accessing subscription-based software products, services, or other provisioning items associated with the cloud infrastructure environment. According to other embodiments, the various components, processes, and features described herein can be used with other types of cloud computing environments.

[0040] like Figure 1 As shown, according to an embodiment, cloud infrastructure environment 100 can operate on cloud computing infrastructure 102, which includes hardware (e.g., processors, memory), software resources, and one or more cloud interfaces 104 or other application programming interfaces (APIs) that provide access to shared cloud resources via one or more load balancers A 106, B 108. Cloud interface 102 includes user interfaces and APIs provided by cloud service providers for interacting with their cloud services. This includes tools and platforms that allow users and administrators to manage, configure, and monitor cloud resources and services. Cloud interface 102 may include a console, such as a web-based user interface, that provides a visual way to interact with and manage cloud resources. Through the console, users can, for example, create, configure, and monitor cloud services such as compute instances, databases, storage devices, and networking components. Cloud interface 102 may also include a command-line interface for users who prefer to operate the cloud infrastructure using command-line tools. In this embodiment, the CLI allows for scripting and automating cloud management tasks.

[0041] According to embodiments, load balancers A 106 and B 108 are services that distribute incoming network traffic across multiple servers, instances, or other resources to ensure that no single resource is overwhelmed by excessive demand. By distributing requests evenly across resources, load balancers enhance the responsiveness and availability of resources such as applications, websites, or databases. Load balancers A 106 and B 108 can be public load balancers accessible from the Internet and used for distributing external traffic, or they can be private load balancers used within a Virtual Cloud Network (VCN) and not accessible from the public Internet (and therefore well-suited for internal traffic distribution). In embodiments, load balancers A 106 and B 108 are designed for high availability and fault tolerance and are implemented in a redundant configuration across multiple availability or fault domains.

[0042] According to an embodiment, the cloud infrastructure environment supports the use of availability domains (such as availability domain A 180 and availability domain B 182), which enable customers to create and access cloud networks 184 and 186 and run cloud instances A 192 and B 194. In this embodiment, availability domain A 180 and availability domain B 182 may represent data centers, or a group of data centers located within a region. These availability domains may be isolated from each other, meaning they may not share the same physical infrastructure, such as power or cooling systems. This design provides a high degree of fault independence and robustness. In this embodiment, a fault domain can provide additional protection and resilience within a single availability domain by grouping hardware and infrastructure into availability domains isolated from other fault domains. This isolation may involve power, cooling, and other potential sources of failure.

[0043] According to an embodiment, a lease (a container of resources used by the tenant) can be created for each cloud tenant / customer (e.g., tenants A 142 and B 144). This lease provides a secure and isolated partition within the cloud infrastructure environment, where customers can create, organize, and manage their cloud resources. Cloud tenants / customers can access availability domains and cloud networks to access each of their cloud instances. Leases are isolated from other leases, ensuring that each customer's data and resources are secure and inaccessible to other customers. Within a lease, customers can create, manage, and organize various cloud resources, including compute instances, storage volumes, and networks. Identity and Access Management (IAM) services enable the management of users, groups, and policies within a lease. Through IAM, customers can control who has access to their resources and what actions they can perform. Leases also represent a level of disposal billing and subscription management. Usage and costs associated with resources within a lease are tracked and billed uniformly under that lease. Each lease can be associated with specific service limits and quotas for various resources. These limits can be used to help manage capacity and facilitate resource allocation among tenants.

[0044] According to an embodiment, a computing device (such as a client device 120 having device hardware 122 (e.g., a processor, memory) and a graphical user interface 126) can enable administrators or other users to communicate with a cloud infrastructure environment via a network (such as a wide area network, a local area network, or the Internet) to create or update cloud services.

[0045] According to an embodiment, the cloud infrastructure environment provides access to the shared cloud resource 140 via, for example, a compute resource layer 150, a network resource layer 160, and / or a storage resource layer 170. Customers can launch cloud instances as needed to meet computing and application requirements. After a customer has provisioned and launched a cloud instance, the provisioned cloud instance can be accessed from a client device (such as client device 120).

[0046] According to an embodiment, computing resources 150 may include resources such as bare-metal cloud instances 152, virtual machines 154, graphics processing unit (GPU) computing cloud instances 156, and / or containers 158. A bare-metal instance represents a physical server with dedicated hardware that is fully allocated to a single tenant. Bare-metal instances provide direct access to the server's processor, memory, storage devices, and other hardware resources. A virtual machine (VM) is a software emulation of a physical computer that runs an operating system and applications like a physical computer. VMs allow multiple operating systems to run on a single physical machine or across multiple physical machines. A hypervisor layer sits between the hardware and the VMs, thereby allocating physical resources (such as CPU, memory, and storage devices) to each VM. In an embodiment, GPU computing cloud instances provide GPUs as well as traditional CPU resources. These instances are designed for tasks requiring high levels of parallel processing power, making them well-suited for applications such as machine learning, scientific computing, 3D rendering, and video processing. In an embodiment, container 158 uses a virtualization approach that allows multiple isolated applications to run on a single controlling host, thereby virtualizing the operating system. Each container shares the host system's kernel but runs in an isolated user space, making the containers lightweight and efficient.

[0047] The components of computing resource 150 can be used to provision and manage bare-metal computing cloud instances, or to provision cloud instances as needed to deploy and run applications, just like in an on-premises data center. For example, according to an embodiment, the cloud infrastructure environment can provide control over physical host (bare-metal) machines within the computing resource tier that run directly on bare-metal servers as computing cloud instances without the need for a hypervisor.

[0048] According to an embodiment, the cloud infrastructure environment can also provide control over virtual machines within the computing resource layer, which can be launched from, for example, an image, wherein the type and quantity of resources available to the virtual machine cloud instance can be determined, for example, based on the image from which the virtual machine is launched.

[0049] According to an embodiment, the network resource layer may include several network-related resources, such as a Virtual Cloud Network (VCN) 162, a load balancer 164, an edge service 166, and / or a connectivity service 168. In this embodiment, the Virtual Cloud Network (VCN) is a customizable and private network in a cloud environment. The VCN provides a virtual version of a traditional network, including subnets, routing tables, and gateways. It allows users to set up a cloud-based network architecture according to their requirements. In this embodiment, the edge service 166 includes services and technologies designed to bring computing, data storage, and networking capabilities closer to the desired location. The edge service 166 can be used to optimize traffic, reduce latency, or provide other advantages.

[0050] According to embodiments, the storage resource layer may include several resources, such as data / block volume 172, file storage device 174, object storage device 176, and / or local storage device 178. Data / block volume 172 provides unformatted block-level storage, which can be used to create a file system for a managed database or for other purposes requiring unformatted storage. File storage device 174 provides a file system in embodiments and can provide a shared file system that can be accessed simultaneously by multiple instances using standard file storage protocols. Object storage device 176 manages data as objects within buckets. Objects have certain attributes, which may include data, metadata, and unique identifiers. Local storage device 178 refers to a storage device physically attached to a host computer.

[0051] like Figure 2 As shown, according to an embodiment, the cloud infrastructure environment may include a series of complementary cloud-based components, such as cloud infrastructure applications and services 200, which enable organizations or enterprise customers to operate their applications and services in a highly available managed environment.

[0052] According to an embodiment, a self-contained cloud region can be provided as a dedicated area of ​​Oracle Cloud Infrastructure (OCI) within an organization's data center, which provides data center operators with the flexibility, scalability, and cost-effectiveness of, for example, the OCI public cloud, while retaining full control over its data and applications to meet security, regulatory, or data residency requirements.

[0053] For example, according to an embodiment, such an environment may include racks physically and logically managed by a cloud infrastructure provider (e.g., Oracle), customer racks, access rights for cloud operators to perform setup and hardware support, customer data center power and cooling, customer footprint, customer data center personnel area, and physical access cages.

[0054] According to an embodiment, a dedicated zone provides tenants / customers with the same set of Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS) products or services, such as ERP, finance, HCM, and SCM, available in the public cloud regions of a cloud infrastructure provider (e.g., Oracle). Customers can seamlessly extract and replace legacy workloads using services from the cloud infrastructure provider (e.g., bare metal computing, VMs, and GPUs), database services (e.g., Oracle Autonomous Database), or container-based services (e.g., Oracle Kubernetes Container Engine).

[0055] According to an embodiment, a cloud infrastructure environment can operate according to an Infrastructure as a Service (IaaS) model, which enables the environment to provide virtualized computing resources over a public network (e.g., the Internet).

[0056] In the IaaS model, cloud infrastructure providers can host infrastructure components (e.g., servers, storage devices, network nodes (e.g., hardware), deployment software, platform virtualization (e.g., hypervisor layer), etc.). In some cases, cloud infrastructure providers can also supply various services to complement these infrastructure components; example services include billing software, monitoring software, logging software, load balancing software, or clustering software. Therefore, because these services can be policy-driven, IaaS users can implement policies to drive load balancing, thereby maintaining application availability and performance.

[0057] According to the implementation, IaaS customers can access resources and services via a wide area network (WAN) such as the Internet, and can use the cloud infrastructure provider's services to install the remaining elements of the application stack. For example, a user can log in to the IaaS platform to create virtual machines (VMs), install an operating system (OS) on each VM, deploy middleware such as databases, create buckets for workloads and backups, and install enterprise software into that VM. The customer can then use the provider's services to perform various functions, including balancing network traffic, troubleshooting application problems, monitoring performance, or managing disaster recovery.

[0058] According to the embodiments, cloud infrastructure providers may, but are not necessarily, third-party services that exclusively provide (e.g., provision, lease, or sell) IaaS. Entities may also choose to deploy private clouds, thereby becoming their own infrastructure service providers.

[0059] According to an embodiment, IaaS deployment is the process of placing a new application or a new version of an application onto a prepared application server, etc. It may also include the processing of a preparation server (e.g., an installation library or daemon). This is typically managed by the cloud infrastructure provider, below the hypervisor layer (e.g., servers, storage devices, network hardware, and virtualization). Therefore, the customer can be responsible for disposal (OS), middleware, and / or application deployment (e.g., on self-service virtual machines, etc., which can be started on demand).

[0060] According to embodiments, IaaS provisioning may refer to acquiring computers or virtual hosts for use and installing necessary libraries or services on them. In most cases, deployment does not include provisioning, and provisioning may need to be performed first.

[0061] According to embodiments, the challenges of IaaS provisioning include: the initial challenge of provisioning an initial set of infrastructure before anything is operational; and the challenge of evolving the existing infrastructure (e.g., adding new services, changing services, or removing services) once everything is provisioned. In some cases, these two challenges can be addressed by enabling configuration that declaratively defines the infrastructure. In other words, the infrastructure (e.g., which components are needed and how they interact) can be defined by one or more profiles. Therefore, the overall topology of the infrastructure (e.g., which resources depend on other resources and how they work together) can be described declaratively. In some cases, once the topology is defined, workflows for creating and / or managing the different components described in the profiles can be generated.

[0062] According to embodiments, cloud infrastructure can have many interconnected elements. For example, there may be one or more Virtual Private Clouds (VPCs) (e.g., potential on-demand pools of configurable and / or shared computing resources), also known as the core network. In some examples, one or more inbound / outbound traffic group rules may also be provided to define how inbound / outbound traffic will be set up for one or more virtual machines (VMs) in the network. Other infrastructure elements, such as load balancers, databases, etc., may also be provided. The infrastructure can evolve incrementally as more and / or more infrastructure elements are expected and added.

[0063] According to embodiments, continuous deployment techniques can be employed to enable the deployment of infrastructure code across various virtual computing environments. Furthermore, the described techniques enable infrastructure management within these environments. In some examples, service teams may write code that is intended to be deployed to one or more, but typically many, different production environments (e.g., across various geographical locations). However, in some examples, the infrastructure for deploying the code needs to be provisioned. In some cases, provisioning can be done manually, resources can be provisioned using provisioning tools, and / or once the infrastructure is provisioned, the code can be deployed using deployment tools.

[0064] Figure 3 The illustration shows an example cloud infrastructure architecture according to an embodiment.

[0065] like Figure 3 As shown, according to an embodiment, service operator 202 can communicatively couple to secure host lease 204, which may include virtual cloud network (VCN) 206 and secure host subnet 208.

[0066] In some examples, service operators may use one or more client computing devices, which may be portable handheld devices (e.g., telephones, computing tablets, personal digital assistants (PDAs)) or wearable devices (e.g., head-mounted displays), running software such as Microsoft Windows and / or various mobile operating systems (e.g., iOS, Android, etc.), and supporting the Internet, email, short message service (SMS), or other communication protocols. Alternatively, client computing devices may be general-purpose personal computers, including, for example, personal computers and / or laptops running various versions of Microsoft Windows®, Apple Macintosh®, and / or Linux operating systems. Client computing devices may be workstation computers running various commercially available UNIX® or UNIX-like operating systems, including but not limited to any of the various GNU / Linux operating systems (such as Chrome OS). Additionally or alternatively, client computing devices may be any other electronic device, such as thin client computers, Internet-enabled gaming systems (e.g., Microsoft Xbox game consoles), and / or personal messaging devices capable of communicating over networks with access to VCNs and / or the Internet.

[0067] According to an embodiment, the VCN may include a local peering gateway (LPG) 210, which may be communicatively coupled to a secure shell (SSH) VCN 212 via an LPG included in the SSH VCN. The SSH VCN may include an SSH subnet 214, and the SSH VCN may be communicatively coupled to a control plane VCN 216 via an LPG included in the control plane VCN. Furthermore, the SSH VCN may be communicatively coupled to a data plane VCN 218 via an LPG. The control plane VCN and the data plane VCN may be contained in a service lease 219 that may be owned and / or operated by a cloud infrastructure provider.

[0068] According to an embodiment, the control plane VCN may include a control plane demilitarized zone (DMZ) layer 220 that acts as a peripheral network (e.g., part of an enterprise network between an internal and external network). DMZ-based servers may have limited responsibility to help contain potential vulnerabilities. Furthermore, the DMZ layer may include one or more load balancer (LB) subnets 222, a control plane application layer 224 that may include an application subnet 226, and a control plane data layer 228 that may include database (DB) subnets 230 (e.g., one or more front-end DB subnets and / or one or more back-end DB subnets). One or more LB subnets included in the control plane DMZ layer may communicatively couple to one or more application subnets included in the control plane application layer and an Internet gateway 234 that may be included in the control plane VCN. One or more application subnets may communicatively couple to one or more DB subnets, a service gateway 236, and a Network Address Translation (NAT) gateway 238 included in the control plane data layer. The control plane VCN may include a service gateway and a NAT gateway.

[0069] According to an embodiment, the control plane VCN may include a data plane mirror application layer 240, which may include one or more application subnets. The one or more application subnets included in the data plane mirror application layer may include a virtual network interface controller (VNIC) capable of executing compute instances. The compute instances may communicatively couple the one or more application subnets of the data plane mirror application layer to the one or more application subnets that may be included in the data plane application layer.

[0070] According to an embodiment, the data plane VCN may include a data plane application layer, a data plane DMZ layer, and a data plane data layer. The data plane DMZ layer may include one or more LB subnets communicatively coupled to one or more application subnets of the data plane application layer and the internet gateway of the data plane VCN. The one or more application subnets may be communicatively coupled to the service gateway and NAT gateway of the data plane VCN. The data plane data layer may also include one or more DB subnets communicatively coupled to one or more application subnets of the data plane application layer.

[0071] According to an embodiment, the internet gateways of the control plane VCN and the data plane VCN can be communicatively coupled to a metadata management service 252, which can be communicatively coupled to a public internet 254. The public internet can be communicatively coupled to a NAT gateway of the control plane VCN and the data plane VCN. The service gateways of the control plane VCN and the data plane VCN can be communicatively coupled to a cloud service 256.

[0072] According to an embodiment, the service gateway of the control plane VCN or data plane VCN can make application programming interface (API) calls to cloud services without traversing the public internet. API calls from the service gateway to the cloud service can be unidirectional; the service gateway can make API calls to the cloud service, and the cloud service can send requested data to the service gateway. Generally, the cloud service cannot initiate API calls to the service gateway.

[0073] According to one embodiment, a secure host lease can be directly connected to a service lease that might otherwise be isolated. A secure host subnet can communicate with an SSH subnet via an LPG, which enables bidirectional communication between otherwise isolated systems. Connecting a secure host subnet to an SSH subnet allows the secure host subnet to access other entities within the service lease.

[0074] According to embodiments, a control plane VCN can allow users of a service lease to set up or otherwise provision desired resources. The desired resources provisioned in the control plane VCN can be deployed or otherwise used in the data plane VCN. In some examples, the control plane VCN can be isolated from the data plane VCN, and the data plane mirror application layer of the control plane VCN can communicate with the data plane application layer of the data plane VCN via VNICs, which can be included in both the data plane mirror application layer and the data plane application layer.

[0075] According to an embodiment, users or clients of the system can make requests, such as create, read, update, or delete (CRUD) operations, via the public internet that can transmit requests to the metadata management service. The metadata management service can transmit the request to the control plane VCN via an internet gateway. The request can be received by one or more LB subnets contained in the control plane DMZ layer. The one or more LB subnets can determine that the request is valid, and in response to this determination, the one or more LB subnets can transmit the request to one or more application subnets contained in the control plane application layer. If the request is verified and requires a call to the public internet, the call to the internet can be transmitted to a NAT gateway that can make the call to the internet. The metadata to be stored by the request can be stored in one or more DB subnets.

[0076] According to an embodiment, the data plane mirroring application layer can facilitate direct communication between the control plane VCN and the data plane VCN. For example, it may be desirable to apply configuration changes, updates, or other suitable modifications to resources contained in the data plane VCN. With the help of the VNIC, the control plane VCN can communicate directly with the resources contained in the data plane VCN, and thus can perform configuration changes, updates, or other suitable modifications.

[0077] According to one embodiment, the control plane VCN and data plane VCN can be included in a service lease. In this case, the system's users or customers may not own or operate the control plane VCN or data plane VCN. Alternatively, the cloud infrastructure provider may own or operate both the control plane VCN and data plane VCN, both of which can be included in a service lease. This embodiment enables network isolation, which can prevent users or customers from interacting with the resources of other users or customers. Furthermore, this embodiment allows the system's users or customers to privately store databases without relying on the public internet for storage, which may not provide the desired level of threat protection.

[0078] According to one embodiment, one or more LB subnets included in the control plane VCN can be configured to receive signals from the serving gateway. In this embodiment, the control plane VCN and the data plane VCN can be configured to be invoked by customers of the cloud infrastructure provider without invoking the public internet. Customers of the cloud infrastructure provider may expect this embodiment because the database(s) used by the customer can be controlled by the cloud infrastructure provider and can be stored on a service lease that is isolated from the public internet.

[0079] Figure 4 The illustration shows another example of a cloud infrastructure architecture according to an embodiment.

[0080] like Figure 4 As shown, according to an embodiment, the data plane VCN may be included in customer lease 221. In this case, the cloud infrastructure provider may provide a control plane VCN for each customer, and the cloud infrastructure provider may establish a unique compute instance for each customer included in the service lease. Each compute instance may allow communication between the control plane VCN included in the service lease and the data plane VCN included in the customer lease. The compute instance may allow resources provisioned in the control plane VCN included in the service lease to be deployed or otherwise used in the data plane VCN included in the customer lease.

[0081] According to an embodiment, a cloud infrastructure provider's customer may have a database managed and operated within a customer lease. In this example, the control plane VCN may include a data plane mirror application layer, which may include one or more application subnets. The data plane mirror application layer may reside in the data plane VCN, but it may not be provided within the data plane VCN. That is, the data plane mirror application layer may access the customer lease, but it may not exist in the data plane VCN, or be owned or operated by the customer. The data plane mirror application layer may be configured to invoke the data plane VCN, but it cannot be configured to invoke any entity contained within the control plane VCN. A customer may expect to deploy or otherwise use resources provisioned in the control plane VCN within the data plane VCN, and the data plane mirror application layer may facilitate the customer's desired deployment or other use of resources.

[0082] According to one embodiment, a cloud infrastructure provider's customers can apply filters to a data plane VCN. In this embodiment, the customer can determine what the data plane VCN can access and can restrict access to the public internet from the data plane VCN. The cloud infrastructure provider may not be able to apply filters or otherwise control the data plane VCN's access to any external networks or databases. Applying filters and controls to the data plane VCN included in the customer's lease helps isolate the data plane VCN from other customers and the public internet.

[0083] According to embodiments, cloud services can be invoked by a service gateway to access services that may not exist on the public internet, the control plane VCN, or the data plane VCN. The connection between the cloud service and the control plane VCN or data plane VCN may not be contiguous. Cloud services can reside on different networks owned or operated by a cloud infrastructure provider. Cloud services can be configured to accept calls from the service gateway and can be configured not to accept calls from the public internet. Some cloud services may be isolated from other cloud services, and the control plane VCN may be isolated from cloud services that may not be in the same region as the control plane VCN.

[0084] For example, according to an embodiment, the control plane VCN may be located in "Region 1", and the cloud service "Deployment 1" may be located in both Region 1 and "Region 2". If a service gateway contained in the control plane VCN located in Region 1 makes a call to Deployment 1, then the call can be transmitted to Deployment 1 in Region 1. In this example, the control plane VCN or Deployment 1 in Region 1 may not be communicatively coupled to or otherwise communicate with Deployment 1 in Region 2.

[0085] Figure 5The illustration shows another example of a cloud infrastructure architecture according to an embodiment.

[0086] like Figure 5 As shown, according to an embodiment, a trusted application subnet 260 can be communicatively coupled to a service gateway contained in a data plane VCN, a NAT gateway contained in a data plane VCN, and one or more DB subnets contained in a data plane data layer. An untrusted application subnet 264 can be communicatively coupled to a service gateway contained in a data plane VCN and one or more DB subnets contained in a data plane data layer. The data plane data layer may include one or more DB subnets that can be communicatively coupled to a service gateway contained in a data plane VCN.

[0087] According to an embodiment, one or more untrusted application subnets may include one or more primary VNICs (1)-(N) communicatively coupled to tenant virtual machines (VMs). Each tenant VM may be communicatively coupled to a corresponding application subnet 267 (1)-(N) that may be contained in a corresponding container egress VCN 268 (1)-(N), which may be contained in a corresponding customer lease 270 (1)-(N). A corresponding secondary VNIC may facilitate communication between the one or more untrusted application subnets contained in the data plane VCN and the application subnets contained in the container egress VCN. Each container egress VCN may include a NAT gateway communicatively coupled to the public internet.

[0088] According to an embodiment, the public internet can communicatively couple to a NAT gateway contained in a control plane VCN and a data plane VCN. Service gateways contained in the control plane VCN and the data plane VCN can communicatively couple to cloud services.

[0089] According to an embodiment, the data plane VCN can be integrated with customer leases. This integration may be useful or desirable for cloud infrastructure provider customers where additional support may be required when executing code. For example, a customer may provide code to be run that may be potentially destructive, may communicate with other customer resources, or may otherwise cause undesirable effects.

[0090] According to an embodiment, a cloud infrastructure provider's customer can grant temporary network access to the cloud infrastructure provider and request functionality to be attached to the data plane application layer. The code running this functionality can execute within a VM, and this code may not be configured to run anywhere else on the data plane VCN. Each VM can be connected to a customer lease. The corresponding containers (1)-(N) contained within the VM can be configured to run the code. In this case, dual isolation can exist (e.g., containers running code, where the containers may be contained within at least one VM contained in an untrusted application subnet), which helps prevent incorrect or otherwise unintended code from corrupting the cloud infrastructure provider's network or the networks of different customers. Containers can be communicatively coupled to the customer lease and can be configured to transmit or receive data from the customer lease. Containers can be configured not to transmit or receive data from any other entity in the data plane VCN. After the code execution is complete, the cloud infrastructure provider can dispose of these containers.

[0091] According to an embodiment, one or more trusted application subnets may run code that can be owned or operated by a cloud infrastructure provider. In this embodiment, one or more trusted application subnets may be communicatively coupled to one or more database subnets and configured to perform CRUD operations within the one or more database subnets. One or more untrusted application subnets may be communicatively coupled to one or more database subnets and configured to perform read operations within the one or more database subnets. Containers that may be contained within each customer's VM and may run code from the customer may not be communicatively coupled to the one or more database subnets.

[0092] According to embodiments, the control plane VCN and data plane VCN may be coupled without direct communication, or there may be no direct communication between them. However, communication can occur indirectly, where the cloud infrastructure provider can establish an LPG that facilitates communication between the control plane VCN and the data plane VCN. In another example, either the control plane VCN or the data plane VCN can invoke cloud services via a service gateway. For example, an invocation of a cloud service from the control plane VCN may include a request for a service that can communicate with the data plane VCN.

[0093] Figure 6 The illustration shows another example of a cloud infrastructure architecture according to an embodiment.

[0094] like Figure 6As shown, according to an embodiment, one or more trusted application subnets may be communicatively coupled to a service gateway contained in the data plane VCN, a NAT gateway contained in the data plane VCN, and one or more database subnets contained in the data plane data layer. One or more untrusted application subnets may be communicatively coupled to a service gateway contained in the data plane VCN and one or more database subnets contained in the data plane data layer. The data plane data layer may include one or more database subnets that can be communicatively coupled to a service gateway contained in the data plane VCN.

[0095] According to an embodiment, one or more untrusted application subnets may include primary VNICs that are communicatively coupled to tenant virtual machines (VMs) residing within one or more untrusted application subnets. Each tenant VM may run code in a corresponding container and may be communicatively coupled to an application subnet that may be included in a data plane application layer, which may be included in a container egress VCN 280. Corresponding auxiliary VNICs 282(1)-(N) may facilitate communication between the one or more untrusted application subnets included in the data plane VCN and the application subnets included in the container egress VCN. The container egress VCN may include a NAT gateway that is communicatively coupled to the public internet.

[0096] According to an embodiment, an Internet gateway contained in a control plane VCN and a data plane VCN can be communicatively coupled to a metadata management service, which can be communicatively coupled to the public Internet. The public Internet can be communicatively coupled to a NAT gateway contained in both the control plane VCN and the data plane VCN. A service gateway contained in both the control plane VCN and the data plane VCN can be communicatively coupled to a cloud service.

[0097] According to an embodiment, Figure 6 The pattern shown can be regarded as Figure 5 This is an exception to the pattern shown, and it may be the pattern desired by customers if the cloud infrastructure provider cannot communicate directly with them (e.g., in a disconnected region). Customers have live access to the corresponding containers contained within each customer's VM. Containers can be configured to invoke appropriate secondary VNICs contained in one or more application subnets within the data plane application layer, which may be contained in the container's egress VCN. The secondary VNICs can then route the calls to a NAT gateway, which can then route the calls to the public internet. In this example, the containers that customers can access live can be isolated from the control plane VCN and from other entities contained within the data plane VCN. Containers can also be isolated from resources from other customers.

[0098] In other examples, customers can use containers to invoke cloud services. In this example, a customer can run code within a container that requests a service from the cloud service. The container can then forward the request to a secondary VNIC, which in turn forwards it to a NAT gateway, which in turn forwards it to the public internet. The public internet can then be used to forward the request via an internet gateway to one or more load balancer (LB) subnets contained within the control plane VCN. In response to determining that the request is valid, the LB subnets can forward the request to one or more application subnets, which in turn forward the request to the cloud service via a service gateway.

[0099] It should be recognized that the IaaS architecture depicted in the figures above may have components other than those depicted. Furthermore, the embodiments shown in the figures are merely some examples of cloud infrastructure systems that can be combined with embodiments of this disclosure. In some other embodiments, the IaaS system may have more or fewer components than shown in the figures, may combine two or more components, or may have different component configurations or arrangements.

[0100] In some embodiments, the IaaS system described herein may include a suite of application, middleware, and database service providers delivered to customers in a self-service, subscription-based, elastically scalable, reliable, highly available, and secure manner.

[0101] Private Tag Cloud Environment

[0102] According to an embodiment, a cloud infrastructure environment can be used to provide a dedicated cloud environment, such as one or more privately tagged cloud environments, for tenants of the cloud infrastructure environment to use when accessing subscription-based software products, services, or other provisioning items associated with the cloud infrastructure environment.

[0103] Figure 7 The illustration shows how a system according to an embodiment can provide a dedicated or private tagged cloud environment for use by tenants or customers of a cloud infrastructure environment.

[0104] like Figure 7 As shown, according to an embodiment, a cloud infrastructure provider (e.g., OCI) may supply one or more private tag cloud (PLC) environments to a PLC operator 320 (e.g., an OCI customer operating as a reseller). The PLC operator / reseller can then customize and extend the private tag cloud for use by its customer 330 when accessing subscription-based software products, services, or other offerings associated with the cloud infrastructure environment.

[0105] For illustrative purposes, examples of such subscription-based products, services, or other offerings may include various Oracle cloud infrastructure software products, Oracle Fusion Applications products, or other types of products or services that allow customers to subscribe to use these products or services.

[0106] Figure 8 The illustration further illustrates the use of a privately labeled cloud domain by a tenant or customer of a cloud infrastructure environment, according to an embodiment.

[0107] like Figure 8 As shown, according to an embodiment, the system may include a cloud subscription service or component, such as an Oracle Cloud Subscription (OCS) service or component, which exposes one or more subscription management APIs for creating orders for joining new customers or initiating workflows for creating subscriptions and orchestrating billing and pricing services or other components for the PLC domain 400.

[0108] According to an embodiment, when a PLC operator or its customer requests a PLC environment, the system creates a PLC domain for lease use by one or more providers. A domain is a logical collection of one or more cloud regions that are isolated from each other and do not allow customer content to cross domain boundaries to regions outside that domain. Each domain is accessed independently. The PLC operator accesses cloud resources and services through cloud leases. A cloud lease is a secure and isolated partition of the cloud infrastructure environment, and it exists only within a single domain. Within this lease, if policies permit, the operator can access services in all regions within that domain and deploy workloads.

[0109] According to an embodiment, the first step of this process is to create an operator lease for the PLC operator, and then transfer the domain and associated areas to the PLC operator for subsequent management. The PLC operator then becomes the administrator of the lease, able to view and manage everything that happens within the domain, including its customer accounts and those customers' use of cloud resources.

[0110] Generally, once the domain is handed over or provided to the PLC operator, the cloud infrastructure provider cannot subsequently access the data within the operator's lease unless the operator authorizes the cloud infrastructure provider to do so, for example, to provide troubleshooting for potential problems.

[0111] According to an embodiment, the PLC operator can then create additional internal leases intended for its own internal use, such as for evaluating the end-customer experience, providing sales demonstration leases, or operating a database for its own internal use. The operator can also create one or more customer leases, with the end customer acting as its administrator. Cloud infrastructure usage metrics (e.g., compute usage, storage usage, and usage of other infrastructure resources) can be aggregated by the operator to reflect usage by both the operator and customers. Cloud infrastructure usage can be reported to the cloud infrastructure provider.

[0112] According to embodiments, a user interface or console may be provided that allows PLC operators to manage their customer accounts and customer-supplied services. Cloud infrastructure providers may also use cloud infrastructure leasing (e.g., converged application leasing) to install any required infrastructure services for use by operators and their customers.

[0113] Figure 9 The illustration further illustrates the use of a privately labeled cloud domain by a tenant or customer of a cloud infrastructure environment, according to an embodiment.

[0114] like Figure 9 As shown, according to an embodiment, a cloud subscription service or component exposes one or more subscription management APIs for creating orders for new customers or initiating workflows for creating subscriptions and orchestrating billing and pricing services or other components.

[0115] According to an embodiment, the system may also include a billing service or component that operates on a logical container for billing accounts or subscriptions and preferences used to generate invoices for customers.

[0116] According to an embodiment, the system may also include a subscription pricing service (SPS) or component that operates on a product catalog defining the products that customers can purchase. The subscription pricing service may also be used to provide a price list (e.g., a rate card) that is also available through the pricing service.

[0117] According to an embodiment, to support sales processing for creating subscriptions in the PLC domain, products can be selected from the product hub. Once an order is created, a subscription is created in a cloud subscription service, which then manages the subscription's lifecycle and provisiones the content required for provisioning in downstream services. The SPS component then manages pricing and usage aspects for the ability to collect final fees from PLC operators or for operators to charge their customers. Usage events are forwarded to a billing service or component, where, depending on the subscription's billing preferences, an invoice is created and pushed to the accounts receivable component.

[0118] According to an embodiment, although the services provided in the field report their usage to a metering service or component, such usage does not have any associated price. Billing processing, for example, determines the cost of each specific event by applying a rate card, determines the unit and cost of the subscription, associates the cost with the record, and then forwards it to the billing service or component.

[0119] like Figure 9 As further illustrated, according to an embodiment, a PLC operator can control multiple domains A and B. For example, an operator operating in multiple countries might wish to operate a data center completely isolated from the United States, and a separate data center completely isolated from Europe, for example, to meet governance or regulatory requirements. According to an embodiment, usage associated with these multiple domains can be aggregated for billing the operator.

[0120] The examples of the various systems provided above are intended to illustrate computing environments that can be used to provide dedicated or privately tagged cloud environments for tenants of cloud infrastructure to use when accessing subscription-based software products, services, or other provisioning items associated with the cloud infrastructure environment. According to other embodiments, the various components, processes, and features described herein can be used with other types of cloud computing environments.

[0121] Private tag cloud subscription

[0122] Figure 10 The illustration depicts a system according to an embodiment for providing access to software products or services in a cloud computing or other computing environment.

[0123] like Figure 10 As shown, according to an embodiment, the system can be provided as a cloud computing or other computing environment, referred to herein as a platform in some embodiments, which supports the use of subscription-based products, services or other offerings.

[0124] Examples of such subscription-based products, services, or other offerings may include various Oracle Cloud Infrastructure (OCI) software products, Oracle Fusion Applications products, or other types of products or services that allow customers to subscribe to use these products or services.

[0125] According to embodiments, subscriptions may include artifacts such as products, commitments, billing models, and states. The cloud subscription service may expose one or more subscription management APIs for creating orders to join new customers or initiating workflows to create subscriptions and orchestrate appropriate footprints in the billing and pricing services or components, as further described below.

[0126] According to an embodiment, a billing service or component operates on a logical container of billing accounts or subscriptions and preferences used to generate invoices. Each billing account generates one or more invoices per billing period. The billing service includes a first pipeline that receives usage and costs from a metering service or component. Usage can be received via a REST API or other interface. The billing service writes the usage to a database from which the billing service or other services can calculate and aggregate balances. The billing service may include a second pipeline responsible for obtaining aggregated usage and commitments and calculating costs for one or more billing intervals.

[0127] According to an embodiment, a subscription pricing service (SPS) or component operates on a product catalog that defines the products a customer can purchase. The product catalog forms the backbone of a price list (i.e., rate cards) also owned by the pricing service. Rate cards are modeled as pricing rules on top of publicly listed prices. The pricing service maintains a single price list for each product; new product prices can be added, and existing prices can be changed. The price lists have a complete history, with the latest version being the current rate card. Because some contracts may require snapshots of the rate cards, the pricing service handles this by recording when a customer's rate card was created and then querying the price list at that time.

[0128] According to embodiments, the SPS or pricing service is responsible for providing information about products, a global price list, and end-customer subscription-specific price lists and discounts. For example, according to embodiments, the SPS may synchronize product information from a product hub (e.g., Oracle Fusion Product Hub) and a global price list from a pricing hub (e.g., Oracle Fusion Pricing Hub).

[0129] According to an embodiment, the cloud subscription service operates as an upstream service to receive new order requests, for example, from the Oracle Fusion Order Management environment. The cloud subscription service can provide subscription information to the SPS service. Subscription details such as the timing of the quote, configuration, and subscription type (commitment, PayG) help the SPS determine the effective base price (rate card) for the subscription. The cloud subscription service can also send subscription discounts, for example, received from Oracle Fusion Order Management, which the SPS stores as a pricing rule entity.

[0130] In one embodiment, the SPS service runs as a background process to manage a rate card service or component responsible for generating rate cards for new subscriptions and updating them when prices change. The SPS service can expose APIs to access rate cards and pricing rules. The metering inline billing engine can leverage these APIs to obtain subscription-specific rate cards and pricing rules, using this data for cost calculations.

[0131] According to an embodiment, additional SPS components may include, for example, the Pricing / Product Hub Oracle Integration Cloud (OIC) integration component, which allows PLC operator entities to manage, for example, their product and price lists provided by the Oracle Fusion Product Hub and the Oracle Fusion Pricing Hub, respectively, within the environment to offer subscription-based products, services, or other offerings.

[0132] For example, according to this embodiment, the SPS OIC product integration process can listen for creation / update events in the product hub and invoke the SPS product API. Similarly, the SPS OIC pricing integration process can retrieve new price list creation from the pricing hub and invoke the corresponding SPS pricing API.

[0133] According to one embodiment, the system may also include an SPS core module that provides APIs for managing and accessing pricing entities. Pricing can be accessed through internal services such as an inline billing engine.

[0134] According to an embodiment, the system may also include a rate card manager component. The SPS service maintains a single base price for a product at a given time. However, the price of a subscribed product depends on the base price at the time of the quote configuration and the subscription's price list change policy attributes. The SPS service uses these attributes to internally maintain the prices to be used for subscriptions. These price lists are grouped in rate cards. The rate card manager can create and maintain rate cards, as well as listen for price list changes and update existing rate cards with the new prices. It also listens for new subscriptions and assigns rate cards based on subscription properties.

[0135] According to an embodiment, the system may also include a rule decoder engine. The SPS service is responsible for managing the pricing rules for subscriptions, including discounts offered to end customers. Pricing rule applicability can be based on product attributes such as discount groups, product categories, or specific SKUs. Internally, the SPS needs to identify a list of products to which these rules will apply. To achieve this, the rule decoder engine can compile the pricing rules into a format that the inline billing engine can use for cost calculations. This compilation process can be triggered when a product or pricing rule is created / updated.

[0136] If passed Figure 10As illustrated in the example, according to the embodiment: At 441, product and pricing information, such as that managed in a converged application, is sent to the SPS component. At 442, an order is sent to the Cloud Subscription Service component to create a subscription, rate card, and billing account. At 443, the pricing configuration and pricing rules for the new order are sent to the SPS. At 444, the Cloud Subscription Service is used to establish a billing account in the Billing Service or component. At 445, the Cloud Subscription Service publishes events to the Cloud Infrastructure Streaming component. At 446, billing data is sent to the Accounts Receivable component to generate an invoice. At 447, the Cloud Subscription Service uses Recycling and Subscription Lifecycle (RASL) events from the Cloud Infrastructure Streaming component. At 448, the Activation Service reads the Cloud Subscription Service event stream. At 449, the customer retrieves activation data from the portal. At 450, the Lease Lifecycle Service provisiones the lease as part of the subscription activation. At 451, the Lease Lifecycle Service creates an account footprint during account provisioning. At 452, the Lease Lifecycle Service sets a limit template during account provisioning. At 453, the account component acts as a downstream RASL client to handle legacy repossession. At 454, aggregated costs and usage are sent to the billing service or component. At 455, organizations can create subleases using the lease lifecycle service. At 456, the metering service or component retrieves subscription mapping data. At 457, the subscription service retrieves organizational data used for subscription mapping. At 458, RASL reads the cloud subscription service event stream. At 459, the subscription service reads the cloud subscription service event stream; and at 460, the metering service or component retrieves rate card data for each subscription, which can then be used to collect final charges from PLC operators or enable PLC operators to charge their customers.

[0137] The examples provided above are intended to illustrate computing environments that can be used to provide dedicated or privately tagged cloud environments for use by tenants of cloud infrastructure when accessing subscription-based software products, services, or other provisioning items associated with the cloud infrastructure environment. According to other embodiments, the various components, processes, and features described herein can be used with other types of cloud computing environments.

[0138] 2. Deploy and orchestrate cloud system

[0139] Figure 11 The illustration depicts a cloud system 1100 according to one or more embodiments. For example... Figure 11As shown, cloud system 1100 includes domains 1102 and 1104; each domain is a cloud partition of the cloud environment and a logical grouping of one or more geographic regions. An entity referred to herein as a cloud service provider (CSP) operates the cloud environment, where other entities (e.g., its customers) can host services in their respective partitions. As described herein, according to one or more embodiments, the cloud environment can be partitioned into a dedicated environment managed by the CSP customer or a PLC environment.

[0140] In one or more embodiments, system 100 may include a ratio Figure 11 The components shown may have more or fewer components. Figure 11 The components shown can be located locally or remotely to each other. Figure 11 The components shown can be implemented using software and / or hardware. These components can be distributed across multiple applications and / or machines. Multiple components can be combined into a single application and / or machine. Operations described for one component can be performed by another component.

[0141] In one or more embodiments, domain 1102 (which may be referred to as a CSP domain) includes data centers in at least the primary region 1106 and the commercial region 1108. The CSP may use domain 1102 to manage other domains within the cloud system 1100. The CSP, exercising a degree of control over the cloud system 1100, directs the propagation of resource deployments in domain 1104, but is limited to requesting customer permission to perform these resource deployments. In some cases, the CSP is permitted to perform resource deployments even if its customers object.

[0142] Resource deployments typically update / change a set of services available to a specific entity or its customers. It should be noted that the resource deployments described in this article cover the deployment of any type of resource, including storage, network, and compute resources. Some resources include microservices and cloud services, such as application services, network services, storage services, infrastructure services, platform services, etc.

[0143] In one or more embodiments, domain 1104 (which may be referred to as a PLC domain) includes at least a data center in home zone 1110. Domain 1104 may be synonymous with a lease for the aforementioned CSP customer (i.e., a tenant customer). Other embodiments may have more or fewer additional (e.g., geographically distinct) zones in domain 1102 and / or domain 1104. The data center in home zone 1110 may be configured to run multiple hardware / software components (such as a partition-specific deployment orchestrator 1112) and store various types of data (such as constraints 1114). Similarly, the data center in the main zone 1106 of domain 1102 may be configured to run multiple hardware / software components (such as a global deployment orchestrator 1116) and store various types of data (such as constraints 1118 and status 1120).

[0144] Generally, the partition-specific deployment orchestrator 1112 and the global deployment orchestrator 1116 coordinate the disposal of pending resource deployments. Both are configured to reject or approve the pending resource deployment by evaluating it based on at least a portion of the state 1120 of the cloud system 1110. The rejection or approval of any deployment request may depend on the evaluation of the deployment request based on at least a portion of the state of the specific cloud domain hosted by the cloud system 1100. This evaluation may include the partition-specific deployment orchestrator 1112 and / or the global deployment orchestrator 1116 evaluating the pending resource deployment according to their respective deployment constraints in constraints 1114 and / or 1118.

[0145] In one embodiment, the partition-specific deployment orchestrator 1112 directly invokes the deployment service in home zone 1110 to deploy resources to domain 1104. In another embodiment, the global deployment orchestrator 1116 directly invokes the same deployment service in home zone 1110 to deploy resources to domain 1104. Regardless of which component invokes the deployment service, CSP access constraints 1118 are prevented, while tenant clients retain control over the application of these constraints 1118. Therefore, the global deployment orchestrator 1116 can be referred to as a CSP-operated service, and the partition-specific deployment orchestrator 1112 can be referred to as a client- or tenant-operated service. Domain 1102 can be referred to as a CSP domain, and domain 1104 can be referred to as a tenant domain (or, in some cases, a PLC domain).

[0146] Generally, both constraint 1114 and constraint 1118 refer to a set of deployment constraints that will be imposed on any resource deployment in a specific domain (such as domain 1104). A typical resource deployment refers to the installation of a new version / upgrade of one or more software products. For example, a sample deployment request might describe a service deployment in the form of a set of microservice versions of a specific service. Therefore, deployment constraints can be viewed as rules that one of the CSPs or its clients (hereinafter referred to as "tenants") can set to control whether / when a resource deployment (or "deployment") is performed in any domain (such as domain 1104).

[0147] Constraint 1114 includes any global deployment constraints established by the CSP, and constraint 1118 includes any partition-specific deployment constraints established by a specific CSP customer with a lease in the cloud environment. Similar to how tenant operators (e.g., PLC operators) need to impose constraints on deployments within their domains, CSP operators also need to impose constraints on these deployments.

[0148] In some embodiments of cloud system 1100, constraint 1114 and / or constraint 1118 are combined into a two-layer constraint, where a global deployment orchestrator 1116 performs the evaluation of constraint 1114, and a partition-specific deployment orchestrator 1112 performs a separate evaluation of constraint 1118. If both evaluations are successful for a pending deployment request for a resource, then the partition-specific deployment orchestrator 1112 continues to deploy the resource in domain 1104. In other embodiments, only constraint 1118 is evaluated.

[0149] The following describes various example global deployment constraints according to some embodiments of constraint 1114. One example global deployment constraint prevents resources from being deployed to more than a threshold number (e.g., five) of regions within a given timeframe. Another example global deployment constraint prevents resources from being deployed to the government domain before deployment to the commercial domain is completed. Another example of a global deployment constraint defines at least one logical region group and then prevents deployment to any region not in that at least one logical group while deployment to that at least one logical group is in progress. Another example of a global deployment constraint enforces a redundancy set of public notices. Another example of a global deployment constraint enforces a rule to begin deployments to less frequently used regions. Another example of a global deployment constraint enforces pre-production deployments to environments not publicly available. Another example of a global deployment constraint prevents resources from being deployed to a PLC domain (such as domain 1104) before a deployment completion threshold percentage for a CSP domain (such as domain 1102) is reached. Another example of a global deployment constraint prevents an upgraded cloud environment from being instantiated during a change freeze. This example may apply during the deployment of new versions of services across multiple domains of an upgraded cloud environment.

[0150] State 1120 generally refers to the global state and / or partition-specific state of at least a portion of the cloud environment hosted by cloud system 1100 at a specific moment. State 1120 may include information describing a set of services currently deployed and hosted by a specific cloud partition (such as domain 1102 and domain 1104) of the cloud environment. State 1120 includes (recent) state information for regions within domains 1102 and 1104. In one embodiment of state 1120 of cloud system 1100, the state information includes the state of resource deployments in domain 1104.

[0151] The global deployment orchestrator 1116 can dispose of constraint 1118 by evaluating constraint 1118 before setting the order in which pending resource deployments are performed (e.g., deployment order). For any given set of software products, the deployment order is the order in which the service environments are deployed.

[0152] The partition-specific deployment orchestrator 1112 can handle constraint 1114 by also evaluating constraint 1114 before setting the deployment order. In this case, constraint 1114 is used to specify intent on behalf of the tenant customer (or simply tenant). Evaluating constraint 1114 can produce a dynamic deployment order, thus eliminating the need for the CSP operator to predefine an ordered graph of the service environment. Therefore, the deployment order is not determined at the start of the version set deployment; rather, it is dynamically determined, for example, by continuously evaluating what to do next as the deployment progresses. While the version set deployment is underway, the global deployment orchestrator 1116 updates the state information stored in state 1120 according to the deployment progress and evaluates which environments should have begun deployment.

[0153] In one embodiment, the CSP imposes partition-specific deployment constraints 1114 with global constraints. For example, a global constraint prohibiting a constraint on one domain that would hinder deployment in another domain. Within a given domain, if the request satisfies any existing domain-specific deployment constraints, then the deployment is allowed to proceed.

[0154] One or more embodiments may include a snapshot of at least a portion of cloud system 1100 in the deployment request sent to the disconnected domain. The disconnected domain has no knowledge of the state of the “global” cloud infrastructure; therefore, the disconnected domain cannot evaluate constraints based on the state of the (global) cloud system 1100. Partition-specific deployment constraints for the disconnected domain can be evaluated based on the snapshot.

[0155] Given the software-defined networking (SDN) architecture of some embodiments of cloud system 1100, each region of the domain includes a control plane and a data plane. A partition-specific deployment orchestrator 1112 may include applications and / or services (e.g., PLC valve services) running on the data plane and / or control plane in the home region 1110, and is configured to generate application programming interfaces (APIs) and accessible user interfaces (UIs). Tenant interface 1122 (e.g., such as...) Figure 13 The console shown in the diagram represents an example embodiment of an accessible UI that allows a tenant operator (e.g., a PLC operator) to manage a set of partition-specific deployment constraints for resource deployments in domain 1104. The PLC operator can use tenant interface 1122 to maintain and store a set of up-to-date partition-specific deployment constraints in constraint 1118 by defining new constraints, viewing / removing current constraints, and / or adding predefined constraints. Similarly, a CSP operator can use CSP interface 1124 to maintain and store a set of up-to-date global deployment constraints in constraint 1114. The CSP operator can also use CSP interface 1124 to define resource dependencies, security version ranges / upgrade paths for any resource, etc.

[0156] A partition-specific deployment orchestrator 1112, running another API in the data plane of home zone 1110, extends the evaluation of constraint 1118 to the global deployment orchestrator 1116. Thus, via another API in the data plane, the global deployment orchestrator 1116 communicates with home zone 1110, while the partition-specific deployment orchestrator 1112 receives and evaluates any deployment requests transmitted from zone 1102. In response to a deployment request, the partition-specific deployment orchestrator 1112 determines whether the requested deployment can be executed (e.g., immediately); if yes, then the partition-specific deployment orchestrator 1112 can return a message indicating approval of the deployment request (e.g., a "yes" response); if no, then the partition-specific deployment orchestrator 1112 can return a message indicating rejection (e.g., a "no" response). In one embodiment, the message may indicate a yes / no response. When a deployment attempt passes both global deployment constraints and partition-specific deployment constraints, the partition-specific deployment orchestrator 1112 returns an approval message, and then the global deployment orchestrator 1116 executes the requested deployment in region 1104. For example, the global deployment orchestrator 1116 may send a message to the in-region deployment service, prompting that the requested deployment be automatically performed at least in the home region 1110.

[0157] In one or more embodiments, a tenant is a company, organization, enterprise, or other entity that accesses shared computing resources, such as applications or another cloud resource (e.g., storage resources, network resources, etc.). One tenant's business or operations are separate from those of another tenant. The CSP described herein constitutes the entity. In an embodiment, the tenant of management domain 1104 is a second entity from the CSP and is independent of other tenants in other domains operated by the CSP. In one embodiment, the tenant of management domain 1104 hosts a cloud environment operated by the tenant, which is further partitioned within at least one third entity. In this way, the tenant of management domain 1104 can provide cloud services to its own customers independently of the CSP.

[0158] In one or more embodiments, both the partition-specific deployment orchestrator 1112 and the global deployment orchestrator 1116 refer to hardware and / or software configured to perform the operations described herein to enable tenant control over resource deployments within a tenant's domain. An example of the operation of the global deployment orchestrator 1116 is referenced below. Figure 12 Describe it.

[0159] In this embodiment, the global deployment orchestrator 1116 is implemented on one or more digital devices. The term "digital device" generally refers to any hardware device that includes a processor. A digital device can refer to a physical device that executes an application or virtual machine. Examples of digital devices include computers, tablets, laptops, desktops, netbooks, servers, web servers, network policy servers, proxy servers, general-purpose machines, function-specific hardware devices, hardware routers, hardware switches, hardware firewalls, hardware network address translation (NAT), hardware load balancers, mainframes, televisions, content receivers, set-top boxes, printers, mobile handheld terminals, smartphones, personal digital assistants (PDAs), wireless receivers and / or transmitters, base stations, communication management equipment, routers, switches, controllers, access points, and / or client devices.

[0160] In one or more embodiments, tenant interface 1122 refers to hardware and / or software configured to facilitate communication between the operator and global deployment orchestrator 1116. Tenant interface 1122 renders user interface elements and receives input via these elements. Examples of interfaces include graphical user interfaces (GUIs), command-line interfaces (CLIs), haptic interfaces, and voice command interfaces. Examples of user interface elements include checkboxes, dialog boxes, radio buttons, drop-down lists / menus, list boxes, buttons, toggle buttons, text fields, date and time pickers, command lines, sliders, pages, and forms.

[0161] In this embodiment, different components of the tenant interface 1122 are specified using different languages. The behavior of user interface elements is specified using a dynamic programming language such as JavaScript. The content of user interface elements is specified using a markup language such as Hypertext Markup Language (HTML) or XML User Interface Language (XUL). The layout of user interface elements is specified using a stylesheet language such as Cascading Style Sheets (CSS). Alternatively, the tenant interface 1122 may be specified using one or more other languages ​​such as Java, C, or C++.

[0162] 4. Control the deployment of cloud resources

[0163] Figure 12 The illustrations depict a set of example operations of a system for controlling resource deployment in a cloud environment according to one or more embodiments. One example implementation of the system includes a global deployment orchestrator for the cloud system. Figure 12 One or more operations shown can be modified, rearranged, or omitted together. Therefore, Figure 12 The specific order of operations shown should not be construed as a limitation on the scope of one or more embodiments.

[0164] In one embodiment, the system activates deployment constraints (operation 1200) after evaluating global tenant constraints. One embodiment of the system includes a global deployment orchestrator configured to evaluate tenant-defined rules (i.e., deployment constraints) against global constraints and then activate any authorized rules as partition-specific deployment constraints. The global deployment orchestrator can obtain proposed tenant-defined rules from communications received from cloud partitions (e.g., domains). Tenants of a cloud partition can define attributes of the constraints selected by the tenant using a graphical user interface (GUI). A console can present a library of permissible constraints and selectable attributes on the GUI to further define one or more of the permissible constraints.

[0165] The aforementioned global restrictions refer to restrictions imposed on partition-specific deployment constraints. In one embodiment of the system, if a constraint defined by an example tenant satisfies each global restriction, then the global deployment orchestrator authorizes that constraint. Once satisfied, the global deployment orchestrator authorizes activation of the tenant-defined constraint. Once activated, the authorized constraint becomes an established partition-specific deployment constraint for use by the partition-specific deployment orchestrator when controlling resource deployment within the cloud partition. Conversely, in response to determining that a tenant-selected deployment constraint does not satisfy one or more global restrictions on permissible partition-specific deployment constraints, the partition-specific deployment orchestrator refuses to authorize / activate that tenant-defined constraint as a partition-specific deployment constraint.

[0166] In this embodiment, the system stores cloud environment status information based on monitoring data associated with one or more cloud partitions (Operation 1202). Monitoring data refers to any information that describes at least a portion of the status of a specific cloud partition (which may be a cloud domain or a specific area within that cloud domain). An example of monitoring data includes version information (e.g., version number) of different resources (e.g., running software products) within a specific cloud partition. Numerous hardware / software components are configured to collect or detect such information (such as health services) and maintain the up-to-date status of any cloud partition. The health service described herein can generate and / or receive monitoring data from a specific cloud partition and continue to update any previous status information of that partition. In one embodiment, a global deployment orchestrator uses the health service to continuously monitor at least a portion of the cloud environment to determine the status of the cloud environment.

[0167] In some embodiments, the system uses state information to prepare deployment requests for transmission to at least one cloud partition. In one embodiment of the system, a global deployment orchestrator generates deployment requests to indicate the resources to be deployed and their version information based on monitoring data. Example deployment requests include example monitoring data indicating version information of software products being upgraded / modified or otherwise deployed in a cloud partition. Another example deployment request includes snapshots of one or more of the same software products already deployed in a reference cloud partition (e.g., a CSP realm). In this way, the global deployment orchestrator can provide accurate deployment plans for partition-specific deployment orchestrators, and partition-specific state information and / or one or more partition-specific deployment constraints are evaluated on an immediate (i.e., dynamically) basis. In one or more embodiments, the system is limited to including a version set in the deployment request and therefore does not use any other state information to prepare the deployment request.

[0168] In one embodiment, the system transmits a deployment request to one or more cloud partitions (Operation 1204). One embodiment of the system evaluates global deployment constraints against the deployment to be processed and, after determining that the deployment meets the global deployment constraints, transmits the deployment request to the cloud partition. Therefore, the system confirms the satisfaction of global deployment constraints before transmitting the deployment request. In one embodiment, the system runs automated processing for deployment request transmission within a cloud partition in a cloud environment. These global deployment constraints may include time and date constraints, dependency constraints, minimum version constraints, and any other conceivable rules to be evaluated before transmitting the deployment request. This document addresses... Figure 11 Several example global deployment constraints are described.

[0169] In another embodiment, the system transmits the same deployment request to both a first cloud partition and a second cloud partition, where both cloud partitions are associated with the same tenant or different tenants. In this way, the system can deploy the same resource (or the same resource version) to both cloud partitions depending on their respective states. In one example, the system could deploy a resource in the first cloud partition but avoid such a deployment to the second cloud partition, and vice versa. In another example, the system could transmit separate deployment requests for different versions of the same resource to each cloud partition. A partition-specific deployment orchestrator in each cloud partition can use partition-specific state information obtained from the partition's in-partition data repository to evaluate the deployment request.

[0170] In one embodiment, the system receives a message indicating whether to approve or reject a pending deployment request (operation 1206). In one embodiment, a partition-specific deployment orchestrator in a cloud partition generates and transmits this message after evaluating the deployment request based on the state of the cloud environment and then approving the deployment of a first resource based on a set of partition-specific deployment constraints. Similar to global deployment constraints, partition-specific deployment constraints may include time and date constraints, dependency constraints, minimum version constraints, and any other conceivable rules to be evaluated before approving the deployment request. The partition-specific deployment orchestrator in a cloud partition can perform the evaluation by determining whether the first deployment request satisfies at least one partition-specific deployment constraint associated with the cloud partition. In one embodiment, the partition-specific deployment orchestrator rejects any deployment request that does not satisfy the set of partition-specific deployment constraints by returning a message indicating rejection.

[0171] In one embodiment used to evaluate minimum version constraints, the state of the cloud environment includes the version number of a second resource deployed in a cloud partition; evaluating the deployment request includes determining that the version number of the second resource meets the minimum version requirement. In one embodiment, the first resource depends on the second resource.

[0172] In another embodiment of the minimum version constraint, the state of the cloud environment includes the current version number of resources deployed in the cloud partition; evaluating a deployment request based on the state of the cloud environment includes determining whether the difference between the target version number and the current version number is less than a threshold. In one embodiment, a partition-specific deployment orchestrator approves such a deployment request for the target version number only if the difference between the current version of the resource and the target version of the resource is less than the threshold.

[0173] In one embodiment, the state of the cloud environment includes the state of resource deployments in a second cloud partition (e.g., a reference cloud partition) among multiple cloud partitions. The operations performed by the system include evaluating a deployment request for a first resource based on the state of the cloud environment, and determining that the first resource has been deployed to the second cloud partition. An example of a second cloud partition is... Figure 11 The diagram is illustrated as Domain 1102 (i.e., the business domain). In one embodiment, a partition-specific deployment orchestrator prepares a deployment request to include the status of resource deployments in a second cloud partition out of a plurality of cloud partitions. The status of resource deployments in the second cloud partition may be referred to herein as a snapshot.

[0174] In one embodiment, the system performs a determination regarding whether to deploy the deployment request (operation 1208). In one embodiment, based on receiving a message indicating approval or rejection of the deployment request, the system determines whether the deployment outlined in the deployment request should be performed or whether to avoid performing the deployment. The system may perform the deployment in response to approval by a partition-specific deployment orchestrator. In some cases, the system may also perform the deployment in response to rejection by a partition-specific deployment orchestrator (e.g., when one or more coverage conditions exist). While the system may determine to perform the deployment in response to approval or rejection by a partition-specific deployment orchestrator, conversely, the system may also avoid performing the deployment in response to approval or rejection by a partition-specific deployment orchestrator, as explained below. There may be a variety of reasons for a rejection indicated in the received message.

[0175] In one embodiment, the system avoids executing the deployment outlined in the deployment request (operation 1210). In one embodiment, the system determines to avoid deploying resources in the cloud partition based on receiving a message indicating that the deployment request is rejected. If such a message is detected, the system may halt the transmission of pending automated deployments. As explained below, even after rejection, the system still has several ways to execute the aforementioned deployment.

[0176] Even after the initial failure of the first deployment request, the system can still retry deploying resources in the cloud partition (to satisfy a set of partition-specific deployment constraints associated with the cloud partition). An embodiment of the global deployment orchestrator, after the first deployment request is rejected and resources are avoided from being deployed to the cloud partition, transmits a second deployment request to deploy the resources to the cloud partition. In response to receiving a message from the cloud partition instructing approval of the second deployment request, the global deployment orchestrator performs the deployment by deploying the resources to the cloud partition.

[0177] The system can override a rejection of a deployment request (due to failure to meet a set of partition-specific deployment constraints associated with the cloud partition) and continue deploying resources to the cloud partition. Implementations of the global deployment orchestrator perform resource deployment after determining that a deployment request is associated with constraint coverage conditions. Example constraint coverage conditions may allow resources to be deployed to the cloud partition at times when those constraints would otherwise prohibit deployment. Another example constraint coverage condition may allow urgent deployments that would otherwise be prohibited by partition-specific deployment constraints.

[0178] In one embodiment, a global deployment orchestrator transmits a deployment request instructing an emergency deployment (e.g., an emergency update including critical patches), and subsequently, a partition-specific deployment orchestrator performs the emergency deployment without any evaluation (e.g., for partition-specific deployment constraints). The emergency deployment may contain minimal updates, where changes are limited to those necessary to address the emergency situation with respect to the most recent (e.g., latest) deployment in the CSP domain. In another embodiment, the emergency deployment includes at least one update between the current version and a newer version of the resource.

[0179] In one embodiment, the system executes the deployment outlined in the deployment request (operation 1212) within a cloud partition. In one embodiment, the system transmits deployment instructions to a specific deployment service within the partition (e.g., a domain or region deployment service), where these instructions guide the execution of the deployment.

[0180] After deployment, the system updates any status information of the cloud partition (operation 1214). In this embodiment, the system presents the deployment status on the operator's console (operation 1216). Assuming the deployment has been successfully installed in the cloud partition, the system can present a status indicating that one or more upgrades / changes have been successfully deployed to the software product running in the cloud partition.

[0181] 5. Example Implementation

[0182] For clarity, detailed examples are described below. The components and / or operations described below should be understood as specific examples and may not be applicable to some embodiments. Therefore, the components and / or operations described below should not be construed as limiting the scope of any claims.

[0183] Figure 13 The illustration shows an example embodiment of the operator interface for a partition-specific deployment orchestrator. Figure 13 The operator interface depicted is represented by console 1300. A partition-specific deployment orchestrator, configured to facilitate tenant control of cloud resource deployments within specific cloud partitions of cloud environment 1302, can generate a graphical user interface (GUI) for console 1300.

[0184] Various embodiments of console 1300 present content including various information and / or UI controls. Some examples of UI controls enable the definition of rules that control changes / updates that can be made to one or more specific cloud partitions. These rules are partition-specific deployment constraints used when evaluating deployment requests from entities in the operational cloud environment 1302. Some examples of the various information describe partition-specific deployment constraints.

[0185] Specifically, console 1300 is presented in... Figure 13 This describes various information and / or UI controls corresponding to a cloud environment 1302, which is depicted as a network (e.g., an SDN-based network). As described herein, cloud environment 1302 includes multiple data centers located in different geographical regions and interconnected via networking elements. Cloud environment 1302 can be partitioned into multiple domains (such as domain 1304), where a domain includes logical groups of data centers across multiple geographical regions. Cloud environment 1302 forms a network hierarchy in which multiple end-user devices are located at the lowest level.

[0186] As described herein, a CSP can operate its customers' leased cloud environments 1302 and can use console 1300 to specify constraints for deployments to their respective cloud partitions (e.g., domains). It should be noted that these constraints can apply to domain 1304 or multiple domains. The CSP can run services within a CSP domain of cloud environment 1302, including a global deployment orchestrator 1306. The global deployment orchestrator 1306 transmits deployment requests to domain 1304, and upon receiving such requests, the domain 1304's partition-specific deployment orchestrator automatically evaluates the partition-specific deployment constraints applicable to that domain 1304.

[0187] One of the CSP's tenants can be assigned to domain 1304 to manage and / or run its own partition as cloud environment 1302. In one embodiment, the tenant can host its own (guest) cloud environment on that partition. Thus, the tenant's operator can use console 1300 to define candidates for partition-specific deployment constraints. These candidates are constrained by global limits before the global deployment orchestrator 1306 can activate any candidates to apply to domain 1304. The tenant's operator can activate control 1308 (in... Figure 13The "Constraint Definition" attribute (illustrated in the image) defines candidate deployment constraints through one or more attributes: resource name, constraint type, and / or (one or more) regions. The resource name identifies one or more cloud resources (e.g., one or more software products) that the operator expects to control via deployment constraints that are being deployed and / or upgraded. If the subject of the deployment request is the same resource, then the partition-specific deployment orchestrator applies any deployment constraints with that resource name. The constraint type refers to the category of deployment constraints that the operator can choose to define candidate deployment constraints. Example categories of such candidates could include dependency constraints, time / date constraints, minimum version requirement constraints, etc. The (one or more) region attribute specifies one or more geographical regions to which the deployment constraints can be applied. Figure 13 The selected geographic area in field 1304 is distinguished from other areas by using shading. Control 1308 also includes a drop-down menu for selecting the appropriate value for the attribute.

[0188] The tenant's operator can use control 1310 (which is in Figure 13 The operator selects one or more predefined constraints (labeled "Constraint Library") to activate as one or more partition-specific deployment constraints. Predefined constraints are permissible partition-specific deployment constraints applied to future resource deployments in domain 1304. Control 1310 can present a curated and supported set of permissible partition-specific deployment constraints that a tenant's operator can select. The global deployment orchestrator 1306 evaluates the operator-selected one or more partition-specific deployment constraints against global constraints defined by the CSP; if the operator-selected constraint satisfies the global constraints, then the global deployment orchestrator 1306 activates the operator-selected constraint to apply to domain 1304. Otherwise, the global deployment orchestrator 1306 rejects the selected constraint because it fails to satisfy all global constraints.

[0189] Some example constraints can restrict resource deployments to only one region within Domain 1304 at a given time, and / or when there are at least two days between subsequent deployments or between deployments in one region and deployments in another region. Another example can restrict deployments so that each deployment starts and ends within a deployment window (e.g., between 9 PM and 6 AM or on weekends). Another example can constrain deployments to certain resource types (e.g., services such as compute services) and / or specific versions of resources. Another example can pause deployments until successful deployment to the CSP domain is complete and / or after a certain number of days. Another example can pause deployments a few days after a release. Another example can only deploy resources after consent has been granted or after a three-day timeout period has elapsed. Another example can allow changes to be frozen, such as between January 3rd and January 12th. The supported constraint library may evolve over time.

[0190] Regardless of whether the operator selects permissible partition-specific deployment constraints from the constraint library in control 1310, or defines permissible partition-specific deployment constraints using control 1308, deployment constraints are subject to global limits. Console 1300, with the support of a partition-specific deployment orchestrator, enforces global limits by rejecting any violating candidate constraints. Some example global limits prevent one partition from hindering deployments in another partition. One example limit requires rejecting any candidate constraints for deployments in domain 1304 until deployments in another domain (other than the CSP domain) or every other domain in the cloud environment 1302 are completed. Some example limits require rejecting candidates that constrain deployments to a limited deployment window (e.g., deploying to my domain on the first Saturday of each month, or deploying the data plane to my domain on the weekend). Infrequent deployments may cause malfunctions in domain 1304.

[0191] As described in this article, one example constraint type is a time and date constraint, which restricts one or more of the time, duration, or frequency of deployments allowed in a cloud partition. An example time and date constraint refers to a specific time period (i.e., a deployment window) during which resources are allowed (or disallowed). Time and date constraints are partition-specific deployment constraints that restrict one or more of the time, duration, or frequency of deployments allowed in the first cloud partition. Tenant operators can use control 1312 (in...) Figure 13 The "Time and Date Constraints" option sets time and date constraints for any future resource deployments.

[0192] As described herein, another example constraint type is a dependency constraint, which refers to a resource dependency between a first resource and at least one second (dependent) resource. A tenant's operator can use control 1314 to set one or more dependent resources for a constraint definition. In one embodiment, the CSP operator defines dependencies between resources for the tenant's operator to select as one or more dependent resources for the constraint definition. The CSP operator can also define version ranges applicable to one or more dependent resources.

[0193] Another example constraint type is the minimum version requirement constraint, which refers to setting a minimum version requirement and / or a difference threshold between the current version and the proposed target version of the resource to be deployed in domain 1304. In one embodiment, the CSP operator can set a range of security versions and / or security upgrade paths for any resource and / or any dependent resources of that resource. In this way, the tenant's operator can use control 1308 to set the minimum version requirement constraint and use control 1314 to set dependency constraints in a combined deployment constraint definition.

[0194] Console 1300 includes multiple information dialog boxes, including control 1316 (in... Figure 13 (Catalyzed as "Constraint List"). Control 1316 can be configured to enumerate candidate and active deployment constraints in domain 1304. Control 1316 can also be configured to identify constraints that have been authorized by the global deployment orchestrator 1306.

[0195] 6. Other matters; extension

[0196] Unless otherwise defined, all terms (including technical and scientific terms) shall be given the meanings that are common and customary to those skilled in the art, and are not limited to special or customary meanings, unless otherwise expressly defined herein.

[0197] This application may include references to certain trademarks. While the use of trademarks is permitted in a patent application, the exclusivity of the trademark should be respected, and every effort should be made to prevent the trademark from being used in any way that may adversely affect its validity.

[0198] The embodiments are directed to a system having one or more devices, which include a hardware processor and are configured to perform any of the operations described herein and / or recited in any of the following claims.

[0199] In an embodiment, one or more non-transitory computer-readable storage media include instructions that, when executed by one or more hardware processors, cause to perform any operation described herein and / or recited in any of the claims.

[0200] In an embodiment, a method includes the operations described herein and / or recited in any of the claims, the method being performed by at least one device including a hardware processor.

[0201] Any combination of the features and functions described herein may be used according to one or more embodiments. In the foregoing description, embodiments have been described with reference to numerous specific details that may vary depending on the implementation. Therefore, the description and drawings should be considered illustrative rather than restrictive. The sole and exclusive measure of the scope of this disclosure, and what the applicant intends to define as the scope of this disclosure, is the literal and equivalent scope of the set of claims issued in this application, in the specific form of such claims, including any subsequent corrections.

Claims

1. A method comprising: Transmit a first deployment request to the first cloud partition among multiple cloud partitions in the cloud environment to deploy the first resource to the first cloud partition; Receive a message from the first cloud partition instructing the approval of the first deployment request, wherein the approval of the first deployment request depends on an assessment of the first deployment request based on the state of the cloud environment; In response to receiving a message instructing approval of the first deployment request, the first resource is deployed to the first cloud partition; The method is performed by at least one device including a hardware processor.

2. The method as described in claim 1: The status of the cloud environment includes the version number of the second resource deployed in the first cloud partition; The first deployment request, based on the state assessment of the cloud environment, includes determining whether the version number of the second resource meets the minimum version requirements.

3. The method as described in claim 1: The cloud environment status includes the resource deployment status in the second cloud partition among the multiple cloud partitions; The first deployment request, based on the state assessment of the cloud environment, includes determining that the first resource has been deployed to the second cloud partition.

4. The method as described in claim 1: The first deployment request targets the first resource's version number; The status of the cloud environment includes the current version number of the first resource that has been deployed in the first cloud partition; The first deployment request, based on the state assessment of the cloud environment, includes determining whether the difference between the target version number and the current version number is less than a threshold.

5. The method as described in claim 1: The transmission of the first deployment request is performed by a global deployment orchestrator targeting the plurality of cloud partitions; The first deployment request, based on the state assessment of the cloud environment, is executed by a partition-specific deployment orchestrator for the first cloud partition.

6. The method of claim 5, wherein the first deployment request includes a snapshot of at least a portion of the state of the cloud environment.

7. The method of claim 5, wherein the state of the cloud environment includes the state of the first cloud partition.

8. The method of claim 5, further comprising: Before transmitting the first deployment request: the global deployment orchestrator determines that the first deployment request satisfies one or more global deployment constraints applicable to the plurality of cloud partitions.

9. The method of claim 1, further comprising: Continuously monitor the cloud environment to determine its status.

10. The method of claim 1, wherein assessing the first deployment request based on the state of the cloud environment includes: Determine whether the first deployment request meets the partition-specific deployment constraints associated with the first cloud partition.

11. The method of claim 10, wherein partition-specific deployment constraints limit one or more of the time, duration, or frequency of deployments permitted in the first cloud partition.

12. The method of claim 10, wherein the first deployment request includes status information associated with at least a second cloud partition among the plurality of cloud partitions.

13. The method of claim 1, further comprising: A second deployment request is transmitted to the second cloud partition among the plurality of cloud partitions in the cloud environment to deploy the second resource to the second cloud partition; Receive a message from the second cloud partition indicating that the second deployment request is rejected; In response to receiving a message indicating that the second deployment request should be rejected: Avoid deploying the second resource to the second cloud partition.

14. The method of claim 13, further comprising: After avoiding deploying the second resource to the second cloud partition: transmit a third deployment request to the second cloud partition to deploy the second resource to the second cloud partition; Receive a message from the second cloud partition instructing the approval of the second deployment request; In response to receiving a message instructing approval of a second deployment request, the second resource is deployed to the second cloud partition.

15. The method of claim 1, wherein the plurality of cloud partitions comprises one or more of the following: (a) a plurality of geographically distinct regions of the cloud environment; or (b) a logical grouping of the plurality of geographically distinct regions of the cloud environment.

16. The method as described in claim 1: The cloud environment is operated by a cloud service provider (CSP). The first cloud partition is operated at least in part by a second entity that is different from the CSP.

17. The method of claim 16, wherein the second entity provides access to the first cloud partition to a third entity that is different from both the CSP and the second entity.

18. The method of claim 17, wherein the second entity is a customer of the CSP, and the third entity is a customer of the second entity.

19. One or more non-transitory computer-readable media storing instructions that, when executed by one or more hardware processors, cause to perform the operation as described in any one of claims 1-18.

20. A system comprising: At least one device including a hardware processor; and The system is configured to perform the operations described in any one of claims 1-18.

21. A system comprising components for performing the operations as described in any one of claims 1-18.

Images