Autonomous terraforming on cloud infrastructure

Abstract

This document provides a method for defining cloud services in Infrastructure as Code (IaC) tools and a computer-readable medium for doing so. [Solution] The method for defining a cloud service in an IaC tool is designed to dynamically determine the number of clusters, availability domains, compute nodes, and load balancers at runtime. These values ​​are then used to determine the number of subnet levels required for classless inter-domain routing slice operations to generate subnets for each compute node and load balancer node in the service. Since the IaC language does not provide nested loop control structures, labels for each subnet are generated and assigned using the Cartesian product of label elements.

Description

[Technical Field]

[0001] Cross-references to related applications This application claims the interests and priority of U.S. Patent Application No. 16 / 730,656, “AUTONOMOUS TERRAFORMING ON CLOUD INFRASTRUCTURES,” filed on 30 December 2019, which is incorporated herein by reference in its entirety. [Background technology]

[0002] background Cloud computing offers both Infrastructure as a Service (IaaS) and Platform as a Service (PaaS), enabling customers to securely run mission-critical enterprise applications and database workloads on shared infrastructure. These cloud computing infrastructures can host a variety of applications, and industries are rapidly moving towards microservices-based frameworks to deliver cloud-based technologies. For example, microservices projects utilize Kubernetes. In a cluster of containerized projects such as Engine (Kubernetes Engine), Even if it's shut down, all of these are hosted and accessible via a shared cloud infrastructure that ensures data security among tenants.

[0003] When a customer begins the process of migrating their computing infrastructure to the cloud, the cloud infrastructure can create a new tenancy for that customer. Once the tenancy is created, the customer may use a console web application to create the necessary environment build blocks to achieve the desired functionality of any microservice project hosted within the cloud environment. With existing technologies, a new tenant can manually create these build blocks by selecting existing technologies and adding them to their cloud footprint. Alternatively, some systems allow tenants to orchestrate the build blocks for their cloud footprint using automated scripting techniques. Automated scripts may be written in any “Infrastructure as Code” software tool that allows users to define and provision data center infrastructure using a high-level configuration language. Scripts written in these languages ​​can be executed to generate build blocks for the cloud footprint, significantly simplifying the provisioning process. However, because these scripting languages ​​are intended to represent instances of cloud hardware / software, they typically do not provide the features and code structures that can be used to write robust and reusable code across many different applications. [Overview of the project] [Problems that the invention aims to solve]

[0004] overview Infrastructure as Code (IaC) tools are useful for declaring resources that will be provisioned as part of a service in cloud infrastructure. However, IaC tools use a declarative language to specify the resources to be provisioned, rather than an imperative language that executes a set of commands and control structures. Therefore, an IaC script can define specific resources and structures that should be provisioned for a particular service, and the script can be significantly modified. Furthermore, it is not easily reused for different services or provisioning tasks. [Means for solving the problem]

[0005] The embodiments described herein present a process for defining a cloud service in an IaC tool, which is designed to dynamically determine the number of clusters, availability domains, compute nodes, and / or load balancers at runtime. These values ​​are then used to determine the number of subnet levels required for Classless Inter-Domain Routing (CIDR) slicing operations to generate subnets for each of the compute nodes and load balancer nodes in the service. Since the IaC language does not provide nested loop control structures, labels for each subnet can be generated and assigned using the Cartesian product of label elements. These labels can be modified by a count variable that is incremented each time a resource is duplicated in the script to simulate the effect of a nested loop structure.

[0006] The script may receive or determine a baseline set of parameters that may change with each service being provisioned. These parameters may be received or determined at runtime, when the script is executed, and may include the number of clusters to be provisioned as part of the service, the number of availability domains in each cluster, the number of compute nodes in each availability domain, and the number of load balancers in each cluster. The script may then determine a number of CIDR levels to guarantee an appropriate number of subnets for the compute nodes and / or load balancer nodes to be provisioned, along with extra subnets to provide room for future expansion. Instead of calculating these values ​​and hardcoding them into the script for each different provisioning task, these values ​​may be provided and calculated at runtime so that the script can be reused for any similar provisioning task without modification.

[0007] Assigning the calculated subnets and labels to each of the declared resources would typically require a nested loop control structure within the code. However, since IaC tools typically do not provide such a structure, the script can simulate the behavior of a nested loop by calculating the Cartesian product of vectors containing the components of each subnet label. For example, subnet labels generated using availability domain and cluster labels can be generated by cross-multiplying these label vectors together to produce a list of all combinations of availability domains and clusters. These labels may be further modified to include counter values ​​used to replay the resources declared within the script.

[0008] Brief explanation of the drawing A further understanding of the nature and merits of various embodiments can be achieved by referring to the remainder of the specification and the drawings, where similar reference numbers are used to refer to similar components through several drawings. In some cases, sublabels are associated with reference numbers to indicate one of several similar components. When a reference number is referred to without specifying an existing sublabel, it is intended to refer to all such several similar components. [Brief explanation of the drawing]

[0009] [Figure 1] This figure shows the architectural design of structural tenancy and cloud environments according to several embodiments. [Figure 2] This figure shows how a single network address can be "sliced" into multiple different networks, according to several embodiments. [Figure 3A] This figure shows a first input specifying the number of clusters that may be determined and / or received by an algorithm, according to some embodiments. [Figure 3B] This figure shows a second input specifying the number of availability domains that may be determined and / or received by an algorithm, according to some embodiments. [Figure 3C] This figure shows a third input specifying the number of worker subnets that may be determined and / or received by an algorithm, according to some embodiments. [Figure 3D] This figure shows a fourth input specifying the number of load balancer subnets, which may be determined and / or received by an algorithm, according to some embodiments. [Figure 4] This figure shows how the Cartesian product can be used to form a single list of labels from multiple different variables, according to several embodiments. [Figure 5] This figure shows how the Cartesian product can be calculated using modular arithmetic in several embodiments. [Figure 6A] FIG. 1 shows a first part of a flowchart of a method for provisioning services in a cloud infrastructure using accompanying pseudo-code, according to some embodiments. [Figure 6B] FIG. 4 shows a continuation of a flowchart of a method for provisioning services in a cloud infrastructure, according to some embodiments. [Figure 6C] FIG. 7 shows a continuation of a flowchart of a method for provisioning services in a cloud infrastructure, according to some embodiments. [Figure 6D] FIG. 10 shows a continuation of a flowchart of a method for provisioning services in a cloud infrastructure, according to some embodiments. [Figure 6E] FIG. 13 shows the completion of a flowchart of a method for provisioning services in a cloud infrastructure, according to some embodiments. [Figure 7] FIG. 16 is a flowchart of a method for provisioning variable subnets with corresponding labels at runtime, which may be implemented in any IaC language, according to some embodiments. [Figure 8] FIG. 19 is a simplified block diagram of a distributed system for implementing some embodiments. [Figure 9] FIG. 22 is a simplified block diagram of the components of a system environment in which services provided by the components of the system of the embodiments may be provided as cloud services. [Figure 10] FIG. 25 shows an exemplary computer system in which various embodiments may be implemented. DETAILED DESCRIPTION

[0010] Detailed Description A cloud computing environment may include a number of individual building blocks each representing its own hardware and / or software technology. Tenants of a cloud computing infrastructure may wish to design very complex cloud-based applications that can handle large amounts of web traffic and store large amounts of information for a number of individual users. These cloud-based applications can represent the vast complexity of interactions between different cloud technologies. However, to simplify the process of designing and provisioning complex applications, many modern cloud-based applications can be designed as microservices applications. Microservices applications use individual simple microservices as building blocks for complex technologies. When designing large-scale applications, a tenant may assemble many individual microservices building blocks to form the overall functionality of the application. This process can be simplified such that large complex design tasks can be assembled by simple building blocks.

[0011] When a cloud-based application is designed, the individual hardware / software components of the design may be provisioned within the cloud environment, and the application may be made available to users. Cloud provisioning is the allocation of resources and services from a cloud provider to a specific tenant. This disclosure focuses on the ability of a tenant or provider to automatically provision applications in a cloud service. For example, in this model, a tenant may select and assemble the building blocks of their cloud application via a web interface or portal. This provisioning model gives the user direct control over how the application is built and organized within the cloud environment. This typically reduces the need for interaction between the cloud provider support service and the customer. Instead, the customer can provision the cloud application themselves, and the cloud environment can then automatically receive instructions from the customer and provision the necessary resources without requiring extensive administrative interaction with the process.

[0012] Despite the use of microservices and simplified build blocks during the provisioning process, the complexity of some applications can still be very challenging for users. Therefore, some embodiments may use scripting languages ​​to implement an "Infrastructure as Code" (IaC) environment for managing and provisioning cloud applications. The IaC language provisions computer data centers through machine-readable definition files, instead of, or in addition to, physical hardware configuration or interactive configuration tools. The IaC language can manage both bare-metal servers and virtual machines associated with configuration resources. This enables users to model infrastructure using code, which then gives them the ability to design, realize, and deploy application infrastructure using software best practices and reusable coding techniques. IaC can be incorporated by cloud computing architectures, which can then assemble build blocks declared by code and arrange the build blocks in a connected manner based on code definitions.

[0013] Despite the clear advantages of using IaC technology, various limitations inherent to IaC languages ​​exist. Because IaC languages ​​are intended to represent the actual parts of cloud infrastructure and their interoperability, they are typically designed to lack control structures common in true software programming languages. These inherent limitations in IaC languages ​​make it difficult for simple scripts to scale across multiple infrastructures. These languages ​​are not designed to adapt to new changes that may be required by existing, growing infrastructure. For example, classless inter-domain routing (CIDR) slicing required values ​​that were manually pre-calculated and hardcoded into scripts when the cloud infrastructure was built. Such techniques required human intervention to recalculate changes, modify the code, and re-run scripts before the code could be reused for new applications. This process is time-consuming, requires human intervention, and increases the chance of introducing flaws into the provisioning process. In contrast to the new algorithms described in this disclosure, other techniques to make IaC languages ​​more general and reusable each have drawbacks that limit their effectiveness or reusability. For example, most existing solutions simply address this issue by writing redundant code.

[0014] The embodiments described herein demonstrate novel coding techniques that can be used to overcome existing limitations of the IaC language. For example, they enable the generation of complete infrastructure blueprints of any size that do not require any manual calculations or changes to generic code. A new algorithm for CIDR slicing has been developed using multiple iterators. Any change in the input variables automatically triggers a recalculation using the new algorithm, and the script may also trigger a realignment of the cloud infrastructure as needed. This type of programming and execution was not possible with previous industrial solutions to this technical problem. This solution uses existing IaC language capabilities along with new coding techniques to augment the language capabilities beyond their original intended purpose.

[0015] Figure 1 illustrates the architectural design of a structural tenancy and cloud environment in several embodiments. Architecture 100 provides an example of an architecture that can be specified using the IaC language. When a script written in the IaC language is received and executed by the cloud infrastructure, architecture 100 can be provisioned according to the infrastructure defined by the IaC language script. Because the IaC script is a formal language that is easily interpreted and can be matched to build-block technologies in the cloud infrastructure, architecture 100 can be automatically provisioned by the cloud infrastructure without significant human intervention. The IaC script can lay out individual technologies as an architecture, and the cloud infrastructure can treat the IaC script as a blueprint for provisioning each of the technologies declared in the script.

[0016] Architecture 100 in Figure 1 is provided merely as an example and is not intended to be limiting. The components in Architecture 100 may provide examples of technology building blocks that can be declared by IaC scripts. Although a discrete number of technologies are shown in Figure 1, it should be understood that the algorithms described below enable each of the technologies in Figure 1 to be scaled in any number and any combination, without limitation. Architecture 100 provides a description of how a particular technology (e.g., availability domains, load balancers, clusters, worker subnets, etc.) can be scaled by the embodiments described herein.

[0017] Architecture 100 includes multiple availability domains 102. Each availability domain 102 may contain a portion of the microservices that make up the entire application. Availability domains 102 may include redundant services to maintain availability if one of the availability domains 102 becomes unavailable. Availability domains 102 may be isolated from each other, fault-tolerant, and have a very low probability of simultaneous failure. In addition, availability domains 102 may correspond to different assemblies of physical hardware located in different geographical locations, such as different regions. Alternatively, availability domains within the same region may be interconnected by a low-latency, high-bandwidth network to build a system replicated across multiple regions for both high availability and disaster recovery.

[0018] Several different technologies may be part of each of the availability domains 102. For example, each of the availability domains 102 may contain one or more subnets 104. Each subnet 104 may be assigned a specific subnet address. Determining the exact number of subnets 104 may be determined in order to properly slice the top-level network address. This procedure, known as CIDR slicing, is described in more detail below. Each subnet 104 may use its assigned subnet address for inbound / outbound network access and communication between subnets 104.

[0019] Subnet 104 may host resources found in each cluster. Many different resources may be hosted within subnet 104, but two specific resources are shown as examples in Figure 1. Some subnets 104 may contain compute nodes 108. Compute nodes 108 are sometimes called worker nodes. Compute nodes 108 perform actions associated with their corresponding worker subnets. For example, the network address of worker subnet 104a may be used to invoke services performed by compute node 108a. Some compute nodes 108 in a cluster may be designated as node pool 106. Subnet 104 containing compute nodes 108 may be referred to as worker subnets.

[0020] In addition to hosting the compute node 108, subnet 104 may alternatively include a load balancer 112. The load balancer 112 may receive and distribute traffic across different worker subnets in each of the availability domains 102. These subnets may be called load balancer subnets.

[0021] Each of the subnets 104 may also be associated with a security node 114. The security node 114 may manage the security policy for each of the worker subnets 104. In some embodiments, the security node 114 may use an access list 110 to manage which systems and / or users may have access to the corresponding security node 114.

[0022] Finally, the architecture 100 shown in Figure 1 can be contained within a single cluster. To ensure high availability of the service, multiple clusters identical to the one shown in Figure 1 may be formed.

[0023] Architecture 100 may be provisioned by providing an IaC script to the cloud infrastructure. The architecture may then interpret the technologies enumerated in the script and automatically provision those technologies. Any IaC language such as Pulumi, Chef, Otter, Puppet SaltStack, CFEngine, DSC, or Paco may be used. As an example, the embodiments described herein may use the Terraform tool 150 provided by Hashicorp. Each of these tools may use a declarative language to define the various technologies to be provisioned as part of the cloud infrastructure. As declarative languages, these IaC tools generally do not support advanced control structures that may be used in other languages. For example, the Terraform tool 150 is... It also does not support looping mechanisms of that type. Instead, it allows you to access a resource a defined number of times using a count meta-argument for that resource. It may be instantiated. This prevents the tool from using any form of nested loops that may be useful for making assignments or calculations using multiple variables. Furthermore, the Terraform tool may not know the number of variables at the time the code is written. It does not provide a simple way to perform CIDR slicing on subnets.

[0024] Figure 2 illustrates how a single network address 202 can be “sliced” into multiple different networks in several embodiments. CIDR slicing uses address aggregation to transform a single network address into multiple addressable networks. As described above in relation to Figure 1, each of the subnets 104 may require its own network address, existing as a subnet of the top-level network address 202. The concept of CIDR slicing subdivides the top-level network address 202 into several subnetworks based on the number of networks required. Prior to this disclosure, deriving multiple subnet addresses from the top-level network address 202 was a static operation. In other words, the user could derive the required The number of subnets will be determined, and then the top-level network address 202 will be divided accordingly. These values ​​will be used in the Terraform script when the script is written and the service is designed for provisioning on the cloud infrastructure. The values ​​were hardcoded. While future scripts could build on existing ones, the hardcoded values ​​had to be constantly modified in new scripts to scale the number of subnets. This required error-prone human intervention and reduced the overall usefulness of the code. It also increased the number of scripts that needed to be stored, as each script could only be used for its corresponding specific provisioning project.

[0025] Embodiments described herein solve these and other technical problems by using novel algorithms and methods that conceptualize CIDR slicing as a binary tree that can be dynamically upsized and / or downsized as the number of availability domains, clusters, worker subnets, and / or load balancers to be provisioned by the user changes. Figure 2 shows how the top-level network address 202 can be sliced ​​into a variable number of subnets based on level designation. For example, if the provisioned service requires two subnets, a level 1 (211) slicing operation may be performed. If up to four subnets are used, a level 2 (212) slicing operation may be performed.

[0026] Various levels may be used based on the minimum number of subnets required. Levels 211, 212, 213, and 214 shown in Figure 2 are based on a base-2 exponential function. For example, level 3 (213) is based on 2 3 = corresponds to 8 subnets. Similarly, level 4 (214) corresponds to 2 4 This corresponds to 16 subnets. To determine the number of levels required for the slicing operation, the Terraform script determines the minimum number of subnets required, and then... Then, you can select a level that provides at least that number of subnets. For example, if six subnets are required for a provisioned service, the script can dynamically determine that a level 3 (213) slice operation should be performed on top-level network address 202. This ensures that the required number of subnets are provided, along with any additional subnets that may be used for future expansion.

[0027] The embodiments described herein allow a script to dynamically determine the level of a CIDR slice operation when it is executed. As will be described in detail below, this does not require any hardcoded values ​​to be included in the script. Rather, the script can calculate the level required for the slice operation at runtime using a desired number of clusters, availability domains, worker subnets, load balancers, etc. In addition, as the project scales up and down, subnet addresses can be recalculated simply by, for example, changing the number of availability domains in the project. The script can then automatically recalculate the number of levels used for the slice operation.

[0028] These techniques not only make the code scalable over the lifespan of a single project, but also allow the code to be reused for future projects of any size. The code may accept values ​​specified by the user during the service design process. For example, the user might specify the number of load balancers or the number of availability domains that should be part of the project. However, the user does not specify the level of CIDR slicing operation that should be used to generate the appropriate number of subnet addresses. This is a low-level detail that the user is unlikely to understand. Subnet addresses should instead be hardcoded into the Terraform script, and the code This makes it unusable for any project without the same specific architecture. Embodiments described herein allow the user to dynamically change the input to the script, thereby moving up and down between the different levels 211, 212, 213, and 214 illustrated in Figure 2. In short, these embodiments achieve a form of automated CIDR slicing that was previously unavailable.

[0029] The following diagrams show different inputs that may be determined and / or provided by a script. Next, this disclosure describes an algorithm for determining the slicing level and automatically generating the corresponding subnets. Figure 3A shows a first input specifying the number of clusters 302, which may be determined and / or received by the algorithm, according to several embodiments. As described above in Figure 1, each cluster 302 may consist of one or more availability domains, each of which may contain any number of worker subnets and / or load balancers. The number of clusters 302 may be referred to herein using the variable c as an abbreviation in the equations described below. The number of clusters 302 may also be explicitly specified by the user as input to the script. Alternatively or additionally, the number of clusters 302 may also be determined by other values ​​and / or behaviors provided by the user. For example, the user may select a given configuration from several configurations available for their project. These configurations may correspond to a given number of clusters, and the script may use a lookup table, database, etc., to translate the configuration into the number of clusters 302.

[0030] Figure 3B shows a second input specifying the number of availability domains 304, which may be determined and / or received by an algorithm in several embodiments. As described above in Figure 1, each of the availability domains 102 may be provided to provide high availability for the provisioned service. Functionality may be replicated across the availability domains 102 so that if one availability domain 202b becomes unavailable, the service may still be provided through the other availability domains 202a, 202c. The number of availability domains 304 may be expressed using the variable a as a shorthand in the equation described below. The number of availability domains 304 may be provided by the user as input to the script. For example, the user may specify three availability domains 102 as input. Alternatively, the user may select an availability level (e.g., high, medium, low) for the service. The script may then look up or calculate the number of availability domains 102 corresponding to the availability level selected for that particular project. Note that the number of availability domains 304 does not need to specify the total number of availability domains in the entire project. Alternatively, the number of availability domains, 304, may be provided on a cluster basis. In other words, the number of availability domains, 304, may represent the number of availability domains within each cluster.

[0031] For example, a configuration may be selected by the user from several predefined configurations. These configurations may have values ​​entered at runtime, as per the Terraform script. This may be presented to the user as part of the line-by-line execution. Each configuration may be associated with a service level agreement (SLA) that specifies the level of availability required for the service. The system may derive the number of availability domains that may be required to meet the level of availability required in the SLA. Note that this same procedure may be used to determine the number of clusters, the number of compute nodes, and / or the number of load balancers.

[0032] In some embodiments, these values ​​may be dynamically received as input to a script at runtime. This defines different services with different characteristics that will require different numbers of clusters, availability domains, compute nodes, and / or load balancers. This allows the script to be reused for various purposes. For example, after provisioning a first service using a script, a second different service may be provisioned using the same script. Different services with different subnet levels, different top-level network addresses, and / or other different topologies may be provisioned by providing different numbers of clusters, availability domains, compute nodes, and / or load balancers as input to the script at runtime. This allows the same script to be reused to provision different services without requiring editing. Previously, if any of these parameters were different, a different script would have been required for each service being provisioned.

[0033] In some embodiments, these values ​​may be changed dynamically after the provisioning process has taken place. For example, a script may be executed to provision the service by receiving runtime values. After the service is delivered, the customer may want to change the level of availability, cluster location / number, and / or any other parameters. Instead of generating a new script, the same script can be run a second time, receiving the changes to the input, recalculating the subnet-level numbers, and regenerating subnets based on the new number of availability domains, clusters, compute nodes, load balancers, etc.

[0034] Figure 3C shows a third input specifying the number of worker subnets 104, which may be determined and / or received by an algorithm in several embodiments. As described above in Figure 1, each of the worker subnets 104 may be assigned one of the subnet addresses from the CIDR slice operation described in Figure 2. Each worker subnet 104 may contain compute nodes 108. The number of worker subnets 306 may be expressed using the variable w as an abbreviation in the equation described below. Similar to the number of availability domains 304, the number of worker subnets 306 may be provided on an availability domain basis. In other words, the number of worker subnets 306 may represent the number of worker subnets in each availability domain. For example, the number of worker subnets 306 per availability domain in Figure 3C is one worker subnet. The number of worker subnets 306 is received and / or determined by a Terraform script. The number of worker subnets per availability domain may be specified. For example, the user may provide 306 worker subnets per availability domain as input to the script. Alternatively, the user may select the functions to be performed in the availability domain, and the script may automatically determine which worker subnets should be included in the design, thereby determining the number of worker subnets 306.

[0035] Figure 3D shows a fourth input specifying the number of load balancer subnets 308, which may be determined and / or received by an algorithm in several embodiments. As described above in Figure 1, each load balancer subnet may contain a load balancer 112 that manages traffic and routes it to worker subnets. The number of load balancer subnets may be expressed using the variable b as a shorthand expression in the formula described below. This number may specify the number of load balancer subnets per cluster. Typically, the user does not need to organize the load balancer subnets into a specific availability domain, but instead simply specifies the total number of load balancers required in each cluster. This number may be specified as input to a script. Alternatively, the number of load balancer subnets 308 may be determined by a script at runtime. For example, the user may specify one or more predefined configurations for a service, and the script may derive the required number of load balancers from those configurations.

[0036] Based on these inputs, the following algorithm may be used to determine the total number of subnets to be used for the provisioned services. In one step, the algorithm may calculate the total number of worker subnets to be provisioned. The total number of worker subnets may be calculated based on the total number of clusters (302), the total number of availability domains per cluster (304), and the total number of worker subnets per availability domain (306). For example, the total number of worker subnets may be calculated using the following formula.

[0037]

number

[0038] In Equation 1, w t This represents the total number of worker subnets provisioned across the entire service.

[0039] In another step, the algorithm may calculate the total number of load balancer subnets that should be provisioned. The total number of load balancer subnets may be calculated based on the total number of clusters (302) and the total number of load balancers per cluster (308). For example, the total number of load balancer subnets may be calculated using the following formula:

[0040]

number

[0041] In equation 2, b t This represents the total number of load balancer subnets that should be provisioned across the entire service.

[0042] In another step, the algorithm may calculate the total number of subnets to be provisioned in the project. The total number of subnets may be calculated based on the total number of worker subnets and the total number of load balancer subnets. For example, the total number of subnets may be calculated using the following formula:

[0043]

number

[0044] In Equation 3, s represents the total number of subnets allocated to the entire service. Next, the algorithm may calculate the level to which the top-level network address should be sliced. As described above in relation to Figure 2, this level may correspond to a base-2 exponential function that determines the number of subnet addresses derived from the top-level network address. In some embodiments, raising the base-2 exponent to the exponent that should correspond to the number of calculated slice levels gives the total number of subnets to be allocated in the service (e.g., 2^(number of subnet levels): a power with base-2 and exponent (number of subnet levels)). The above may also be the case. To determine the level, the base-2 logarithm of the total number of subnets can be calculated, and then the result of the base-2 logarithm can be subjected to a ceiling operation (e.g., rounding to the next highest integer) to determine the level. For example, in a particular embodiment, the following formula may be used to determine the number of levels:

[0045]

number

[0046] Since n ∈ N, we can apply the sealing operation to Equation 7 to calculate the final level n for the slicing operation.

[0047]

number

[0048] Here, a slice operation may be performed on the topmost network address using level number n. In some embodiments, the algorithm slices the network address 2 n By dividing the network into non-overlapping subnets, the appropriate IP addresses for each subnet may be calculated manually. Depending on the specific IaC tool used to define the architecture, there may be a predefined function that performs this calculation automatically. For example, the Terraform tool may perform this mathematical calculation for each subnet. This provides the cidrsubnet(prefix, newbits, netnum) function that can be used. For numbers, the following inputs may be used. The prefix is ​​a predetermined value derived from the top-level network address. • netnum is a value ∈[0,2] representing the index of one of the subnet addresses at the n level from Figure 2. n -1] newbits will check if there are enough CIDR blocks to allocate to each subnet. This is a dynamically calculated value of n, which represents the minimum value guaranteed.

[0049] At this stage, available subnets can be allocated to each of the resources described in the architecture of Figure 1. In the Terraform tool, this is a resource This can be achieved using a single count meta-argument. For example, by inserting the line `count=num_worker_subnets` after the declaration of that resource in your Terraform code. Then, that resource can be replicated once per worker subnet. It should be emphasized that IaC languages ​​and tools typically provide a limited looping mechanism that allows only single-level resources to be replicated.

[0050] However, in practice, it is also useful to generate labels and / or assign them to each of the resources based on various combinations of variables. For example, in some embodiments, each subnet may be labeled according to its cluster and availability domain combination. When dealing with multiple variables that need to be iterated over to generate a label for each resource, multiple levels of looping may be required. However, as mentioned above, the IaC language does not provide control over such nested looping structures as a native feature of the language. Therefore, in order to assign labels based on multiple variables (e.g., cluster, availability domain, etc.), each such label had to be calculated manually and hardcoded into the script. As mentioned above, hardcoding values ​​into the script limits the reusability and scalability of the design and makes it difficult to change the design. Whenever it happens, human intervention is required.

[0051] Embodiments described herein solve this problem by combining various label components into a label vector and performing modular arithmetic to generate a Cartesian product. The Cartesian product is then mimicked by a count meta-argument within the Terraform tool. Loop features may be used.

[0052] Figure 4 illustrates how the Cartesian product can be used to form a single list of labels from several different variables, in several embodiments. A first vector containing a list of cluster labels 402 may include clusters referred to as production, development, staging, quality assurance, etc. A second vector containing a list of availability domains 404 may include domains referred to by numerical identifiers (e.g., AD-1, AD-2, etc.). Previously, a script might have statically defined all pairs that formed a single hardcoded label vector from the combination of these two vectors of labels, and then applied that hardcoded single label vector to each subnet. However, this limits the reusability of the script. This was necessary because traditional programming languages ​​provide nested loops so that clusters 402 can be iterated over in an outer loop and availability domains 404 can be iterated over in an inner loop, but IaC tools typically do not provide such control structures.

[0053] Some embodiments approximate a nested loop control structure by using modular arithmetic, as shown in Figure 4. This involves a script receiving any number of clusters, availability domains, or other labels as input, and then within the Terraform tool. The count meta-argument of allows for the dynamic generation of a single vector of labels 406 that can be applied. The Cartesian product may be generated using the algorithm described below so that it contains all ordered pairs for each of the two input label vectors. It should be emphasized that the two vectors of labels are used here only as examples. Other embodiments may use more than two vector labels, and the Cartesian product may be computed for each combination of labels to form a single label vector that can be applied without nested loops.

[0054] In Figure 4, the Cartesian product produced a vector of labels 406 containing each of the unique pairs of labels from cluster 402 and availability domain 404. Figure 5 shows how the Cartesian product can be calculated using modular arithmetic in several embodiments. In this example, the Cartesian product is represented as a grid with two-dimensional coordinates. The two-dimensional coordinates represent the indices that will be used in nested loops. For example, the production cluster might loop at index 0, the development cluster at index 1, the staging cluster at index 2, and so on. In the case of inner loops, each availability domain would loop at an index incrementing from 0 to 2.

[0055] However, instead of using this nested looping process, each pair obtained from the Cartesian product can be indexed using a single index in the resulting label vector 406 shown in Figure 4. The single index is related to the nested index using modular arithmetic. For example, the index in the outer loop representing the cluster labels is related to the index by the following equation:

[0056]

number

[0057] The variable 'a' corresponds to the number of values ​​in the input vector, which in this case is 3, and the index c This corresponds to the cluster index. Similarly, the index of the inner loop representing the availability domain label may be associated with the index by the following expression.

[0058]

number

[0059] index ad This corresponds to the availability domain index. Figures 6A to 6E show detailed examples of Terraform scripts that implement the algorithm described above. This flowchart and pseudocode are provided as examples and are not intended to limit the scope of the algorithm. The algorithm described above may be implemented using any IaC language with respect to this disclosure.

[0060] Figure 6A shows the first part of a flowchart of a method for provisioning a service in a cloud infrastructure, along with accompanying pseudocode, according to several embodiments. The method may include receiving state variables and collecting data from the cloud infrastructure (602). As described above, some embodiments may receive each of the variables defining the resources to be provisioned as input to the script. In this example, this includes the number of availability domains, the number of worker subnets in each domain, and the number of load balancer subnets per cluster. In this example, it may be assumed that the number of worker subnets per availability domain is 1, and the number of load balancer subnets per cluster is 2. Pseudocode 601 for receiving these values ​​may include assigning each of these values ​​to a variable. Pseudocode 601 also includes FastConnect User It also uses tility, which is a network connection that serves as an alternative to using the public internet to connect to cloud infrastructure and other online services.

[0061] Pseudocode 601 also includes an implementation of the algorithm described above to define the number of CIDR levels. For example, equation 8 above is implemented in the last line of pseudocode 601:

[0062]

number

[0063] This line of pseudocode calculates the total number of subnets required for the load balancer and worker nodes. This total is provided as input to a base-2 logarithmic function, and the result is provided to a ceiling function to determine the minimum number of CIDR levels for the service.

[0064] This method may also include provisioning a gateway for the service (604). Pseudocode 603 provides an example of how to define a service gateway, internet gateway, NAT gateway, etc. As mentioned above, the syntax of pseudocode 603 may be specific to the Terraform tool, and therefore, This can change when working with other tools.

[0065] Figure 6B shows a continuation of the flowchart for a method of provisioning a service in a cloud infrastructure according to several embodiments. This method may further include configuring routing tables and security rules for the service (606). Pseudocode 605 shows an example for declaring a default routing table and load balancer security list. Note that several additional security and / or routing resources may be declared in a real-world project. The example provided in pseudocode 605 is merely representative and may be used as a template for generating other security rules, security lists, and routing tables for different aspects of a service.

[0066] Figure 6C shows a continuation of the flowchart for a method of provisioning services in a cloud infrastructure according to several embodiments. The method may further include determining whether all worker subnets have been provisioned (608). If there are remaining worker subnets to be provisioned, the method may include calculating subnet addresses using the CIDR slicing algorithm described above (610). Each subnet may then be provisioned with the corresponding CIDR block (612). This cycle may be repeated for each worker subnet.

[0067] As is evident in pseudocode 614, the loop structure shown in the flowchart cannot be represented using Terraform pseudocode. Instead, this code is specified It provides a meta-argument that duplicates the resource as many times as it has been declared. For example, the first line of pseudocode after a resource declaration includes:

[0068]

number

[0069] By assigning the number of worker subnets to a count variable, pseudocode 614 can instruct the Terraform tool to create an oci_core_subnet resource for each worker subnet. Pseudocode 614 then uses the above cidrsubnet() along with the count.index variable, which is used to index each subnet at the specified level calculated above.

[0070] Pseudocode 614 also implements the nested loop algorithm described above by calculating the Cartesian product between the cluster name and the availability domain name. These are stored in the availability_domain array, which is indexed by the count.index variable. For example, the following command generates an array of labels for each subnet:

[0071]

number

[0072] Instead of performing a conventional nested loop, this Cartesian product generates a list of labels for the subnets. Finally, the labels can be assigned to the corresponding subnets using the modular arithmetic shown in pseudocode 614 and described in detail above.

[0073] The process shown in Figure 6C divides each worker subnet in the project. This may be performed to make a prediction. Although not specifically shown, the process shown in Figure 6C may be repeated for each load balancer subnet. For example, the top-level network addresses may be sliced ​​based on the levels calculated above, and a Cartesian product may be used to generate a list of labels. However, the netnum parameter for the cidrsubnet() function may be shifted so that the load balancer subnet is assigned to a CIDR block after the worker subnet.

[0074] Figure 6D shows a continuation of the flowchart for a method of provisioning services in a cloud infrastructure according to several embodiments. The method may further include determining whether a sufficient number of clusters have been provisioned based on the number of clusters described above (616). If clusters remain to be provisioned, the method may determine which load balancer subnets are needed (618) and provision the correct load balancers to the clusters (620). As described above, this loop procedure may be modeled in pseudocode 622 to dynamically generate a specified number of clusters using a count variable. The second part of pseudocode 622 may assign load balancer subnets to each of the clusters.

[0075] Figure 6E completes the flowchart of a method for provisioning a service in a cloud infrastructure, according to several embodiments. The method may also include determining whether a sufficient node pool has been provisioned (624). The method continues as long as additional node pools need to be provisioned. This may include using calculations to divide the subnets into node pools for allocation (626). In pseudocode 630, the ChunkList function from the Terraform tool can be used to divide a single list into fixed-size chunks and thus return a list of lists. This method may then include provisioning the node pools along with the corresponding worker subnets (628). In pseudocode 630, each node pool has a specified number of nodes per subnet described in the family map.

[0076] The flowchart and explanation above are specific to the Terraform tool, but the CIDR subscript Algorithms for simulating nested loops to generate and assign subnets and apply labels to subnets may be implemented in any IaC language. Figure 7 shows a flowchart of a method for provisioning a variable number of subnets, with corresponding labels, at runtime, in several embodiments, which may be implemented in any IaC language. The method may include determining the number of clusters to be provisioned as part of a service in the cloud infrastructure (702). The method may also include determining the number of availability domains in each cluster (704), the number of compute nodes in each availability domain (706), and / or the number of load balancers in each cluster (708). In each of these steps, these values ​​may be determined by being received as input to a script, by being stored as constants in the script, and / or by calculation or function in the script. For example, each value may be derived or retrieved based on a predetermined configuration selected by the user. In some embodiments, these values ​​may be determined as described above in relation to Figures 3A-3B, or as shown in Figure 6A.

[0077] The method may also include calculating (710) the number of subnet levels based on the number of clusters, the number of availability domains, the number of compute nodes, and the number of load balancers. The compute nodes may correspond to the worker subnets described above. Each of the load balancers may also be described as a load balancer subnet. The number of subnet levels may be calculated as related to Equation 2 - Equation 8 and as described above in relation to FIG. 6C. The method may further include generating (712) a plurality of subnets based on the number of subnet levels. This step may be performed using the cidrsubnet function described above. Alternatively or additionally, this step may be performed by mathematically subdividing the top-level network address based on the subdivision of 2 n . The method may also include allocating (714) the plurality of subnets to compute nodes and load balancers within the cloud infrastructure. As described above, each of the worker subnets and load balancer subnets may be assigned a label based on the Cartesian product and modular arithmetic algorithms described above.

[0078] It should be understood that the specific steps shown in FIG. 7 provide a specific way of provisioning variable subnets, along with corresponding labels, at runtime, according to various embodiments. Other sequences of steps may also be performed according to alternative embodiments. For example, alternative embodiments of the present invention may perform the steps outlined above in a different order. Further, the individual steps shown in FIG. 7 may include a plurality of sub-steps that may be performed in various sequences appropriate for the individual steps. Additionally, additional steps may be added or removed depending on the specific application. Those skilled in the art will recognize many variations, modifications, and alternatives.

[0079] Each of the methods described herein may be implemented by a computer system. Each step of these methods may be performed automatically by the computer system and / or may involve user inputs / outputs. For example, a user may provide inputs to each step of a method, each of which may respond to a specific output requesting such input, which is generated by the computer system. Each input may be received in response to a corresponding requested output. Furthermore, inputs may be received from the user as a data stream from another computer system, retrieved from a memory location, retrieved over a network, or requested from a web service. Similarly, outputs may be provided to the user as a data stream to another computer system, stored in a memory location, transmitted over a network, or provided to a web service. In short, each step of the methods described herein may be performed by a computer system and may include any number of inputs, outputs, and / or requests to and from the computer system, with or without user involvement. Steps that do not involve the user may be said to be performed automatically by the computer system without human intervention. Accordingly, in light of this disclosure, it will be understood that each step of each method described herein may be modified to include user input and user output, or may be performed automatically by a computer system without human intervention, if any decisions are made by a processor. Furthermore, some embodiments of each of the methods described herein may be implemented as a set of instructions stored on a tangible non-temporary storage medium to form a tangible software product.

[0080] Figure 8 shows a simplified diagram of a distributed system 800 for realizing one embodiment. In the illustrated embodiment, the distributed system 800 includes one or more client computing devices 802, 804, 806, and 808, which are configured to run and operate client applications such as web browsers and ownership clients (e.g., Oracle Forms) over one or more networks 810. A server 812 may be coupled to communicate with the remote client computing devices 802, 804, 806, and 808 over the network 810.

[0081] In various embodiments, the server 812 may be connected to one or more of the system's components. They may be adapted to run one or more services or software applications provided by them. In some embodiments, these services may be provided to users of client computing devices 802, 804, 806, and / or 808 as web-based services or cloud services, or under a software-as-a-service (SaaS) model. Users operating client computing devices 802, 804, 806, and / or 808 may then use one or more client applications to interact with server 812 and utilize the services provided by these components.

[0082] In the configuration shown in the figure, software components 818, 820, and 822 of system 800 are shown as being implemented on server 812. In other embodiments, one or more components of system 800 and / or services provided by these components may be implemented by one or more client computing devices 802, 804, 806, and / or 808. Users operating the client computing devices may then use one or more client applications to access the services provided by these components. These components may be implemented in hardware, firmware, software, or a combination thereof. It should be understood that a variety of different system configurations are possible, which may differ from the distributed system 800. The embodiment shown in the figure is therefore an example of a distributed system for implementing the system of the embodiment and is not intended to be limiting.

[0083] Client computing devices 802, 804, 806 and / or 808 may be portable handheld devices (e.g., iPhone®, cellular phones, iPad®, computing tablets, personal digital assistants (PDAs)) or wearable devices (e.g., Google Glass® head-mounted displays) running software such as Microsoft Windows Mobile® and / or various mobile operating systems such as iOS®, Windows Phone, Android, BlackBerry 10, Palm OS, and enabling the Internet, email, short message service (SMS), Blackberry®, or other communication protocols. Client computing devices may be general-purpose personal computers, including, for example, personal computers and / or laptop computers running various versions of Microsoft Windows®, Apple Macintosh®, and / or Linux® operating systems. Client computing devices may be workstation computers running any of various commercially available UNIX® or UNIX-like operating systems, including, but not limited to, various GNU / Linux® operating systems such as Google Chrome OS. Alternatively, or in addition, client computing devices 802, 804, 806, and 808 may be any other electronic devices capable of communicating via network 810, such as thin client computers, internet-enabled game systems (e.g., Microsoft Xbox game consoles with or without Kinect® gesture input devices), and / or personal messaging devices.

[0084] The illustrated distributed system 800 is shown with four client computing devices, but any number of client computing devices may be supported. Other devices, such as devices with sensors, may interact with the server 812.

[0085] Network 810 within the distributed system 800 uses TCP / IP (Transmission Control Protocol). This includes, but is not limited to, various technologies such as Internet Protocol (ISP), SNA (System Network Architecture), IPX (Internet Packet Switching), and AppleTalk. It may be any type of network well known to those skilled in the art that can support data communication using any of the commercially available protocols. For example, network 810 may be a local area network (LAN), such as one based on Ethernet®, Token Ring, etc. Network 810 may be a wide area network and the Internet. It may be a virtual private network (VPN), intranet, extranet, public switched telephone network (PSTN), infrared network, wireless network (e.g., IEEE 802.11 protocol suite, Bluetooth®, and / or any other wireless network). Virtual networks may include, but are not limited to, networks operating under any of the linear protocols; and / or any combination of these and / or other networks.

[0086] Server 812 may consist of one or more general-purpose computers, dedicated server computers (including, for example, PC (personal computer) servers, UNIX® servers, midrange servers, mainframe computers, rack-mount servers, etc.), server farms, server clusters, or any other suitable configuration and / or combination. In various embodiments, Server 812 may be adapted to run one or more services or software applications as described in the preceding disclosure. For example, Server 812 may correspond to a server for performing the processing described above in accordance with a certain embodiment of this disclosure.

[0087] Server 812 may run an operating system including any of the above, and any server operating system available on the market. Server 812 may also run any of a variety of other server applications and / or middle-tier applications, including HTTP (Hypertext Transfer Protocol) servers, FTP (File Transfer Protocol) servers, CGI (Common Gateway Interface) servers, Java® servers, database servers, etc. Exemplary database servers include, but are not limited to, those available on the market from Oracle, Microsoft, Sybase, IBM® (International Business Machines), and others.

[0088] In some implementations, server 812 may include one or more applications for analyzing and organizing data feeds and / or event updates received from users of client computing devices 802, 804, 806, and 808. For example, data feeds and / or event updates may include, but are not limited to, real-time events related to sensor data applications, financial stock market boards, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, automotive traffic monitoring, etc., and may include Twitter® feeds, Facebook® updates, or real-time updates received from one or more third-party sources and continuous data streams. It is not the case that it is included. Server 812 may also include one or more applications for displaying data feeds and / or real-time events via one or more display devices of client computing devices 802, 804, 806, and 808.

[0089] The distributed system 800 may also include one or more databases 814 and 816. Databases 814 and 816 may reside in various locations. For example, one or more of databases 814 and 816 may be local to server 812 (and / or They may reside on non-temporary storage media (or permanently). Alternatively, databases 814 and 816 may be remote from server 812 and communicate with server 812 via a network-based connection or a dedicated connection. In one embodiment, databases 814 and 816 may reside within a storage area network (SAN). Similarly, any necessary files for performing functions attributable to server 812 may be stored locally and / or remotely on server 812, as appropriate. In one embodiment, databases 814 and 816 may include relational databases, such as those provided by Oracle, adapted to store, update, and retrieve data in response to SQL-formatted commands.

[0090] Figure 9 is a simplified block diagram of one or more components of a system environment 900 that may provide services provided by one or more components of the system of the embodiment as cloud services, according to one embodiment of the present disclosure. In the illustrated embodiment, the system environment 900 includes one or more client computing devices 904, 906, and 908 that may be used by a user to interact with a cloud infrastructure system 902 that provides cloud services. The client computing devices may be configured to run client applications such as a web browser, an intellectual property-owned client application (e.g., Oracle Forms), or any other application, which may be used by a user of the client computing device to interact with the cloud infrastructure system 902 to use services provided by the cloud infrastructure system 902.

[0091] It should be understood that the illustrated cloud infrastructure system 902 may have components other than those shown. Furthermore, the embodiment shown in the figure is merely one example of a cloud infrastructure system that may incorporate embodiments of the present invention. In some other embodiments, the cloud infrastructure system 902 may have more or fewer components than shown in the figure, may combine two or more components, or may have different configurations or arrangements of components.

[0092] Client computing devices 904, 906, and 908 may be similar to the devices described above for 802, 804, 806, and 808.

[0093] An exemplary system environment 900 is shown with three client computing devices, but any number of client computing devices may be supported. Other devices, such as devices with sensors, may interact with the cloud infrastructure system 902.

[0094] Network 910 may facilitate the communication and exchange of data between clients 904, 906, and 908 and the cloud infrastructure system 902. Each network may be any type of network well known to those skilled in the art, capable of supporting data communication using any of the various commercially available protocols, including those described above for network 810.

[0095] The cloud infrastructure system 902 may include one or more computers and / or servers, which may include those described above for server 812.

[0096] In one embodiment, a service provided by a cloud infrastructure system The cloud infrastructure system may include hosting services that are made available on demand to users of the cloud infrastructure system, such as online data storage and backup solutions, web-based email services, hosted office suites and document collaboration services, database processing, and managed technical support services. The services provided by the cloud infrastructure system can be dynamically scaled to meet the needs of its users. A specific instance of a service provided by the cloud infrastructure system is referred to herein as a “service instance.” Generally, any service made available to users from a cloud service provider’s system via a communication network such as the Internet is referred to as a “cloud service.” Typically, in a public cloud environment, the servers and systems that make up the cloud service provider’s system are different from the customer’s own on-premises servers and systems. For example, the cloud service provider’s system may host applications, and users may order and use those applications on demand via a communication network such as the Internet.

[0097] In some examples, services within a computer network cloud infrastructure may include secure computer network access to storage, hosted databases, hosted web servers, software applications, or other services provided to users by the cloud vendor or known in the art. For example, a service may include password-protected access to remote storage on the cloud over the internet. Another example is a service that may include a web service-based hosted relational database and a scripting language middleware engine for private use by networked developers. Yet another example is a service that may include access to an email software application hosted on the cloud vendor's website.

[0098] In one embodiment, the cloud infrastructure system 902 may include a set of application, middleware, and database service offerings that are self-service, subscription-based, flexibly scalable, reliable, highly available, and delivered to customers in a secure manner. An example of such a cloud infrastructure system is the Oracle Public Cloud offered by the Assignee.

[0099] In various embodiments, the cloud infrastructure system 902 may be adapted to automatically provision, manage, and track customer subscriptions to services provided by the cloud infrastructure system 902. The cloud infrastructure system 902 may provide cloud services through different deployment models. For example, the services may be provided under a public cloud model, where the cloud infrastructure system 902 is owned by an organization that sells cloud services (e.g., owned by Oracle), and the services are made available to the general public or different industry companies. As another example, the services may be provided under a private cloud model, where the cloud infrastructure system 902 operates only for a single organization and provides services to one or more entities within that organization. The cloud services may also be provided under a community cloud model, where the cloud infrastructure system 902 and the services provided by the cloud infrastructure system 902 are shared by several organizations within a relevant community. The cloud services may also be provided under a hybrid cloud model, which is a combination of two or more different models.

[0100] In some embodiments, the services provided by the cloud infrastructure system 902 may include one or more services provided under the categories of Software as a Service (SaaS), Platform as a Service (PaaS), Infrastructure as a Service (IaaS), or other categories of services including hybrid services. A customer may order one or more services provided by the cloud infrastructure system 902 through a subscription order. The cloud infrastructure system 902 then performs processing to provide the services in the customer's subscription order.

[0101] In some embodiments, the services provided by the cloud infrastructure system 902 may include, but are not limited to, application services, platform services, and infrastructure services. In some examples, application services may be provided by the cloud infrastructure system via a SaaS platform. The SaaS platform may be configured to provide cloud services that fall under the SaaS category. For example, the SaaS platform may provide the ability to build and deliver a set of on-demand applications on an integrated development and deployment platform. The SaaS platform may manage and control the underlying software and infrastructure for providing the SaaS services. By using the services provided by the SaaS platform, customers can utilize applications that run on the cloud infrastructure system. Customers can obtain application services without having to purchase separate licenses and support. A variety of different SaaS services may be provided. Examples include, but are not limited to, services that provide solutions for sales performance management, enterprise integration, and business flexibility for large organizations.

[0102] In some embodiments, platform services may be provided by a cloud infrastructure system via a PaaS platform. The PaaS platform may be configured to provide cloud services that fall under the PaaS category. Examples of platform services may include, but are not limited to, services that enable an organization (such as Oracle) to integrate existing applications on a shared common architecture, and the ability to build new applications that leverage the shared services provided by the platform. The SaaS platform may manage and control the underlying software and infrastructure for providing SaaS services. Customers can obtain PaaS services provided by the cloud infrastructure system without having to purchase separate licenses and support. Examples of platform services include, but are not limited to, Oracle Java Cloud Service (JCS) and Oracle Database Cloud Service (DBCS).

[0103] By utilizing the services provided by the PaaS platform, customers can adopt programming languages ​​and tools supported by the cloud infrastructure system and also control the deployed services. In some embodiments, the platform services provided by the cloud infrastructure system may include database cloud services, middleware cloud services (e.g., Oracle Fusion Middleware services), and Java cloud services. In one embodiment, the database cloud service may support a shared services deployment model that enables an organization to pool database resources and provide customers with databases as a service in the form of a database cloud. The middleware cloud service may provide a platform for customers to develop and deploy various business applications, and the Java cloud service may enable customers to use the cloud infrastructure. A platform may be provided for deploying Java applications within a structure system.

[0104] Various different infrastructure services may be provided by IaaS platforms within a cloud infrastructure system. Infrastructure services facilitate the management and control of underlying computing resources such as storage, networking, and other basic computing resources for customers using services provided by SaaS and PaaS platforms.

[0105] In one embodiment, the cloud infrastructure system 902 may also include infrastructure resources 930 for providing resources used to provide various services to customers of the cloud infrastructure system. In one embodiment, the infrastructure resources 930 may include a pre-integrated and optimized combination of hardware such as servers, storage, and networking resources to run services provided by the PaaS platform and SaaS platform.

[0106] In some embodiments, resources in the cloud infrastructure system 902 may be shared by multiple users and dynamically reallocated according to demand. In addition, resources may be allocated to users in different time zones. For example, the cloud infrastructure system 930 may enable a first group of users in a first time zone to use the resources of the cloud infrastructure system for a specified number of hours, and then reallocate the same resources to another group of users in a different time zone, thereby maximizing resource utilization.

[0107] In one embodiment, several internal shared services 932 may be provided by different components or modules of the cloud infrastructure system 902 and by services provided by the cloud infrastructure system 902. These internal shared services may include, but are not limited to, security and identity services, integration services, enterprise repository services, enterprise manager services, virus scanning and whitelisting services, high availability, backup and recovery services, services to enable cloud support, email services, notification services, file transfer services, and the like.

[0108] In certain embodiments, the cloud infrastructure system 902 may provide comprehensive management of cloud services (e.g., SaaS, PaaS, and IaaS services) within the cloud infrastructure system. In one embodiment, the cloud management functionality may include the ability to provision, manage, and track customer subscriptions received by the cloud infrastructure system 902.

[0109] In one embodiment, as shown in the figure, the cloud management functionality may be provided by one or more modules, such as an order management module 920, an order orchestration module 922, an order provisioning module 924, an order management and monitoring module 926, and an identity management module 928. These modules may include, or be provided using, one or more computers and / or servers, which may be general-purpose computers, dedicated server computers, server farms, server clusters, or any other suitable configuration and / or combination.

[0110] In exemplary operation 934, a customer using a client device such as client device 904, 906, or 908 may interact with the cloud infrastructure system 902 by requesting one or more services provided by the cloud infrastructure system 902 and placing an order for a subscription to one or more services provided by the cloud infrastructure system 902. In one embodiment, the customer may access a cloud user interface (UI), cloud UI 912, cloud UI 914, and / or cloud UI 916 and place a subscription order through these UIs. The order information received by the cloud infrastructure system 902 in response to the customer placing an order may include information identifying the customer and one or more services provided by the cloud infrastructure system 902 that the customer intends to subscribe to.

[0111] After an order is placed by the customer, the order information is received via the cloud UI 912, 914, and / or 916.

[0112] In operation 936, the order is stored in the order database 918. The order database 918 may be one of several databases operated by the cloud infrastructure system 918 and operated in conjunction with other system elements.

[0113] In operation 938, the order information is transferred to the order management module 920. In some examples, the order management module 920 may be configured to perform billing and accounting functions related to the order, such as verifying the order and, once verified, recording the order.

[0114] In operation 940, information about the order is communicated to the order orchestration module 922. The order orchestration module 922 may use the order information to orchestrate the provisioning of services and resources for the order placed by the customer. In some examples, the order orchestration module 922 may use the services of the order provisioning module 924 to orchestrate the provisioning of resources to support the subscribed services.

[0115] In certain embodiments, the order orchestration module 922 enables the management of business processes associated with each order and applies business logic to determine whether the order should proceed to provisioning. In operation 942, upon receiving a new subscription order, the order orchestration module 922 sends a request to the order provisioning module 924 to allocate resources and configure the resources required to fulfill the subscription order. The order provisioning module 924 enables the allocation of resources for the services ordered by the customer. The order provisioning module 924 provides a level of abstraction between cloud services provided by the cloud infrastructure system 900 and a physical implementation layer used to provision resources to provide the requested services. Thus, the order orchestration module 922 may be isolated from implementation details, such as whether services and resources are actually provisioned on the fly or pre-provisioned and allocated / assigned only on demand.

[0116] In operation 944, once the services and resources are provisioned, a notification of the provided services may be sent by the order provisioning module 924 of the cloud infrastructure system 902 to the customers on client devices 904, 906, and / or 908.

[0117] In operation 946, customer subscription orders may be managed and tracked by the order management and monitoring module 926. In some cases, the order management and monitoring module 926 may be configured to collect usage statistics about the services in the subscription order, such as the amount of storage used, the amount of data transferred, the number of users, and the amount of system uptime and system downtime.

[0118] In some embodiments, the cloud infrastructure system 900 may include an identity management module 928. The identity management module 928 may be configured to provide identity services in the cloud infrastructure system 900, such as access management and authorization services. In some embodiments, the identity management module 928 may control information about customers who wish to use services provided by the cloud infrastructure system 902. Such information may include information that authenticates the identity of such customers and information that describes what actions those customers are permitted to perform on various system resources (e.g., files, directories, applications, communication ports, memory segments, etc.). The identity management module 928 may also include managing descriptive information about each customer, as well as descriptive information about how and by whom that descriptive information can be accessed and modified.

[0119] Figure 10 shows an exemplary computer system 1000 in which various embodiments of the present invention may be realized. System 1000 may be used to realize any of the computer systems described above. As shown in the figure, computer system 1000 includes a processing unit 1004 that communicates with several peripheral subsystems via a bus subsystem 1002. These peripheral subsystems may include a processing acceleration unit 1006, an I / O subsystem 1008, a storage subsystem 1018, and a communication subsystem 1024. The storage subsystem 1018 includes a tangible computer-readable storage medium 1022 and system memory 1010.

[0120] The bus subsystem 1002 provides a mechanism for various components and subsystems of the computer system 1000 to communicate with each other as intended. Although the bus subsystem 1002 is schematically shown as a single bus, alternative embodiments of the bus subsystem may utilize multiple buses. The bus subsystem 1002 may be one of several types of bus structures, including a memory bus or memory controller, peripheral bus and local bus, using any of the various bus architectures. For example, such architectures may include the Industry Standard Architecture (ISA) bus, the Microchannel Architecture (MCA) bus, the Extended ISA (EISA) bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus, which can be implemented as a mezzanine bus manufactured according to the IEEE P1386.1 standard.

[0121] The processing unit 1004 can be implemented as one or more integrated circuits (e.g., conventional microprocessors or microcontrollers) and controls the operation of the computer system 1000. One or more processors may be included in the processing unit 1004. These processors may include single-core processors or multi-core processors. In certain embodiments, the processing unit 1004 may be implemented as one or more independent processing units 1032 and / or 1034, each containing a single-core processor or a multi-core processor. In other embodiments, the processing unit 1004 may also be implemented as a quad-core processing unit formed by integrating two dual-core processors onto a single chip.

[0122] In various embodiments, the processing unit 1004 can execute various programs in response to program code and can maintain multiple programs or processes running simultaneously. At any given time, some or all of the program code to be executed can reside in the processor 1004 and / or the storage subsystem 1018. Through suitable programming, the processor 1004 can provide the various functionalities described above. The computer system 1000 may further include a processing acceleration unit 1006, which may include a digital signal processor (DSP), a special-purpose processor, and the like.

[0123] The I / O subsystem 1008 may include user interface input devices and user interface output devices. User interface input devices may include pointing devices such as keyboards, mice or trackballs, touchpads or touchscreens integrated into displays, scroll wheels, click wheels, dials, buttons, switches, keypads, voice input devices with voice command recognition systems, microphones, and other types of input devices. User interface input devices may also include motion sensing and / or gesture recognition devices such as Microsoft Kinect® motion sensors, which enable users to control and interact with input devices such as Microsoft Xbox® 360 game controllers through a natural user interface using gesture and speech commands. User interface input devices may detect eye movements from the user (e.g., blinking while taking a picture and / or making a menu selection) and use eye gestures to input devices (e.g., Google The user interface input device may also include eye gesture recognition devices such as the Google Glass® blink detector, which converts the input into Glass®. In addition, the user interface input device may include a voice recognition sensing device that enables the user to interact with a voice recognition system (e.g., Siri® Navigator) via voice commands.

[0124] User interface input devices may include, but are not limited to, three-dimensional (3D) mice, joysticks or pointing sticks, gamepads and graphic tablets, as well as auditory / visual devices such as speakers, digital cameras, digital camcorders, portable media players, webcams, image scanners, fingerprint scanners, barcode readers, 3D scanners, 3D printers, laser rangefinders, and eye-tracking devices. In addition, user interface input devices may include, for example, medical imaging input devices such as computed tomography, magnetic resonance imaging, positron emission tomography, and medical ultrasound equipment. User interface input devices may also include, for example, audio input devices such as MIDI keyboards and digital musical instruments.

[0125] User interface output devices may include non-visual displays such as display subsystems, indicator lights, or audio output devices. Display subsystems may include flat panel devices such as those using cathode ray tubes (CRTs), liquid crystal displays (LCDs), or plasma displays, projection devices, touchscreens, etc. Generally, the use of the term “output device” is intended to include all conceivable types of devices and mechanisms for outputting information from computer system 1000 to a user or another computer. For example, user interface output devices may include, but are not limited to, a variety of display devices that visually convey text, graphics, and audio / video information, such as monitors, printers, speakers, headphones, car navigation systems, plotters, audio output devices, and modems.

[0126] The computer system 1000 may also include a storage subsystem 1018 containing software elements, which are currently shown as being located in the system memory 1010. The system memory 1010 may store program instructions that can be loaded and executed on the processing unit 1004, as well as data generated during the execution of these programs.

[0127] Depending on the configuration and type of the computer system 1000, the system memory 1010 may be volatile (such as random access memory (RAM)) and / or non-volatile (such as read-only memory (ROM), flash memory, etc.). RAM typically includes data and / or program modules that are immediately accessible to the processing unit 1004 and / or currently being operated and executed by the processing unit 1004. In some implementations, the system memory 1010 may include several different types of memory, such as static random access memory (SRAM) or dynamic random access memory (DRAM). In some implementations, a basic input / output system (BIOS) containing basic routines that help transfer information between elements within the computer system 1000, such as during startup, may typically be stored in ROM. As an example, but not an limitation, the system memory 1010 also shows application programs 1012, program data 1014, and an operating system 1016, which may include client applications, web browsers, middle-tier applications, relational database management systems (RDBMS), etc. For example, operating system 1016 is compatible with various versions of Microsoft Windows®, Apple Macintosh®, and / or Linux®. This may include trademark operating systems, various commercially available UNIX® or UNIX-like operating systems (including, but not limited to, various GNU / Linux operating systems, Google Chrome® OS, etc.), and / or mobile operating systems such as iOS, Windows® Phone, Android® OS, BlackBerry® 10 OS, and Palm® OS.

[0128] The storage subsystem 1018 may also provide a tangible, computer-readable storage medium for storing basic programming and data structures that provide the functionality of some embodiments. Software (programs, code modules, instructions) that, when executed by the processor, provides the above-described functionality may be stored in the storage subsystem 1018. These software modules or instructions may be executed by the processing unit 1004. The storage subsystem 1018 may also provide a repository for storing data used in accordance with the present invention.

[0129] The storage subsystem 1000 may also include a computer-readable storage medium reader 1020 which may be further connected to the computer-readable storage medium 1022. Together with the system memory 1010, and optionally in combination with the system memory 1010, the computer-readable storage medium 1022 may comprehensively represent a combination of a storage medium and remote, local, fixed, and / or removable storage devices for temporarily and / or more permanently storing, storing, transmitting, and retrieving computer-readable information.

[0130] Computer-readable storage medium 1022 containing code or a portion of code also includes, but is not limited to, any suitable medium known or used in the art, including storage and communication media such as volatile and non-volatile, removable and non-removable media, implemented by any method or technique for storing and / or transmitting information. This may include tangible computer-readable storage media such as RAM, ROM, electronically erasable programmable ROM (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile disk (DVD) or other optical storage devices, magnetic cassettes, magnetic tapes, magnetic disk storage devices or other magnetic storage devices, or other tangible computer-readable media. It may also include intangible computer-readable media such as any other media that can be used to transmit data signals, data transmissions, or desired information and can be accessed by the computing system 1000.

[0131] For example, the computer-readable storage medium 1022 is a hard disk drive that reads and writes to a non-removable non-volatile magnetic medium, a magnetic disk drive that reads and writes to a removable non-volatile magnetic disk, a CD-ROM, a DVD, and a Blu-ray (registered trademark). The computer-readable storage medium 1022 may include, but is not limited to, optical disc drives that read from and write to removable non-volatile optical discs such as disks, or other optical media. The computer-readable storage medium 1022 may also include, but is not limited to, Zip® drives, flash memory cards, Universal Serial Bus (USB) flash drives, Secure Digital (SD) cards, DVD discs, digital videotapes, etc. The computer-readable storage medium 1022 may also include flash memory-based SSDs, enterprise flash drives, solid-state drives (SSDs) based on non-volatile memory such as solid-state ROM, SSDs based on volatile memory such as solid-state RAM, dynamic RAM, static RAM, DRAM-based SSDs, magnetoresistive RAM (MRAM) SSDs, and hybrid SSDs that use a combination of DRAM and flash memory-based SSDs. Disk drives and computer-readable media associated therewith may provide computer system 1000 with non-volatile storage of computer-readable instructions, data structures, program modules, and other data.

[0132] The communication subsystem 1024 provides an interface to other computer systems and networks. The communication subsystem 1024 acts as an interface for sending and receiving data between other systems and the computer system 1000. For example, the communication subsystem 1024 may enable the computer system 1000 to connect to one or more devices via the Internet. In some embodiments, the communication subsystem 1024 may include radio frequency (RF) transceiver components for accessing wireless voice and / or data networks (using, for example, cellular telephone technology, 3G, 4G, or EDGE (High Speed ​​Data Rate for Global Evolution)), a Global Positioning System (GPS) receiver component, and / or other components. In some embodiments, the communication subsystem 1024 may provide wired network connectivity (e.g., Ethernet®) in addition to, or instead of, a wireless interface.

[0133] In some embodiments, the communication subsystem 1024 may also receive input communications in the form of structured and / or unstructured data feeds 1026, event streams 1028, event updates 1030, etc., on behalf of one or more users who may use a computer system 1000.

[0134] For example, the communication subsystem 1024 handles Twitter® feeds, Facebook ( (Registered Trademark) Updates, Web feeds such as Rich Site Summary (RSS) feeds, and / or real-time updates from one or more third-party sources, such as real-time data feeds from users of social networks and / or other communication services 10 It may be configured to receive 26.

[0135] In addition, the communication subsystem 1024 may also be configured to receive data in the form of a continuous data stream, which may include an event stream 1028 and / or event update 1030 of real-time events, which may be inherently continuous or infinite, without an explicit termination. Examples of applications that generate continuous data may include, for example, sensor data applications, financial stock market boards, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, and automotive traffic monitoring.

[0136] The communication subsystem 1024 may also be configured to output structured and / or unstructured data feeds 1026, event streams 1028, event updates 1030, etc., to one or more databases which may communicate with one or more streaming data source computers connected to the computer system 1000.

[0137] The computer system 1000 can be one of various types, including handheld portable devices (e.g., iPhone® mobile phones, iPad® computing tablets, PDAs), wearable devices (e.g., Google Glass® head-mounted displays), PCs, workstations, mainframes, kiosks, server racks, or any other data processing systems.

[0138] Due to the ever-changing nature of computers and networks, the description of computer system 1000 shown in the figure is intended merely as a specific example. Many other configurations are possible, having more or fewer components than the system depicted in the figure. For example, customized hardware may also be used, and / or certain elements may be implemented in hardware, firmware, software (including applets), or a combination thereof. Furthermore, connections to other computing devices, such as network input / output devices, may be employed. Based on the disclosures and teachings provided herein, those skilled in the art will understand other aspects and / or methods for implementing various embodiments.

[0139] In the preceding description, for illustrative purposes, numerous specific details have been provided for a complete understanding of the various embodiments of the present invention. However, it will be apparent to those skilled in the art that embodiments of the present invention may be carried out without some of these specific details. In other examples, well-known structures and devices are shown in block diagram form.

[0140] The foregoing description provides only exemplary embodiments and is not intended to limit the scope, applicability, or configuration of the present disclosure. Rather, the foregoing description of exemplary embodiments will provide a practical description for carrying out the exemplary embodiments for those skilled in the art. It should be understood that various modifications can be made to the function and configuration of the elements without departing from the spirit and scope of the invention as set forth in the claims.

[0141] The above description provides specific details to give a complete understanding of the embodiments. However, it will be understood by those skilled in the art that embodiments may be carried out without these specific details. For example, circuits, systems, networks, processes, and other components may be shown as components in the form of block diagrams to avoid obscuring the embodiments with unnecessary details. In other cases, well-known circuits, processes, algorithms, structures, and techniques may be shown without unnecessary details to avoid obscuring the embodiments.

[0142] Furthermore, note that individual embodiments may be described as processes depicted as flowcharts, flow diagrams, data flow diagrams, structural diagrams, or block diagrams. While some flowcharts may describe operations as sequential processes, many operations may be performed in parallel or simultaneously. In addition, the order of operations may be rearranged. A process terminates when its operations are completed, but it may have additional steps not shown in the diagram. A process may correspond to a method, function, procedure, subroutine, subprogram, etc. If a process corresponds to a function, its termination may correspond to the function returning to a calling function or main function.

[0143] The term “computer-readable medium” includes, but is not limited to, portable or fixed storage devices, optical storage devices, wireless channels, and various other media that can store, contain, or carry instructions and / or data. A code segment or machine-executable instruction may represent a procedure, function, subprogram, program, routine, subroutine, module, software package, class, or any combination of instructions, data structures, or program statements. A code segment may be coupled to another code segment or hardware circuit by passing and / or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc., may be passed, transferred, or transmitted via any preferred means, including memory sharing, message passing, token passing, network transmission, etc.

[0144] Furthermore, embodiments may be implemented by hardware, software, firmware, middleware, microcode, a hardware description language, or any combination thereof. When implemented by software, firmware, middleware, or microcode, the program code or code segments that perform the required tasks may be stored in a machine-readable medium. The required tasks may be performed by a processor.

[0145] While aspects of the invention are described in the aforementioned specification with reference to specific embodiments, those skilled in the art will recognize that the invention is not limited thereto. The various features and aspects of the invention described above may be used individually or in combination. Furthermore, embodiments can be used in any number of environments and applications beyond those described herein without departing from the broader spirit and scope of this specification. Accordingly, the specification and drawings should be considered illustrative rather than restrictive.

[0146] Furthermore, for illustrative purposes, the method has been described in a specific order. It should be understood that in alternative embodiments, the method may be performed in an order different from that described. It should also be understood that the above method may be performed by hardware components, or it may be embodied in a sequence of machine-executable instructions, which, when used, cause a machine such as a general-purpose or dedicated processor or logic circuit programmed with such instructions to perform the above method. These machine-executable instructions may be stored on one or more machine-readable media, such as a CD-ROM or other type of optical disk, a floppy diskette, ROM, RAM, EPROM, EEPROM, magnetic or optical card, flash memory, or other types of machine-readable media suitable for storing electronic instructions. Alternatively, the method may be performed by a combination of hardware and software.

Claims

1. A non-temporary computer-readable medium having instructions that cause one or more processors to perform an operation when executed by said one or more processors, wherein the operation is, Determining the number of clusters to be provisioned as part of a service in the cloud infrastructure, Determining the number of availability domains for each cluster to be provisioned in the aforementioned cloud infrastructure, Determining the number of compute nodes in each availability domain to be provisioned in the aforementioned cloud infrastructure, Determining the number of load balancers for each cluster to be provisioned in the aforementioned cloud infrastructure, The number of subnets is calculated based on the number of clusters, the number of availability domains, the number of compute nodes, and the number of load balancers. Multiple subnets are generated based on the number of subnet levels, A non-temporary computer-readable medium, which includes allocating the plurality of subnets to the compute nodes and load balancers in the cloud infrastructure.

2. The computer-readable medium according to claim 1, wherein the operation further includes causing each of the compute nodes and each of the load balancers to provision the plurality of subnets in the cloud infrastructure.

3. The aforementioned operation further, Receiving changes to the number of availability domains, the number of clusters, the number of compute nodes, or the number of load balancers, Recalculating the number of subnet levels based on the aforementioned changes, The computer-readable medium according to claim 1, further comprising regenerating the plurality of subnets based on the number of subnet levels.

4. The aforementioned operation further, This includes determining a second number of clusters, a second number of availability domains, a second number of compute nodes, and a second number of load balancers to be provisioned as part of a second service in the cloud infrastructure, wherein the second service differs from the service, and the operation further includes: This includes calculating a second number of subnet levels based on the second number of clusters, the second number of availability domains, the second number of compute nodes, and the second number of load balancers, wherein the second number of subnet levels is different from the number of subnet levels, and the operation further includes: The computer-readable medium according to claim 1, comprising generating a second plurality of subnets based on the second number of subnet levels.

5. The number of subnets is equal to base 2^(number of subnet levels), according to claim 1. The computer-readable media described above.

6. Determining the number of availability domains is: For the aforementioned service, the service receives a selection of a configuration from a plurality of predetermined configurations, The computer-readable medium according to claim 1, comprising determining a Service Level Agreement (SLA) for the said configuration, wherein the SLA defines the number of availability domains.

7. Determining the number of availability domains is: The computer-readable medium according to claim 1, which includes receiving the number of availability domains as runtime input when the instruction is executed by one or more processors.

8. The computer-readable medium according to claim 1, wherein one availability domain in the availability domain includes at least one of the compute nodes and at least one of the load balancers.

9. Generating the plurality of subnets based on the number of subnet levels is, The computer-readable medium according to claim 1, comprising executing a function that performs a classless interdomain routing (CIDR) slice of the top-level network address.

10. The computer-readable medium according to claim 9, wherein the function accepts the number of subnet levels as a parameter so that the number of subnet levels can be calculated at runtime.

11. The computer-readable medium according to claim 1, wherein the operation further comprises assigning a plurality of labels to each of the plurality of subnets.

12. The computer-readable medium according to claim 11, wherein the plurality of labels are based on the combination of the availability domain and the cluster.

13. The computer-readable medium according to claim 12, further comprising generating the combination of the availability domain and the cluster by calculating the Cartesian product of the operations.

14. The computer-readable medium according to claim 12, wherein calculating the Cartesian product includes generating a vector cross product between the availability domain and the cluster.

15. The computer-readable medium according to claim 14, wherein the index in the plurality of labels is related to the index in the cluster by dividing the index in the plurality of labels by the number of clusters.

16. The computer-readable medium according to claim 14, wherein the index in the plurality of labels is associated with the index in the availability domain by performing modular division of the index in the plurality of labels by the number of availability domains.

17. The computer-readable medium according to claim 12, wherein the plurality of labels include an index that is modified by a counter that is incremented each time a compute node or load balancer is declared.

18. The computer-readable medium according to claim 12, wherein the plurality of labels are generated without using a nested loop structure.

19. It is a system, One or more processors, The system comprises one or more memory devices which, when executed by the one or more processors, include instructions that cause the one or more processors to perform an operation, and the operation is, Determining the number of clusters to be provisioned as part of a service in the cloud infrastructure, Determining the number of availability domains for each cluster to be provisioned in the aforementioned cloud infrastructure, Determining the number of compute nodes in each availability domain to be provisioned in the aforementioned cloud infrastructure, Determining the number of load balancers for each cluster to be provisioned in the aforementioned cloud infrastructure, The number of subnets is calculated based on the number of clusters, the number of availability domains, the number of compute nodes, and the number of load balancers. Multiple subnets are generated based on the number of subnet levels, A system comprising allocating the plurality of subnets to the compute nodes and load balancers in the cloud infrastructure.

20. A method for running a provisioning script with a runtime adjustment number of resource declarations, Determining the number of clusters to be provisioned as part of a service in the cloud infrastructure, Determining the number of availability domains for each cluster to be provisioned in the aforementioned cloud infrastructure, Determining the number of compute nodes in each availability domain to be provisioned in the aforementioned cloud infrastructure, Determining the number of load balancers for each cluster to be provisioned in the aforementioned cloud infrastructure, The number of subnets is calculated based on the number of clusters, the number of availability domains, the number of compute nodes, and the number of load balancers. Multiple subnets are generated based on the number of subnet levels, A method comprising allocating the plurality of subnets to the compute nodes and load balancers in the cloud infrastructure.

Images