Orchestration of data center creation on cloud platform

Abstract

A computing system, e.g., a multi-tenant system, deploys software artifacts in a datacenter created in a cloud platform using a cloud platform independent cloud platform infrastructure language. The system receives a declarative specification for creating a datacenter on the cloud platform. The system generates an aggregate pipeline including a hierarchy of pipelines. The system generates an aggregate deployment version map that associates datacenter entities of the datacenter with versions of software artifacts targeted for deployment on the datacenter entities. The system collects a set of software artifacts according to the aggregate deployment version map. The system executes the aggregate pipeline in conjunction with the aggregate deployment version map to create a datacenter according to the cloud platform independent declarative specification.

Description

[Technical field]

[0001] The present disclosure relates generally to the composition of services in a cloud computing platform, and more particularly to orchestrating the creation of data centers in a cloud computing platform. [Background technology]

[0002] Organizations are increasingly relying on cloud platforms (or cloud computing platforms) such as AWS (AMAZON WEB SERVICES), GOOGLE CLOUD PLATFORM, MICROSOFT AZURE, etc. for their infrastructure needs. Cloud platforms provide servers, storage, databases, networking, software, etc. to organizations over the Internet. Traditionally, organizations have had data centers that house the hardware and software used by the organization. However, maintaining a data center can incur significant overhead in terms of maintenance, personnel, etc. As a result, organizations are migrating their data centers to cloud platforms that offer scalability and elasticity of computing resources.

[0003] A large-scale system, such as a multi-tenant system, may manage services for multiple organizations that represent tenants of the multi-tenant system and may interact with multiple cloud platforms. The multi-tenant system may have to maintain thousands of such data centers on the cloud platform. Each data center may have different requirements for software releases. Furthermore, the software, languages, features supported by each cloud platform may differ. For example, different cloud platforms may support different mechanisms for implementing network policies or access control. Furthermore, it takes a lot of effort to provision resources such as databases / accounts / computing clusters and deploy software on cloud platforms. Thus, configuring a data center with multiple services on a cloud platform is complex. The configuration often involves manual steps and is prone to errors and security violations. These errors often lead to downtime. Such downtime of a large-scale system, such as a multi-tenant system, may affect a very large number of users and significant interruptions in services. [Brief description of the drawings]

[0004] [Figure 1] FIG. 1 is a block diagram of a system environment illustrating a multi-tenant system that configures a data center on a cloud platform, according to one embodiment. [Figure 2A] 2 is a block diagram illustrating a system architecture of a deployment module 210 according to one embodiment. [Figure 2B] 1 illustrates an overall process for deploying software artifacts in a data center, according to one embodiment. [Diagram 3] FIG. 2 is a block diagram illustrating the architecture of a software release management module, according to one embodiment. [Figure 4] 1 illustrates an example of a declarative specification of a data center, according to one embodiment. [Diagram 5] 1 illustrates an exemplary data center created on a cloud platform based on a declarative specification, according to one embodiment. [Figure 6] FIG. 1 is a block diagram illustrating the creation of a data center on a cloud platform based on a declarative specification, according to one embodiment. [Figure 7] 1 illustrates an overall process for generating a pipeline for deploying software artifacts on a data center configured on a cloud platform, according to one embodiment. [Figure 8] 1 illustrates an exemplary master pipeline, according to one embodiment. [Figure 9] 1 illustrates the overall process performed by stages for an environment of a master pipeline on a cloud platform, according to one embodiment. [Figure 10] 1 illustrates an exemplary data center configuration specified using a declarative specification, according to one embodiment. [Figure 11] 1 illustrates an exemplary master pipeline, according to one embodiment. [Figure 12] 1 illustrates a system architecture of an orchestration engine, according to one embodiment. [Figure 13] 1 shows a flowchart illustrating a process for creating a data center in a cloud platform, according to one embodiment. [Figure 14] 1 illustrates a process for building an aggregation pipeline, according to one embodiment. [Figure 15] 1 illustrates a process for gathering information to build a pipeline, according to one embodiment. [Figure 16] 1 illustrates a process for setting up a pipeline for execution according to one embodiment. [Figure 17] 1 illustrates a process for building an aggregate deployment manifest, according to one embodiment. [Figure 18] 1 depicts a flowchart illustrating a process for performing operations on a data center configured on a cloud platform, according to one embodiment. [Figure 19] 1 illustrates a process for generating an aggregate version map, according to one embodiment. [Figure 20] 2 is a block diagram illustrating a functional diagram of an exemplary computer system for use in the environment of FIG. 1, according to one embodiment.

[0005] The figures depict various embodiments for purposes of example only. Those skilled in the art will readily recognize from the following discussion that alternative embodiments of the structures and methods shown herein may be employed without departing from the principles of the embodiments described herein.

[0006] The figures use like reference numbers to identify like elements. Letters following a reference number, such as "115a," indicate that the text specifically refers to the element with that particular reference number. A reference number in text without a following letter, such as "115," refers to any or all of the elements in the figure with that reference number. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] A cloud platform provides computing resources, such as storage, computing resources, and applications, to computing systems on demand over a public network connection, such as the Internet. Cloud platforms allow enterprises to minimize up-front investments for setting up computing infrastructure and allow enterprises to launch and move applications faster with less maintenance overhead. Cloud platforms also allow enterprises to adjust computing resources to meet rapidly fluctuating and unpredictable demand. Enterprises can create data centers using the computing resources of the cloud platform. However, implementing a data center on each cloud platform requires expertise in the technology of the cloud platform.

[0008] An embodiment creates a datacenter in a cloud platform using a cloud platform infrastructure language that is cloud platform agnostic. The system receives a cloud platform agnostic declarative specification of the datacenter. The declarative specification describes a structure of the datacenter and may not provide instructions specifying how to create the datacenter. The cloud platform agnostic declarative specification is configured to generate a datacenter on any of a plurality of cloud platforms and is specified using the cloud platform infrastructure language. The system receives information identifying a target cloud platform for creating the datacenter and compiles the cloud platform agnostic declarative specification to generate a cloud platform specific datacenter representation. The system transmits the cloud platform specific datacenter representation and a set of instructions for execution on the target cloud platform. The target cloud platform executes the instructions to configure the datacenter using the platform specific datacenter representation. The system provides a user with access to computing resources of the datacenter configured by the cloud platform.

[0009] According to one embodiment, a system receives a cloud platform agnostic declarative specification for creating a datacenter on a cloud platform. The system generates an aggregation pipeline including a hierarchy of pipelines. The hierarchy of pipelines includes pipelines for creating datacenter entities of the datacenter. The aggregation pipeline is configured to create the datacenter. The system generates an aggregate deployment version map that associates datacenter entities of the datacenter with versions of software artifacts subject to deployment on the datacenter entities. The system collects a set of software artifacts according to the aggregate deployment version map. The software artifacts are associated with datacenter entities of the datacenter being created. The system executes the aggregation pipeline in conjunction with the aggregate deployment version map to create the datacenter according to the cloud platform agnostic declarative specification. The aggregation pipeline composes services based on the set of software artifacts.

[0010] When creating a datacenter on a cloud platform, there may be multiple operations that need to be performed on various datacenter entities of the datacenter. For example, the system may receive a request to deploy version V1 of service S1 on service group G1, deploy version V2 of service S2 on service group G1, deploy version V3 of service S3 on service group G2, etc. Since there may be various operations, e.g., deployment, provisioning, patching, etc., that are frequently performed on datacenter entities, it is cumbersome to require a service owner to specify the version of each artifact to be used for each operation, the version of the pipeline to be used, and other configuration details of each operation. A system according to an embodiment receives multiple version maps (also called version manifests or deployment manifests) from a service owner. The version maps identify the versions of the software artifacts, the versions of the pipeline, and any associated configuration details. The system receives information from the service owner indicating the context in which each version map may be used. For example, the context may specify the type of datacenter entity (or specific name of the datacenter entity) for which the version map is applicable, the type of environment (development environment, test environment, production environment), the type of operation for which the version map can be used (provisioning, deployment, patching, ...), etc. The system receives a runtime request to perform an operation for a particular environment on a set of datacenter entities. For example, the operation may be provisioning, the environment may be a development environment, and the datacenter entity may be a particular service instance. The system analyzes the available version maps to identify the most appropriate version map to use for the current context.

[0011] According to one embodiment, a system receives a cloud platform agnostic declarative specification for creating a datacenter on a cloud platform. The system generates an aggregation pipeline including a hierarchy of pipelines. The hierarchy of pipelines includes pipelines for creating datacenter entities of the datacenter. The aggregation pipeline is configured to create the datacenter. The system generates an aggregate deployment version map that associates datacenter entities of the datacenter with versions of software artifacts subject to deployment on the datacenter entities. The system collects a set of software artifacts according to the aggregate deployment version map. The software artifacts are associated with datacenter entities of the datacenter being created. The system executes the aggregation pipeline in conjunction with the aggregate deployment version map to create the datacenter according to the cloud platform agnostic declarative specification. The aggregation pipeline composes services based on the set of software artifacts.

[0012] The disclosed system addresses the problem of creating an aggregate version map that contains all deployment information for all possible targets in a cloud platform on which one or more operations are being performed. A user, e.g., a system administrator of a tenant in a multi-tenant system, may specify this information. Specifying this can be cumbersome for the user and can lead to duplicate information being defined in multiple places, e.g., the user's existing version map and the aggregate version map specified for a particular set of operations. The user provides a path to the aggregate version map via metadata that the system looks up to perform the required operations.

[0013] Advantages of the disclosed system include improved ease of use and ease of use in deployments on cloud platforms since service owners do not need to specify a version map for each deployment instance, improved reliability and faster on-boarding time due to fewer manual steps and less chance of human error, and improved storage efficiency as a result of avoiding redundant definition and storage of information in multiple places. Storage efficiency is improved since the user provides a single copy of each version map that can be used for multiple deployments.

[0014] The cloud platform is also referred to herein as the substrate. The declarative specification of the data center is substrate independent or substrate agnostic. When data center related operations such as software release deployment and resource provisioning are performed using traditional techniques, the user must provide cloud platform specific instructions. Thus, the user requires expertise of the cloud platform being used. Furthermore, the instructions are cloud platform specific and cannot be portable across multiple platforms. For example, instructions for deploying software on the AWS cloud platform are different from instructions on the GCP cloud platform. The developer needs to understand the details of how each function is implemented on that particular cloud platform. The disclosed system provides a cloud platform infrastructure language that allows a user to perform operations on a data center using cloud platform agnostic instructions that can be executed on any cloud platform selected from multiple cloud platforms. A compiler for the cloud platform infrastructure language generates cloud platform specific detailed instructions for the target cloud platform.

[0015] A data center configured on a cloud platform may also be referred to as a virtual data center. Although the embodiments are described with respect to a data center configured on a cloud platform (i.e., a virtual data center), the disclosed techniques may also be applied to a physical data center.

[0016] The system may represent, but is not limited to, a multi-tenant system, and may be any online system or any computing system with network access to a cloud platform. System Environment

[0017] 1 is a block diagram of a system environment illustrating a multi-tenant system constituting a data center on a cloud platform according to an embodiment. System environment 100 includes a multi-tenant system 110, one or more cloud platforms 120, and one or more client devices 105. In other embodiments, system environment 100 may include more or fewer components.

[0018] The multi-tenant system 110 stores information for one or more tenants 115. Each tenant may be associated with a business that represents a customer of the multi-tenant system 110. Each tenant may have multiple users that interact with the multi-tenant system via client devices 105.

[0019] A cloud platform may also be referred to as a cloud computing platform or a public cloud environment. A tenant may use a cloud platform infrastructure language to provide a declarative specification of a data center to be created on the target cloud platform 120 and use the data center to perform operations, e.g., provision resources, perform software releases, etc. A tenant 115 may create one or more data centers on the cloud platform 120. A data center represents a set of computing resources including servers, applications, storage, memory, etc. that may be used by a user, e.g., a user associated with the tenant. Each tenant may provide different functionality to the tenant's users. Thus, each tenant may run different services on the data center configured for that tenant. A multi-tenant system may implement different mechanisms for software release and deployment for each tenant. A tenant may further obtain or develop a version of software that includes instructions for various services that run within the data center. An embodiment allows a tenant to deploy specific versions of software releases for different services that run on different computing resources of the data center.

[0020] The computing resources of a data center are secure and cannot be accessed by users without access permissions. For example, a data center 125a created for a user of tenant 115a cannot be accessed by a user of tenant 115b unless access is explicitly allowed. Similarly, a data center 125b created for a user of tenant 115b cannot be accessed by a user of tenant 115a unless access is explicitly allowed. Furthermore, services provided by a data center can be accessed from a computing system outside the data center only if the computing system is granted access according to the declarative specification of the data center.

[0021] In a multi-tenant system 110, data for multiple tenants may be stored in the same physical database. However, the database is configured such that one tenant's data is kept logically separated from other tenants' data, and one tenant cannot access another tenant's data unless such data is explicitly shared. It is transparent to the tenants that a tenant's data may be stored in tables that are shared with other customers' data. A database table may store rows for multiple tenants. Thus, in a multi-tenant system, various elements of the system's hardware and software may be shared by one or more tenants. For example, the multi-tenant system 110 may run an application server that processes requests for several tenants simultaneously. However, the multi-tenant system enforces tenant-level data isolation to ensure that one tenant's jobs do not access another tenant's data.

[0022] Examples of cloud platforms include AWS (AMAZON web services), GOOGLE cloud platform, or MICROSOFT AZURE. The cloud platform 120 provides computing infrastructure services that can be used on-demand by the tenants 115 or by any computing system outside the cloud platform 120. Examples of computing infrastructure services provided by the cloud platform include servers, storage, databases, networking, security, load balancing, software, analytics, intelligence, and other infrastructure service capabilities. These infrastructure services can be used by the tenants 115 to build, deploy, and manage applications in a scalable and secure manner.

[0023] The multi-tenant system 110 may include a tenant data store that stores data for various tenants of the multi-tenant store. The tenant data store may store data for different tenants in separate physical structures, such as separate database tables or separate databases. Alternatively, the tenant data store may store data for multiple tenants in a shared structure. For example, user accounts for all tenants may share the same database table. However, the multi-tenant system stores additional information to logically separate the data of different tenants.

[0024] Each component shown in Figure 1 represents one or more computing devices. The computing devices may be, for example, conventional computer systems running a Microsoft® Windows® compatible operating system (OS), Apple™ OS X®, and / or a Linux® distribution. The computing devices may also be client devices having computer functionality, such as personal digital assistants (PDAs), mobile phones, video game systems, etc. Each computing device stores software modules that store instructions.

[0025] Interaction between the various components of system environment 100 is typically performed over a network, not shown in Figure 1. In one embodiment, the network uses standard communication technologies and / or protocols. In another embodiment, the entities may use custom and / or proprietary data communication technologies instead of or in addition to those described above.

[0026] Although the techniques disclosed herein are described in the context of a multi-tenant system, the techniques can be implemented using other systems that may not be multi-tenant systems. For example, an online system used by a single organization or enterprise may use the techniques disclosed herein to create one or more data centers on one or more cloud platforms 120. System Architecture

[0027] The multi-tenant system 110 includes a deployment module for deploying software artifacts on a cloud platform. The deployment module can perform various operations associated with a software release, such as, for example, provisioning resources on the cloud platform, deploying the software release, and performing a rollback of the software artifacts installed on the data center entities. FIG. 2 is a block diagram illustrating a system architecture of a deployment module 210, according to one embodiment. The deployment module 210 includes a data center creation module 220 and a software release management module 230. Other embodiments can have different and / or other components than those described herein and can distribute functionality among the components in a different manner.

[0028] The data center creation module 220 includes instructions for creating a data center on the cloud platform. The software release management module 230 includes instructions for deploying software releases for various services or applications that run on the data center created by the data center creation module 220.

[0029] The data center generation module 220 receives a cloud platform independent declarative specification of a data center from a user, e.g., a tenant user. The cloud platform independent declarative specification of a data center specifies various entities of the data center. In one embodiment, the cloud platform independent declarative specification of a data center includes a hierarchical organization of data center entities, where each data center entity may include one or more services, one or more other data center entities, or a combination of both. FIG. 4 describes the various types of data center entities in further detail. The data center generation module 220 receives the platform independent declarative specification and a target cloud platform as input and generates a cloud platform specific metadata representation for the target cloud platform. The data center generation module 220 deploys the generated cloud platform specific metadata representation on the target cloud platform according to the declarative specification to create a data center on the target cloud platform.

[0030] Software release management module 230 receives as inputs (1) artifact version map 225 and (2) master pipeline 235. Artifact version map 225 identifies specific versions of software releases or deployment artifacts targeted for deployment on specific data center entities. Artifact version map 225 maps data center entities to software release versions targeted for deployment on data center entities. Master pipeline 235 contains instructions for operations related to software releases on a data center, e.g., deployment of a service, destruction of a service, provisioning of resources for a service, destruction of resources for a service, etc.

[0031] The master pipeline 235 may include instructions for performing operations related to software releases for different environments, such as a development environment, a test environment, a canary environment, and a production environment, and instructions for determining when a software release is promoted from one environment to another. For example, if the deployment of a software release in a development environment executes more than a threshold number of test cases, the software release is promoted to a test environment for further testing, such as, for example, system-level and integration testing. When the software release in the test environment exceeds a threshold of test coverage, the software release is promoted to a canary environment, where the software release is provided to a small subset of users on a trial basis. When the software release in the canary environment runs without errors for a threshold time, the software release is promoted to a production environment, where the software release is provided to all users.

[0032] The software release management module 230 compiles the input artifact version map 225 and the master pipeline 235 to generate a cloud platform specific detailed pipeline 255 that is sent to the target cloud platform. The cloud platform specific detailed pipeline 255 includes instructions for deploying the appropriate version of the software release or deployment artifact on the data center entity as specified in the artifact version map 225. The software release management module 230 may receive a modification to one of the inputs. For example, a user may modify the input artifact version map 225 and provide the same master pipeline 235. Thus, the same master pipeline is used, but a different software release is deployed on the data center entity. The software release management module 230 recompiles the inputs to generate a new cloud platform specific detailed pipeline 255 that deploys the version of the software release according to the new artifact version map 225.

[0033] An artifact version map may also be referred to as a deployment manifest, a version manifest, a software release map, or a software artifact version map. A master pipeline may also be referred to as a master deployment pipeline or a master orchestration pipeline. An artifact version manifest or deployment manifest specifies information specific to a data center entity, such as the particular software artifact versions to be used for the data center entity, the values ​​of parameters provided as input to the pipeline for that data center entity, the types of computing resources to be used for that data center entity, particular parameter values ​​for the configuration of the computing resources for the data center entity, etc.

[0034] FIG. 2B illustrates an overall process for deploying software artifacts to a datacenter, according to one embodiment. FIG. 2B illustrates a layout of a datacenter 265 including various datacenter entities. As shown in FIG. 2B, an artifact version map 225 identifies different versions of software targeted for release on different datacenter entities 275 of the datacenter 265. A master pipeline represents the flow of deployment artifacts through the various environments of the datacenter. The software release management module 230 combines the information in the master pipeline 235 with the artifact version map 225 to determine a cloud platform specific detailed pipeline 255 that maps appropriate versions of software artifacts onto the datacenter entities according to the artifact version map 225.

[0035] 3 is a block diagram illustrating an architecture of software release management module 230, according to one embodiment. Software release management module 230 includes a parsing module 310, a pipeline generator module 320, an artifact version map store 330, a pipeline store 340, and a pipeline execution engine 360. Other embodiments may include more, fewer, or different modules than those shown in FIG.

[0036] The parsing module 310 parses various types of user inputs, including declarative specifications of data centers, artifact version maps 225, and master pipelines 235. The parsing module 310 generates data structures and metadata representations of the processed inputs and provides the generated data structures and metadata representations to other modules of the software release management module 230 for further processing.

[0037] The metadata store 340 stores various transformed metadata representations of the data center generated by the software release management module 230. The transformed metadata representations may be used to perform rollbacks to previous versions if problems are encountered with the current version of the data center. The transformed metadata representations may be used for validation, auditing, governance, etc. at various stages of the transformation process.

[0038] The pipeline generator module 320 processes the master pipeline together with the artifact version map received as input to generate a detailed pipeline for the target cloud platform. The pipeline includes stages including instructions for provisioning a service or deploying an application to deploy versions of software releases for various services on the cloud platform according to the artifact version map. The artifact version map store 330 stores the artifact version map received from the user, and the pipeline store 340 stores the master pipeline and the pipelines generated by the pipeline generator module 320.

[0039] Pipeline execution engine 360 ​​executes the detailed pipeline generated by pipeline generator module 320. In one embodiment, pipeline execution engine 360 ​​is a system such as SPINNAKER that executes pipelines for releasing / deploying software. Pipeline execution engine 360 ​​parses the pipeline and executes each stage of the pipeline on the target cloud computing platform.

[0040] The orchestration engine 350 performs orchestration of operations related to data centers or data center entities on the cloud platform, including creation, destruction, and modification of data centers or data center entities. The orchestration engine 350 processes a declarative specification of a data center and uses the layout of the data center defined by the declarative specification to generate a pipeline for orchestration of operations associated with the data center. Details of the orchestration engine 350 are shown in and described in connection with FIG. 12. Processes performed by the orchestration engine 350 are further described herein. Creation of cloud platform-based data centers

[0041] 4 illustrates an example of a declarative specification of a data center, according to one embodiment. The declarative specification 410 includes multiple data center entities. A data center entity is an instance of a data center entity type, and there may be multiple instances of each data center entity type. Examples of data center entities include a data center, a service group, a service, a team, an environment, and a schema.

[0042] The declarative specification 410 includes definitions of various types of datacenter entities, including service groups, services, teams, environments, and schemas. The declarative specification includes one or more instances of a datacenter. Below are descriptions of various types of datacenter entities and examples thereof. These examples are illustrative and show some of the attributes of the datacenter entities. Other embodiments may include different attributes, and attributes with the same functionality may be given different names than those shown herein. In one embodiment, the declarative specification is specified using hierarchical objects, e.g., Javascript object notation (JSON), that conform to a predefined schema.

[0043] A service group 520 represents a set of capabilities and features and services provided by one or more computing systems that can be independently constructed and delivered, according to one embodiment. A service group may also be referred to as a logical service group, a functional unit, a business unit, or a bounded context. A service group 520 may also be viewed as a set of services for a set of cohesive technical use case functionality provided by one or more computing systems. A service group 520 enforces security boundaries. A service group 520 defines the scope of modifications. Thus, any modifications to entities such as capabilities, features, or services provided by one or more computing systems in a service group 520 may be propagated as needed or appropriate to entities within the service group, but not to entities residing outside the bounded definition of the service group 520. A data center may include multiple service groups 520. A service group definition specifies attributes including a name, a description, an identifier, a schema version, and a set of service instances. An example of a service group is a blockchain service group that includes a set of services used to provide blockchain functionality. Similarly, a security service group provides security functionality. A user interface service group provides functionality for a particular user interface function. The Shared Documents service group provides functionality for sharing documents between users. There may be several other service groups as well.

[0044] Service groups support reusability of specifications so that a tenant or user interested in developing a data center has a library of service groups that can be easily used. The boundaries of services in a service group are based on security and network concerns, among others. A service group is associated with a protocol for performing interactions with the service group. In one embodiment, a service group provides an application programming interface (API) and a collection of services that implement those APIs. Additionally, service groups are substrate agnostic. A service group provides a blast radius for services within the service group such that failure of a service within the service group has limited impact on services within the service group and minimal impact outside the service group.

[0045] The following is an example of a service group specification: The service group specifies various attributes that represent metadata of the service group and includes the set of services within the service group. There may also be other types of metadata specified for a service group that are not shown here.

number

[0046] As shown in the above example, a service group may specify a set of clusters. A cluster represents a set of computing nodes, e.g., a set of servers, a set of virtual machines, or a set of containers (such as KUBERNETES containers). A physical server can run multiple containers, with each container having its own share of the file system, CPU, memory, process space, etc.

[0047] A service group specifies a set of services. A service group may specify clusters of services such that a data center deployed on a cloud platform runs clusters of computing nodes and, when included in a declarative specification, maps services to clusters based on the specified mapping. For example, in the service group example shown above, service instance serviceinstance0002 is specified to run on cluster instance cluster1.

[0048] A service group may specify security groups, each of which specifies a set of services that are allowed to interact with each other. Services outside of a security group must pass additional authentication to communicate with services in the security group. Alternatively, services in a security group may interact with each other using one protocol, and services outside of the security group may interact with services in the security group using a different protocol that requires enhanced authentication. Thus, a security group specifies policies that determine how services can interact with each other. A security policy may specify one or more environments to which the security policy is applicable. For example, a security policy policy1 may apply to a particular environment env1 (e.g., production environment), and another security policy policy2 may apply to another environment env2 (e.g., development environment). A security policy may be specified for a service group type or for a particular service type.

[0049] In one embodiment, a security policy specifies expressions for filtering service groups based on various attributes such that the security policy is applicable to a filtered set of service groups. For example, a security policy may specify a list of Internet Protocol (IP) addresses that are whitelisted for the set of service groups identified by the filtered set, and therefore those computing systems are allowed access to a particular set of service groups or services within the service groups.

[0050] In one embodiment, a security policy may specify a set of source services and a set of destination services for a service group. A source service for a particular service specifies services outside of the security group that are allowed to connect with this particular service. A destination service for a particular service specifies services outside of the security group to which this particular service must connect. During provisioning and deployment, the data center generation module generates instructions for the cloud platform that enforce the specific network policy using the cloud platform's specific features and network functionality such that the network policy enforces the security policy specified in the declarative specification.

[0051] A datacenter entity called a cell represents a set of services that interact with each other vertically and can be scaled with additional instances or copies of the cell, i.e. copies of the set of services. Creating multiple instances of a cell allows the system to scale a set of services that interact with each other. A datacenter instance can contain one or more cells. Each cell can contain one or more services. A datacenter can contain service groups or instances of cells.

[0052] A service definition specifies metadata about the type of service, e.g., database service, load balancer service, etc. The metadata may describe various attributes of the service, including the name of the service, a description of the service, the location of documentation for the service, any subservices associated with the service, the owner of the service, the team associated with the service, the build dependencies of the service that specify other services this service depends on at build time, the start dependencies of the service that specify other services that should be running when this particular service is started, authorized clients, DNS (Domain Name Server) names associated with the service, the service status, the support level of the service, etc. The service definition specifies a listening port attribute that specifies ports on which the service can listen for different communication protocols, e.g., a service may listen on port p1 for UDP protocol and port p2 for TCP protocol. Other services in the data center can interact with the service through the ports specified by the service.

[0053] A service definition specifies a destination endpoint, e.g., an attribute outbound-access that specifies an external URL (Uniform Resource Locator) that specifies that the service requires access to a specified external URL. During deployment, the data center generation module ensures that the cloud platform implements the access policy such that instances of this service type are provided with the requested access to the external URL.

[0054] An outbound access specification may identify one or more environment types for a service to which the outbound access is applicable. For example, outbound access to a set of endpoints S1 may apply to a particular environment env1 (e.g., a production environment), and outbound access to a set of endpoints S2 may apply to another environment env2 (e.g., a development environment).

[0055] Below is an example of a service definition.

number

[0056] The team definition 450 includes team member names and other attributes of the team, such as, for example, name, email, communication channel, etc. The following is an example of a team definition: A service may be associated with one or more teams that are responsible for modifications made to the service. Thus, any modifications made to the service are approved by the team. A service may be associated with a team that is responsible for the maintenance of the service after it is deployed in the cloud platform. A team may be associated with a service group and, accordingly, with all services of that service group. For example, a team approves any changes to a service group, e.g., a service that is part of the service group. A team may be associated with a data center and, therefore, with all service groups in the data center. A team association specified at the data center level provides a default team for all service groups in the data center and further provides a default team for all services in the service group.

[0057] According to one embodiment, team associations specified at the function level override team associations provided at the datacenter level. Similarly, team associations specified at the service level override defaults that may have been provided by team associations specified at the service group level or datacenter level. The team can determine how certain actions are taken for the datacenter entities associated with the team. The team association also determines the number of accounts on the cloud platform that are created for generating the final metadata representation of the datacenter for the cloud platform by the compiler and for provisioning and deploying the datacenter on the cloud platform. The datacenter generation module 210 creates one or more user accounts in the cloud platform and provides access to the user accounts for the team members. Thus, the team members are permitted to perform certain actions associated with the datacenter entities associated with the team, such as making or approving structural changes to the datacenter entities or maintenance of the datacenter entities when the datacenter entities are deployed, including debugging and testing issues that may be identified for the datacenter entities.

[0058] Conventional techniques associate the same team with a datacenter throughout the design process, resulting in organizational structure influencing the design of the datacenter or service group. Embodiments decouple team definition from the structure that defines the datacenter entity, thereby reducing the influence of teams on the design and architecture of the datacenter entity.

number

[0059] The environment definition 460 specifies the type of system environment represented by the data center, for example, development, staging, testing, or production. The schema definition 470 specifies a schema that specifies the syntax of certain data center entity definitions. The schema definition 470 is used to validate various data center entity definitions. The data center generation module determines the security policies of the data center in the cloud platform specific metadata representation based on the environment. For example, a particular set of security policies may be applicable to environment env1, and a different set of security policies may be applicable to environment env2. For example, the security policies provide much more restricted access in the production environment compared to the development environment. The security policies may specify the length of time that a security token is allowed to exist for a particular purpose. For example, in the development environment, long access tokens (e.g., week-long access tokens) may be allowed, while in the production environment, access tokens with a much shorter life span (e.g., a few hours) may be used. The access tokens may allow a user or service to access a particular cloud platform resource.

[0060] A datacenter definition 420 specifies attributes and components of a datacenter instance. A declarative specification may specify multiple datacenter instances. A datacenter definition 420 specifies attributes including a name, a description, a type of environment, a set of service groups, a team, domain name servers for the datacenter, etc. A datacenter definition may specify a schema definition, and any metadata representation generated from the datacenter definition is validated against the specified schema definition. A datacenter includes a set of core services and capabilities that enable other services to function within the datacenter. An instance of a datacenter may be deployed in a particular cloud platform and associated with a particular environment type, e.g., development, testing, staging, production, etc.

[0061] Below is the definition of a datacenter instance: The datacenter instance definition includes a list of service groups contained in the datacenter instance, as well as other attributes including the datacenter's environment, a datacenter identifier, a name, a region representing a geographic region, one or more teams associated with the datacenter, and a schema version.

number

[0062] FIG. 5 illustrates some exemplary data centers created on a cloud platform based on a declarative specification, according to one embodiment. The data centers 510 may be created based on a declarative specification processed by the data center generation module 210. As shown in FIG. 5, multiple data centers may be configured in the cloud platform 120. Each data center 510 may correspond to a tenant 115 of the multi-tenant system 110. The tenant 115 may create one or more data centers 510. Alternatively, the data centers 510 may be created by any computing system. Each data center includes one or more service groups. For example, data center 510a includes service groups 520a and 520b, and data center 510b includes service group 520c. A data center may include multiple instances of a particular type of service group. Each service group includes a set of services. For example, service group 520a includes services 530a and 530b, service group 520b includes services 530a, 530b, and 530c, and service group 520c includes services 530e, 530f, and 530g. A service group may include multiple instances of services of the same service type.

[0063] The data center generation module 220 creates a data center on a cloud platform based on a declarative specification using the following steps: The data center generation module 210 receives a cloud platform independent declarative specification of a data center. The cloud platform independent declarative specification can be for a tenant of a multi-tenant system or for any other computing system, e.g., an online system. The cloud platform independent declarative specification is specified using a cloud platform infrastructure language. The cloud platform independent declarative specification of the data center is configured to generate a data center on any of multiple cloud platforms.

[0064] The data center generation module 210 receives information identifying a target cloud platform for creating a data center based on the cloud platform independent declarative specification. The target cloud platform may be any of a number of cloud platforms, such as AWS, AZURE, GCP, etc. The data center generation module 210 further receives information for connecting with the target cloud platform, such as authentication information for creating a connection with the target cloud platform. The cloud platform may also be referred to as a cloud computing platform.

[0065] The data center generation module 210 compiles the cloud platform independent declarative specification to generate a cloud platform specific data center representation for creating a data center on a target cloud computing platform. For example, the cloud platform specific data center representation may refer to user accounts, network addresses, etc. specific to the target cloud computing platform.

[0066] The data center generation module 210 transmits the platform-specific data center representation along with instructions for deploying the data center on the target cloud computing platform. The target cloud computing platform executes the instructions for configuring computing resources of the target cloud computing platform to generate a data center according to the platform-specific data center representation. The data center generation module 210 provides users with access to the computing resources of the data center configured by the cloud computing platform. For example, if a data center is created for a tenant of a multi-tenant system, a user associated with the tenant is provided with access to the data center.

[0067] 6 is a block diagram illustrating the generation of a data center on a cloud platform based on a declarative specification, according to one embodiment. The data center generation module 210 receives as input a cloud platform independent declarative specification 610. The cloud platform independent declarative specification 610 may be a version of a declarative specification that has been incrementally modified by a user. The data center generation module 210 processes a specific version of the cloud platform independent declarative specification 610. Because the cloud platform independent declarative specification 610 is not specified for any particular target cloud platform, the data center generation module 210 can configure a data center on any target cloud platform based on the cloud platform independent declarative specification 610.

[0068] The data center generation module 210 processes the cloud platform independent declarative specification 610 to generate a cloud platform independent detailed metadata representation 620 for the data center. The cloud platform independent detailed metadata representation 620 defines details of each instance of a data center entity specified in the cloud platform independent declarative specification 610. The data center generation module 210 creates unique identifiers for the data center entity instances, e.g., service instances.

[0069] In one embodiment, the cloud platform agnostic detailed metadata representation 620 includes an array of instances of a data center entity type, e.g., an array of service group instances of a particular service group type. Each service group instance includes an array of service instances. The service instances may further include details of a team of users authorized to perform specific actions associated with the service instance. The team details are used during provisioning and deployment by the data center creation module 210, e.g., to create user accounts for the service instance and enable members of the team to access the user accounts.

[0070] The cloud platform independent detailed metadata representation 620 includes attributes of each instance of the data center entity. Thus, the description of each instance of the data center entity is expanded to include all details. As a result, the cloud platform independent detailed metadata representation 620 of the data center can be significantly larger than the cloud platform independent declarative specification 610. For example, the cloud platform independent declarative specification 610 may be thousands of lines of specification, and the cloud platform independent detailed data center representation 620 may be millions of lines of generated code. As a result, the data center generation module 210 holds the cloud platform independent detailed metadata representation 620 as immutable, i.e., once the representation is finalized, no modifications are performed on the representation. For example, if any updates, deletions, or additions of the data center entities need to be performed, they are performed on the cloud platform independent declarative specification 610.

[0071] The data center generation module 210 receives a target cloud platform where the data center is expected to be provisioned and deployed, and generates a cloud platform specific detailed metadata representation 630 of the data center. For example, the data center generation module 210 interacts with the target cloud platform to generate specific entities (or resources), e.g., user accounts, networking resources such as virtual private clouds (VPCs) and subnets on the VPCs, various connections between entities in the cloud platform, etc. The data center generation module 210 receives resource identifiers of resources created in the target cloud platform, e.g., user account names, VPC IDs, etc., and incorporates them into the cloud platform independent detailed metadata representation 620 to obtain a cloud platform specific metadata representation 630 of the data center. In one embodiment, the data center generation module 210 creates one unique user account on the cloud platform for each team for a given combination of service group and service. The user account is used by the team to perform interactions with that particular service for that service group, e.g., for debugging, to receive alerts, etc.

[0072] The target cloud platform may perform several steps to process the cloud platform specific detailed metadata representation 630. For example, the cloud platform independent declarative specification may specify permitted interactions between services. These permitted interactions are specified in the cloud platform specific detailed metadata representation 630 and implemented as network policies of the cloud platform. The cloud platform may further create security groups to implement a network strategy for implementing the data center according to the declarative specification.

[0073] The cloud platform independent declarative specification specifies dependencies between services, e.g., start dependencies for each service that list all services that should be executed when a particular service is started. The data center generation module 220 generates a cloud platform specific detailed metadata representation of the data center that includes information describing these dependencies, such that the instructions for deploying the services ensure that the cloud platform starts the services in the order specified by the dependencies, such that, for each service, services that need to be started before the service is executed are running when the service is started. Thus, the dependencies between services represent a dependency graph, and the cloud platform begins executing the services in an order determined based on the dependency graph, such that if service A depends on service B, then service B is started before service A is started.

[0074] The data center generation module 220 creates trust relationships between user accounts that allow services to access other services over secure communication channels. These trust relationships are generated based on the declarative specification, e.g., using board-specific instructions generated based on outbound access attributes specified for the services. The data center generation module 220 sends instructions to the cloud platform to create network policies based on cloud platform-specific mechanisms that control interactions and access across service groups and services, as specified by constructs in the declarative specification, e.g., outbound access, security groups, security policies, etc.

[0075] The data center generation module 210 deploys the cloud platform specific metadata representation 630 onto the particular target cloud platform for which it was generated. The data center generation module 210 may use the generated metadata representation to perform various validations, including policy validation, format validation, etc.

[0076] The cloud platform independent declarative specification 610 may be referred to as a declared data center representation, the cloud platform independent detailed metadata representation 620 may be referred to as a derived metadata representation of the data center, and the cloud platform specific metadata representation 630 may be referred to as a hydrated metadata representation of the data center. The overall process for deploying software artifacts on a data center

[0077] 7 illustrates an overall process for generating a pipeline for deploying software artifacts on a data center configured on a cloud platform, according to one embodiment. A data center generation module generates one or more data centers on a target cloud platform (710). Each data center is generated from a cloud platform independent declarative specification and has a hierarchy of data center entities.

[0078] The software release management module 230 generates (720) a cloud platform agnostic master pipeline. In one embodiment, the cloud platform agnostic master pipeline includes stages corresponding to environments of a data center, e.g., development, test, canary, and production environments. The master pipeline configures a series of incremental and / or conditional deployments across various environments, such as development, test, staging, or production environments. The master pipeline may be triggered by the delivery of an image of the software artifact and includes stages or instructions for deploying the build in a development type environment. The built software artifact is conditionally promoted after one or more test environments, followed by one or more canary environments, before finally being deployed to the production environment. The master pipeline may be customized by a user, e.g., a service owner, to represent a particular orchestration across environments. The master pipeline may be customized to capture a particular promotion criteria for moving from one stage to the next. For example, different tenants of a multi-tenant system may customize the master pipeline in different ways. In one embodiment, the master pipeline by default uses the latest version of software for software artifacts for a service and builds and deploys that version across various environments. Users can use artifact version maps to ensure that specific versions of software artifacts are deployed on specific data center entities.

[0079] In one embodiment, each service deployed in a data center has a cloud platform agnostic master pipeline generated from data center entities as defined by the declarative specification of the data center, e.g., a master pipeline for a data center instance, a master pipeline for a service group, a master pipeline for a cell, a master pipeline for a service, etc. The master pipeline may be triggered upon delivery of an image of a software artifact. The master pipeline may implement continuous deployment controlled by a service owner. The master pipeline may implement an owner-owned data center instance or release an owner-owned on-demand deployment.

[0080] Certain portions of the master pipeline may be customized by users, e.g., by tenants of a multi-tenant system deploying services on a data center. For example, the promotion decision pipeline may be customized by a tenant to determine which test cases are executed and what the thresholds are. The software release management module 230 receives 730 customizations to the logic for promoting software artifacts from one stage to another of the cloud platform agnostic master pipeline.

[0081] The software release management module 230 compiles (740) the cloud platform agnostic master pipeline to generate cloud platform specific detailed deployment pipelines specific to the hierarchy of data center entities for each data center specified by the cloud platform agnostic declarative specification of the data center.

[0082] The software release management module 230 further receives (750) code for releasing one or more features of the service deployed on the data center. The software release management module 230 executes (760) a cloud platform specific detailed deployment pipeline to deploy software artifacts based on the received code.

[0083] FIG. 8 illustrates an exemplary master pipeline 800, according to one embodiment. The master pipeline represents a series of stages representing progressive conditional deployment across various data center environments. FIG. 8 illustrates stages for different environments of a data center, including development, test, canary, and production environments. Each stage further represents the pipeline that is executed for that stage. Thus, the master pipeline 800 includes a development environment pipeline 810, which feeds into a test environment pipeline 820, which feeds into a canary environment pipeline 830, which feeds into a production environment pipeline 840.

[0084] The pipelines at each stage are hierarchical pipelines that contain lower level pipelines. For example, development environment pipeline 810 includes a development master pipeline that feeds into data center pipelines D11, D12, ... depending on the number of data centers specified in the data center declarative specification as having development environments.

[0085] The test environment pipeline 820 includes a test master pipeline that feeds into data center pipelines D21, D22, . . . depending on the number of data centers specified as having test environments in the data center declarative specification.

[0086] The canary environment pipeline 820 includes a canary master pipeline that feeds into data center pipelines D31, D32, ... depending on the number of data centers specified as having a canary environment in the data center declarative specification.

[0087] The production environment pipeline 820 includes a production master pipeline that feeds into data center pipelines D21, D22, . . . depending on the number of data centers specified as having test environments in the data center declarative specification.

[0088] Each environment pipeline 810, 820, 830 includes a promotion decision pipeline 815a, 815b, 815c, respectively. The output of the data center pipelines of the environment pipelines is collected by the promotion decision pipeline 815, which determines whether the software artifact is ready for promotion to the next stage. The promotion decision pipeline 815 may determine whether the software artifact for the service is promoted to the next stage based on the test case results obtained by the data center. For example, if more than a threshold number of test cases pass, the promotion decision pipeline 815 promotes the software artifact to the next stage. The last environment stage, e.g., the production environment pipeline, may not have a promotion decision pipeline because there is no subsequent stage to which the software artifact needs to be promoted. As shown in FIG. 8, the promotion decision pipeline 815a of the development environment pipeline determines whether to promote a software artifact from a development phase to a testing phase, the promotion decision pipeline 815b of the test environment pipeline determines whether to promote a software artifact from a testing phase to a canary phase, and the promotion decision pipeline 815c of the canary environment pipeline determines whether to promote a software artifact from the canary phase to a production phase.

[0089] The master pipeline includes multiple pipelines, such as a provisioning pipeline for provisioning resources of a target cloud platform and a deployment pipeline for deploying software artifacts on a data center entity. Each pipeline includes a series of stages, and each stage represents one or more actions that need to be performed by the target cloud platform toward provisioning and deploying the data center. The data center generation module 210 generates detailed pipelines for deploying versions of software artifacts on the data center entity.

[0090] In one embodiment, the pipeline generator module 320 generates a detailed pipeline using a pipeline template that includes variables. The pipeline template is converted into a pipeline by providing specific values ​​for the variables in the pipeline. The process of generating a pipeline from a template is called hydration of the pipeline template. The pipeline template includes template expressions that are used as placeholders for actual values ​​used in deployment. For example, the template expressions may be replaced by target-specific parameter values ​​or expressions. Multiple pipeline instances may be generated by hydrating the pipeline template for different targets. The template variables represent parameters that may be replaced with target-specific values ​​to generate a pipeline instance specific to the given target. For example, the template variable "account_id" may be replaced with the actual value of account_id, e.g., "12345", during hydration.

[0091] In one embodiment, the pipeline generator module 320 generates pipelines hierarchically based on a hierarchy of data center entities for a data center. For example, a data center includes different types of data center entities including data centers, service groups, services, etc. A data center entity may include one or more child data center entities. For example, a data center includes one or more service groups as child data center entities. A service group includes one or more services as child data center entities. Thus, the data center generation module 210 starts with a data center entity at a level of the hierarchy and generates pipelines for data center entities below that level. For example, the pipeline generator module 320 starts at the data center level and generates pipelines for service groups in the data center. For each service group, the pipeline generator module 320 generates pipelines for the services in the service group.

[0092] A process for executing a pipeline according to one embodiment is as follows: A software release deployment module 230 receives a request to deploy a software artifact on a set of data center entities in a target cloud platform. The software release deployment module 230 executes a master pipeline for one or more data centers. The software release deployment module 230 executes an aggregation pipeline for each service group in each data center. The aggregation pipeline includes pipelines for the services in the service group. For each service in each service group, the pipeline is executed by executing all stages of the pipeline. Execution of the provisioning pipeline results in provisioning of resources for the service, and the deployment pipeline deploys the service to the target cloud platform.

[0093] Figure 9 shows the overall process performed by stages for an environment of a master pipeline on a cloud platform, according to one embodiment. Steps 910, 920, 930, 940, and 950 may be performed by each environment pipeline 810, 820, 830. The production environment pipeline 3 may only perform steps 910 and 920. The steps shown in Figure 9 may be performed for one service or multiple services specified using a manifest file.

[0094] The environment pipeline for environment E includes instructions for deploying 910 the software on a set of datacenter entities, e.g., a set of datacenter entities designated as having environment E. In one embodiment, the software artifacts are generated by compiling source code of the services. The source code may be obtained from version control software. The set of datacenter entities may include datacenter instances, service groups, cells, services, or any combination thereof.

[0095] The environment pipeline for environment E further includes instructions for running tests to test the software artifacts deployed on the set of data center entities (920). The environment pipeline for environment E further includes instructions for evaluating test results against promotion criteria (930), e.g., using promotion decision pipeline 815. If the promotion criteria are not met, steps 910, 920, 930, and 940 may be repeated using a revised software artifact, e.g., a software artifact generated from source code that includes fixes for specific defects identified during testing 920. The environment pipeline for environment E further includes instructions for proceeding to the next stage if the promotion criteria are met (950).

[0096] In one embodiment, the master pipeline includes a hierarchy of pipelines. The hierarchy includes multiple levels, and pipelines at a particular level include pipelines at the next lower level as child pipelines. For example, at the top level of the hierarchy, the master pipeline includes a release master pipeline that deploys a set of services related to a product. The next level of the hierarchy includes a service master pipeline that represents the deployment of a particular service across various environments. The next level of the hierarchy may include a service group master pipeline followed by a service master pipeline.

[0097] FIG. 10 illustrates an exemplary data center configuration specified using a declarative specification, according to one embodiment. Root node 1020x represents a data center that includes a hierarchy of data center entities. Data center entities 1020a, 1020b, 1020c, 1020d, 1020e may represent service groups (e.g., functional domains). Each data center entity that represents a service group includes one or more services. For example, data center entity 1020d includes services 1030c and 1030d, data center entity 1020e includes services 1030i, 1030j, 1030k, data center entity 1020b includes services 1030e and 1030f, and so on. Data center entities may include not only services but also other data center entities. For example, datacenter entity 1020a includes services 1030a, 1030b, and datacenter entity 1020d, datacenter entity 1020c includes services 1030g, 1030h, and datacenter entity 1020e, and so on. The system uses the declarative specification to determine the layout of the datacenter being created (as a datacenter blueprint) and to guide the process of orchestrating the creation of the datacenter. For example, the system creates pipelines for creating datacenter entities and for creating individual services based on the declarative specification.

[0098] FIG. 11 illustrates an exemplary aggregation pipeline generated to create a datacenter based on a declarative specification, according to one embodiment. For example, FIG. 11 illustrates an aggregation pipeline representing a hierarchy of pipelines corresponding to the hierarchy of datacenter entities defined in the declarative specification of FIG. 10. The pipeline structure illustrated in FIG. 11 includes a pipeline corresponding to each datacenter entity of a datacenter specified by the declarative specification, e.g., the datacenter illustrated in FIG. 10. The system receives information identifying the pipelines of an individual service from a service owner. For example, the service owner may provide a pipeline for the service or provide a link to where the pipelines are stored. The pipelines for a service received from the service owner are also referred to as unit pipelines. For example, pipelines 1120a, 1120b, 1120c, ... etc. are unit pipelines for an individual service. Each unit pipeline may be executed to compose a corresponding service on a cloud platform. The system generates an aggregation pipeline 1110 that groups the individual service pipelines. For example, aggregation pipeline 1110a corresponds to data center entity 1010a, aggregation pipeline 1110b corresponds to data center entity 1010b, aggregation pipeline 1110d corresponds to data center entity 1010d, and so on. The system generates aggregation pipeline 1110x for the entire data center 1020x. Once all services and data center entities under a parent data center entity are configured (e.g., services are configured and run), the parent data center entity is configured. A pipeline that is not a leaf-level pipeline but has one or more child pipelines is an aggregation pipeline that orchestrates the execution of the child pipelines.

[0099] A pipeline for a data center may be referred to as a master pipeline. A master pipeline is a hierarchical pipeline where each stage of the pipeline may include a pipeline with detailed instructions for executing the stage. The master pipeline hierarchy may mirror the data center hierarchy. For example, the top level of the master pipeline represents a series of stages for different environments. Each environment may include one or more pipelines for a data center instance, or pipelines for other types of data center entities. A data center instance pipeline 1010 may include a service group pipeline 1020. Each service group pipeline 1020 may include one or more service pipelines 1030. A data center instance pipeline 1010 may include one or more service pipelines 1030. A service pipeline 1030 may include stages, each stage representing a pipeline that represents instructions for deploying a service for a particular environment. The lowest level or leaf level pipelines in the hierarchy are referred to as unit pipelines and may include detailed service-specific instructions for performing operations related to the service. For example, a deployment of a service may include pre-deployment steps, deployment steps, post-deployment steps, and post-deployment testing and validation steps. A pipeline that is not a leaf-level pipeline and has one or more child pipelines is an aggregate pipeline that orchestrates the execution of the child pipelines.

[0100] The master pipeline may be driven by pull requests that occur in a version control system for software, receiving requests to consider changes committed to external repositories for inclusion in the project's main repository. Thus, the master pipeline is automatically triggered when a pull request is received to deploy software artifacts based on the latest software version for which the pull request was received. The master pipeline performs continuous delivery of software artifacts based on the pull requests. The master pipeline may be driven on an on-demand basis, for example, by invoking requests using an application programming interface (API) of the deployment module 210. On-demand deployment based on the master pipeline may be requested for any set of services and for any version for a given service, as specified using the API. The master pipeline may be invoked to request a rollback from a current version to a previous version, or a rollforward from a currently deployed version to a more recent version.

[0101] In one embodiment, the deployment module 210 creates service master pipelines for each service. These pipelines are triggered when a pull request to the software repository is received. The deployment module 210 receives pipeline templates from the service owners for specific services. These pipeline templates contain detailed instructions for testing, validation, building, etc. for a particular service.

[0102] The pipeline generator creates all the pipelines for each datacenter from the templates and combines them hierarchically through the master pipeline. In one embodiment, the pipeline generator generates service pipelines for individual services, the pipeline generator generates service group master pipelines to invoke the service pipelines, and the pipeline generator generates datacenter instance master pipelines to invoke the service group pipelines. Software Artifact Version Map

[0103] In one embodiment, the deployment module 210 receives an artifact version map that associates various software artifacts and their versions with data center entities. The artifact version map provides a declarative specification of the specific versions of software artifacts that need to be deployed for services in different data center entities. Each data center entity may be uniquely identified based on its location in the data center hierarchy as specified by the declarative specification of the data center. For example, with respect to a service, a software library may function as a software artifact. A software artifact may have multiple versions, e.g., V1, V2, V3, etc. The artifact version map may specify that version V1 needs to be deployed in data center entities C1 and C2, and version V2 needs to be deployed in data center entities C3 and C4. The deployment module 210 generates master pipelines and instructions that ensure that the appropriate software artifact versions are deployed in the data center entities as specified in the artifact version map.

[0104] In one embodiment, the artifact version map is specified as a JSON (Javascript object notation) file, a YAML file, or a file using any other syntax for representing nested objects. The artifact version map is associated with various data center entities distributed across a hierarchy of data centers. <service> : <version>It can contain a set of key pairs. The artifact version map key pair acts as a whitelist for the corresponding pipelines. If a service's key is not included in the artifact version map, all pipelines of that service will be excluded during the pipeline execution. Different artifact version maps applied to the same master pipeline will result in different services being included / excluded during the master pipeline execution.

[0105] Below is an example artifact version map. The artifact version map specifies the environment type using the attribute "env_types". In the example below, an environment type development is specified. An environment type may include one or more data center instances, which may include one or more service groups, which may include one or more services. In the example below, the software artifact name is specified as library1 and the version as version1 and is associated with the service instance instance001. However, the software artifact name and version may be associated with a data center entity at any level in the hierarchy. For example, if a software artifact name and version is specified for a service group, the software artifact name and version are applicable to all services within the service group unless the software artifact name and version are overridden with a different value of the software artifact name and version specified for a particular service instance within the service group. Similarly, a software artifact name and version may be specified for a data center instance and is applicable to all service groups or cells within the data center instance unless an override value is specified for the service group.

number

[0106] In one embodiment, the artifact version map specifies a datacenter entity using a full path of the datacenter entity, e.g., "stagger_group1 / datacenter1 / service_group2 / service1." In one embodiment, the artifact version map specifies a set of datacenter entities using regular expressions in the full path of the datacenter entity. For example, a full path that includes service_group[?] includes service_group1, service_group2, service_group3, etc.

[0107] Below is an example of an artifact version map that specifies a regular expression to define a set of services. The environment types are specified as dev and test, the datacenter entity in the full path including datacenter instance and service group is specified as a wildcard, and the service instance is specified as "service*". Therefore, for all datacenter instances in dev and test environments, all service groups, and service names matching service*, version V1 of application app1 will be deployed.

number

[0108] In some embodiments, an artifact version map may specify parameters used by the pipeline such that the specified parameters are applicable to the stagger group for which they are specified. Orchestration Engine System Architecture

[0109] 12 illustrates a system architecture of an orchestration engine according to one embodiment. The orchestration engine 350 includes an orchestration gateway 1210, an orchestration pipeline builder 1220, an orchestration executor 1230, and a version map builder 1240. Other embodiments may include more or fewer components than those shown in FIG.

[0110] The orchestration gateway 1210 is an interface supported by the orchestration engine 350 to receive orchestration requests. The orchestration gateway 1210 intercepts orchestration requests from users and automation systems. In one embodiment, the orchestration gateway 1210 includes an authentication and authorization layer and MTLS (Mutual Transport Layer Security) capabilities to ensure that only authenticated and authorized requests can trigger infrastructure construction.

[0111] Orchestration requests can vary depending on the entities being built. An orchestration request may specify a single service deployment or an entire datacenter deployment hosting multiple services and service groups. The request includes the service name, datacenter, service group, release version of the declarative specification, and the action that needs to be taken (e.g., create, destroy, modify). When a request is received, the orchestration gateway distinguishes whether there is an error in the previously built target, the target under construction, or the previous deployment. If any of these conditions are met, the orchestration gateway may not take any action. If the received request does not belong to these conditions ("built" target, "under construction" target, or "error during deployment"), the orchestration gateway identifies the datacenter entities and services that need to be built and schedules the request for an executor to execute the request. Below is an example request:

number

[0112] The orchestration pipeline builder 1220 generates workflows and pipelines for orchestrating various processes such as creating, destroying, or modifying data centers and data center entities. The orchestration executor 1230 executes actions for the execution of any of the orchestration actions such as creating, destroying, or modifying data centers and data center entities.

[0113] The version map builder 1240 builds an aggregate version map for an operation or set of operations running on a data center configured on the cloud platform. The system allows a user to specify multiple version maps for any service, and the version map builder 1240 selects the best matching version map to use during a particular deployment or provisioning operation. The version map builder 1240 includes a filtering and mapping module 1215, a lookup module 1225, and an aggregation module 1235.

[0114] The version map builder 1240 receives a declarative specification of a data center being configured. The data center may be pre-configured, and the declarative specification represents a target configuration in which the system is requested to modify the data center configuration to match the declarative specification. The deployment module 210 generates a cloud platform agnostic detailed metadata representation based on the declarative specification. The filtering and mapping module 1215 compares the target configuration specified by the cloud platform agnostic detailed metadata representation with the existing configuration of the data center to determine the changes that need to be made to the data center. The version map builder 1240 identifies a best-match version map to be used to perform the requested operation on the data center. There may be multiple version maps that may be applicable to each data center entity. The lookup module 1225 identifies all relevant version maps that are applicable to each data center entity. The version map builder 1240 ranks all applicable version maps that are applicable and finds the best-match version map. The system uses the best-match version map to perform the necessary operation to reconfigure the data center to match the target data center configuration that corresponds to the declarative specification. The aggregation module 1235 generates an aggregated version map based on the set of best matching version maps identified for performing an action on the data center.

[0115] A service owner specifies one or more version maps associated with a service by providing service metadata. Below is an example of a configuration file specifying metadata for a service. The serviceMetadata object specifies a set of attributes including adminTeam, which represents the team working on the service, serviceName, which identifies the service, serviceTypeID, which specifies the type of service, vmfRepo, which represents the path in the repository of version maps, and vmfPath, which represents the location of the version maps associated with the service in the repository.

number

[0116] A service owner can specify multiple version maps applicable to a service using the above service metadata format. Different version maps may be specified for different data center entities, for example, version map M1 for service group G1, version map M2 for service group G2, version map M3 for service instance S1, etc. The version map builder 1240 identifies the best matching version map from the set of version maps specified for each service. The overall orchestration process

[0117] 13-16 show flow charts illustrating various processes associated with creating a data center on a cloud platform. Steps of the various processes are shown as being performed by systems, e.g., multi-tenant system 110 and cloud platform 120. Some steps may be performed by multi-tenant system 110 and some steps may be performed by cloud platform 120. The steps may be performed by various modules, e.g., orchestration engine 350 of software release management module 230 or by other modules shown in FIG. 2, 3, or 12. Other embodiments may perform the steps of the processes shown in FIG. 13 in an order different from the order shown in FIG. 13.

[0118] FIG. 13 shows a flow chart illustrating a process 1300 for creating a data center in a cloud platform, according to one embodiment.

[0119] The system receives 1310 a declarative specification for creating a data center on a cloud platform. The declarative specification is a cloud platform independent declarative specification. The declarative specification specifies a hierarchy of data center entities.

[0120] The system generates 1320 an aggregation pipeline based on the declarative specification. The aggregation pipeline includes a hierarchy of pipelines including a pipeline for creating data center entities for the data center. The aggregation pipeline is configured to create the data center.

[0121] The system generates 1330 an aggregate deployment version map that associates data center entities of the data center with versions of software artifacts that are subject to deployment on the data center entities.

[0122] The system collects 1340 a set of software artifacts according to the aggregate deployment version map, where each software artifact in the set is associated with a data center entity of the data center in which it is being created.

[0123] The system executes 1350 an aggregation pipeline in conjunction with the aggregate deployment manifest to create a data center according to the declarative specification. The aggregation pipeline configures services for the data center based on a set of software artifacts.

[0124] FIG. 14 illustrates a process for building an aggregation pipeline, according to one embodiment.

[0125] The system collects 1410 relevant metadata for building an aggregation pipeline. Step 1410 is also referred to herein as bootstrapping the data center. The bootstrapping step involves collecting relevant metadata for building an aggregation pipeline that is executed to create a data center in a cloud platform. Details of the bootstrapping step are shown in the flow chart shown in FIG.

[0126] The system sets up pipeline stage attributes based on the collected metadata (1420). Details of step 1420 are described in FIG.

[0127] The system performs pipeline property manifestation 1430. Once all the required attributes for the pipeline have been collected and processed, the pipeline properties such as stages, condition variables, preconditions, etc. are ready to be manifested according to the final specification of the pipeline.

[0128] FIG. 15 illustrates a process for collecting metadata information for building a pipeline (1410) according to one embodiment.

[0129] The system retrieves 1510 a data center layout from a declarative specification of a data center. The declarative specification of a data center specifies the data center hierarchy and the boundaries of various data center entities. The data center layout describes the hierarchy of the data center entities and specifies whether a data center entity D1 is within a data center entity D2 according to the hierarchy. The system may extract the layout and build a data structure that represents the layout. The layout specifies various service groups in the data center and the services within each service group. The system also identifies the pipelines required to deploy each service instance in the data center.

[0130] The system determines (1520) pipeline ordering and dependency information from the declarative specification. The declarative specification includes initiation dependencies that specify other datacenter entities that depend on a particular datacenter entity. For example, the declarative specification may specify a set of services that must be running for a particular service to be able to start. The system extracts these dependencies between the datacenter entities. The system may store the dependencies as a dependency graph. The declarative specification may specify groupings of pipelines, for example, for strategic execution of the pipelines or for optimized execution of the pipelines. The system extracts information that describes the pipeline groupings. The declarative specification may specify ordering between the datacenter entities, for example, a datacenter entity D1 associated with a group G1 of an organization may need to be deployed before another datacenter entity D2 associated with a group G2. For example, a datacenter entity associated with a particular storage need may need to be available before any other datacenter entity is deployed. The system extracted the different types of information described above from the declarative specification, including initiation dependencies, pipeline grouping, and datacenter ordering, and incorporated the information into the aggregate pipelines generated for the creation of the datacenters.

[0131] The system performs service attribute overrides (1530) if requested by the service owner. A service owner may define specific steps during installation or execution of a service. For example, a service owner may define a custom workflow that specifies how a service pipeline should be executed. The system receives information describing such customized behaviors and incorporates them into the aggregate pipeline. As another example, a service owner may specify steps that require manual intervention. The system ensures that such steps are included in the pipelines for different services.

[0132] FIG. 16 illustrates a process of setting up a pipeline for execution (1420) according to one embodiment.

[0133] The system determines (1610) an ordering of pipelines and services based on dependencies extracted from the declarative specification. During this step, the system performs pipeline ordering by topologically sorting services based on the starting dependencies defined for each service. Pipelines within each service are ordered according to metadata extracted from the declarative specification and any dependencies specified by the service owner for that particular service. In one embodiment, the pipelines are ordered in the order specified by the dependencies between services.

[0134] The system identifies (1620) data centers, service groups, and services for deployment. In one embodiment, the system receives one or more deployment manifests that identify data center entities and corresponding services that need to be deployed at the data center entities. The deployment manifest also specifies the version of the service that needs to be deployed on the particular data center entities. The deployment manifest may specify a conditional expression to specify which data center entities are deployed and which data center entities need to be skipped. The system evaluates the conditional expression to identify all services that need to be deployed. The conditional expression may evaluate differently at different times. The system aggregates the various deployment manifests received to generate an aggregated deployment manifest that can be applied to an aggregation pipeline. The deployment manifest may further include pipeline parameters for the service instance pipelines. The system extracts the pipeline parameters for specification in the service instance pipelines included in the aggregation pipeline.

[0135] The system assigns identifiers to the pipeline stages (1630). The system may assign a unique stage identifier to each stage. The system uses the identifiers to perform selective execution of stages in the pipeline. According to one embodiment, the system tracks the status of the execution of the aggregation pipeline by identifying previously executed stages and the last stage executed. In the event of a pipeline execution failure, the system determines which stage completed execution, identifies the next stage that did not execute, and resumes execution from that stage. In one embodiment, the system re-executes the pipeline such that each stage checks whether it previously completed execution successfully. If the system determines that a particular stage previously completed execution successfully, the system skips execution of that stage and proceeds to the next stage. The system repeats this process until it encounters a stage that has not been previously executed. If the system encounters a stage that has not been previously executed, the system executes the stage.

[0136] The system aggregates (1640) the pipelines based on the datacenter hierarchy. The system uses the determined pipeline ordering (1610), various pipeline parameters extracted from the declarative specification, or the deployment manifest to build the aggregated pipeline. The system performs service-level aggregation and ordering of the service pipelines using the pipeline groups defined for the services. The pipeline groups represent metadata specified by the service owner. The metadata specified by the service owner defined the pipelines and pipeline group ordering included in each pipeline group. After the service pipeline ordering, the system aggregates all services defined for each service group (e.g., business unit or functional domain). Within the service group, the system orders the pipelines based on dependencies extracted from the declarative specification. The system further performs datacenter aggregation by ordering the service groups.

[0137] The system adds a validation stage to the end of the service group pipeline. The validation stage ensures that the service group is deployed successfully. In one embodiment, the validation stage is fixed in the pipeline, but users can include custom validation scripts that are called by the validation stage when the service group aggregation pipeline runs. The custom validation scripts can perform integration and end-to-end tests. Tests designed for individual services can be completed as part of the service's deployment pipeline.

[0138] Once all the attributes required for the pipeline are collected and processed, the pipeline properties such as stages, condition variables, preconditions, etc. are ready to be manifested according to the final specification of the pipeline. When the orchestration needs to be triggered, runtime metadata may be required for successful creation of the datacenter or datacenter entity. This is done by passing specific software artifacts required by the various sub-pipelines that make up the aggregation pipeline. These software artifacts represent a snapshot of the application and include the application code and any dependent software artifacts required to run the application.

[0139] Similar to building an aggregation pipeline, the system builds an aggregate deployment manifest from a set of input deployment manifests. Figure 17 illustrates the process of building an aggregate deployment manifest, according to one embodiment. A deployable service specifies the software artifacts required for its deployment in a deployment manifest 1710. These deployment manifests 1710 include information about application code versions, runtime configurations, etc. The system aggregates information in a bottom-up manner across a hierarchy of data centers to aggregate the deployment manifest information into an aggregate deployment manifest 1720. A deployment manifest is also called an orchestration manifest.

[0140] Owners of deployable datacenter entities use deployment manifests to specify attributes and runtime parameters of software artifacts that are processed by the process of creating a datacenter. These software artifacts and parameters may vary for different environments. The deployment manifest allows service owners to use regular expressions and wildcards, allowing them to use a common set of values ​​that can be used to manifest the service across different environments.

[0141] The system resolves the deployment manifests expressed by the service owner with wildcards based on runtime orchestration variables such as environment variables. The orchestration manifest resolver uses the appropriate deployment manifest and parameters based on a ranking mechanism. The system may receive multiple deployment manifests applicable to a particular datacenter entity. For example, a service owner may specify different deployment manifests for different environments (development, test, production, etc.). Furthermore, a deployment manifest M1 may be specified for a service, another deployment manifest M2 may be specified for a service group to which the service belongs, and possibly another deployment manifest M3 may be specified for a datacenter. Once the system generates an aggregate deployment manifest, the system resolves the values ​​of various attributes or parameters specified in the various deployment manifests to select the applicable values ​​for the service. The system traverses the hierarchy of datacenter entities specified by the declarative specification of the datacenter to determine the values ​​of certain attributes specified using the deployment manifests. The system aggregates values ​​from multiple deployment manifests to build an aggregate deployment manifest that contains information about the various services and data center entities in the data center hierarchy.

[0142] The system may evaluate wildcards and regular expressions to enumerate a set of values ​​applicable to a particular data center entity. For example, if a deployment manifest for a service uses a wildcard to specify a set of versions, the system evaluates this expression to determine the list of available versions. The list of available versions may change over time, e.g., as new versions are created, additional versions may match the same regular expression.

[0143] In some embodiments, the deployment manifest specifies the number of retries to be performed before determining an orchestration attempt as failed or successful. The deployment manifest may also include a specification of the intended batch processing execution. The batch specification causes the orchestration to be executed in a batch manner. Thus, the system ensures that orchestration batches are counted cumulatively towards a data center entity or data center orchestration. Orchestration based on multiple version maps of services

[0144] 18-19 show flow charts illustrating various processes associated with performing operations related to a data center on a cloud platform. Steps of the various processes are shown as being performed by systems, e.g., multi-tenant system 110 and cloud platform 120. Some steps may be performed by multi-tenant system 110 and some steps may be performed by cloud platform 120. The steps may be performed by various modules, e.g., orchestration engine 350 of software release management module 230 or by other modules shown in FIGS. 2, 3, or 12. Other embodiments may perform the steps of the processes shown in the figures in an order different from the order shown in the figures.

[0145] FIG. 18 shows a flow chart illustrating a process 1800 for performing operations on a data center configured on a cloud platform, according to one embodiment.

[0146] The system receives 1810 a request to perform an operation on a datacenter configured on a cloud platform. According to one embodiment, the request is specified as a declarative specification that represents a target configuration of the datacenter. The system generates a cloud platform agnostic detailed metadata representation based on the declarative specification and compares the target configuration specified by the cloud platform agnostic detailed metadata representation with an existing configuration of the datacenter to determine changes that need to be made to the datacenter. The system determines an operation to be performed on the datacenter based on the comparison. For example, if the target configuration includes a datacenter entity that is not currently available in the datacenter, the system includes an operation to create the datacenter entity, if the target configuration does not include a datacenter entity that is currently available in the datacenter, the system includes an operation to delete the existing datacenter entity, and if the target configuration includes a datacenter entity that has a different configuration than the configuration of a matching datacenter entity that is currently available in the datacenter, the system includes an operation to modify the configuration of the existing datacenter entity.

[0147] The system generates 1820 an aggregate version map that associates data center entities of the data center with versions of software artifacts subject to deployment on the data center entities, as well as versions of service pipelines used for particular services.

[0148] The system generates 1830 an aggregation pipeline based on the declarative specification and the aggregation version map. The aggregation pipeline includes a hierarchy of pipelines including pipelines for performing various operations including creating a datacenter entity, deleting a datacenter entity, modifying a configuration of a datacenter entity, etc. The aggregation pipeline is configured to create a datacenter if the datacenter does not exist and modify a configuration of a datacenter if the datacenter already exists. According to one embodiment, the system aggregates the pipelines based on the datacenter hierarchy. The system uses the version of the service pipeline determined in the aggregation version map for inclusion in the hierarchy of pipelines represented by the aggregation pipeline. The system uses the ordering of the individual pipelines, various pipeline parameters extracted from the declarative specification to build the aggregation pipeline. After ordering the service pipelines, the system aggregates all services defined for each service group (e.g., business unit or functional domain). Within the service group, the system orders the pipelines based on dependencies extracted from the declarative specification. The system further performs datacenter aggregation by ordering the service groups.

[0149] According to one embodiment, the system determines pipeline ordering and dependency information from a declarative specification. The declarative specification includes initiation dependencies that specify other datacenter entities that depend on a particular datacenter entity. For example, the declarative specification may specify a set of services that must be running for a particular service to be able to start. The system extracts these dependencies between the datacenter entities. The system may store the dependencies as a dependency graph. The declarative specification may specify a grouping of pipelines, for example, for strategic execution of the pipelines or for optimized execution of the pipelines. The system extracts information that describes the grouping of the pipelines. The declarative specification may specify ordering between the datacenter entities, for example, a datacenter entity D1 associated with a group G1 of an organization may need to be deployed before another datacenter entity D2 associated with a group G2. For example, a datacenter entity associated with a particular storage need may need to be available before any other datacenter entity is deployed. The system extracted different types of information described above, including start dependencies, pipeline grouping, and datacenter ordering, from the declarative specification and incorporated the information into the aggregate pipelines generated for the creation of the datacenter. The system determines the ordering of pipelines and services based on the dependencies extracted from the declarative specification. During this step, the system performs pipeline ordering by topologically sorting the services based on the start dependencies defined for each service. The pipelines within each service are ordered according to the metadata extracted from the declarative specification and any dependencies specified by the service owner for that particular service. In one embodiment, the pipelines are ordered in the order specified by the dependencies between the services.

[0150] The system collects 1840 a set of software artifacts according to the aggregate deployment version map. Each software artifact in the set is associated with a data center entity of the data center in which it is being created. The versions of the software artifacts in the set of software artifacts are determined based on the aggregate version map and represent best-match versions of the software artifacts applicable to a given context.

[0151] The system identifies 1850 the service pipeline with the best matching version based on the aggregate version map. The matching service pipeline is associated with the aggregation pipeline such that the service pipeline is executed at the appropriate stage of the aggregation pipeline.

[0152] The system executes 1860 an aggregation pipeline in conjunction with the aggregate version manifest to create a data center according to the declarative specification. The aggregation pipeline configures the services of the data center based on the set of software artifacts.

[0153] According to one embodiment, the system may perform multiple iterations to create a datacenter based on the declarative specification. For example, a first attempt to create a datacenter may be partially successful, whereby only a subset of the datacenter entities specified in the declarative specification are created. Thus, in a second attempt, the system compares the partially generated configuration of the datacenter with the declarative specification to determine a set of operations that need to be performed to complete the process of configuring the datacenter. If another failure occurs during the second attempt, the system performs further attempts and repeats the process until the configuration of the datacenter matches the declarative specification.

[0154] FIG. 19 illustrates a process for generating an aggregated version map according to one embodiment. The process illustrated in FIG. 19 provides details of step 1820 of the process illustrated in FIG. 18. The process illustrated in FIG. 19 illustrates how the system selects a best-match version map for a particular service instance. The process illustrated in FIG. 19 is repeated for each service on which an operation is being performed to determine a corresponding best-match version map. The system generates the aggregated version map by aggregating the individual best matches of the version maps determined for each service. In one embodiment, the system generates two aggregated version maps: a provisioning version map for provisioning a service on a data center and a deployment version map for deploying a service on a data center of a cloud platform.

[0155] The system receives (1910) a target service instance for which a best match version map should be determined and a list of input version maps. The target service instance may be specified by specifying a full path that identifies the service instance in a hierarchy of data center entities. For example, the target service instance may be specified by specifying an environment type (development, test, production, etc.), specifying a data center entity above the service instance in the hierarchy, e.g., a data center, one or more service groups (e.g., functional domain, cell, etc.), and the service instance. The output of the process illustrated in FIG. 19 is a best match version map for the target service instance.

[0156] The system identifies (1920) pipelines specified for the service instance, including provisioning pipelines and deployment pipelines. For example, a service owner may have specified multiple pipelines for provisioning and / or deploying different resources associated with the service. Examples of resources include a cluster of servers used by the service instance, a storage system used by the service instance, a database used by the service instance, a queuing service used by the service instance, a notification service used by the service instance, etc. Each resource may have a provisioning pipeline associated with it, specified by the service owner for the target service instance. For example, there may be a pipeline for deploying a database, a pipeline for deploying a schema for the database, a pipeline for deploying an application using the database, deploy code for each microservice associated with the service instance, etc. Each pipeline is identified by a unique identifier, for example, the location of the pipeline's instructions (i.e., a template) in a storage repository. The identifier of the pipeline is specified in a version map. Alternatively, a service definition may specify all the pipelines required to provision / deploy a service instance.

[0157] The system incorporates (1930) map M1 from the pipeline into a version map associated with the pipeline. For example, if a particular version map specifies a set of pipelines, then each pipeline is associated with a version map and added to map M1. Map M1 may be a hash map with keys based on the pipeline identifiers.

[0158] The system initializes 1940 a map M2, such as a hash map that maps pipelines to the current best match version map of pipelines and the match rating of the current best match pipeline. Map M2 may be initialized to an empty map data structure.

[0159] The system iterates through the various pairs of pipelines and version maps stored in the map structure M1. For each pair (pipeline P, version map V) stored in the map structure M1, the system performs steps 1950 and 1960. Thus, for a version map V, the system determines (1950) a match rating R based on various criteria specified herein. If there is no current best match version map for pipeline P stored in the map M2 so far, the version map M and the rating R are added to the map M2 for the key corresponding to the pipeline P. Assume that for pipeline P, a current best match version map Vcurrent_best is stored in the map M2 along with a match rating Rcurrent_best. In this situation, if the match rating R determined for pipeline P indicates a better match of the version map V for pipeline P compared to the match rating of the current best version map Vcurrent_best stored in M2 for pipeline P, the system replaces the entry Vcurrent_best stored in M2 for pipeline P with V and replaces the corresponding match rating Rcurrent_best with the rating R. If the match rating R determined for pipeline P indicates a worse match of the version map V for pipeline P compared to the match rating of the current best version map Vcurrent_best stored in M2 for pipeline P, the system ignores the version map V and leaves the entry Vcurrent_best stored in M2 with the corresponding match rating Rcurrent_best.

[0160] The above steps 1950 and 1960 are repeated for all pairs (pipeline, version map) stored in map M1. Once all pairs (pipeline, version map) stored in map M1 have been processed, the system generates (1970) an aggregate version map that represents the best matching version map for the corresponding pipeline based on the version maps stored in map M2. Version Map Match Criteria

[0161] The system determines the match evaluation of the version map of the service instance as follows: A service instance is identified by specifying the data center entities of the hierarchy in the path from the root of the hierarchy to the service instance. For example, the path of the service instance may include a data center instance (DI), a service group instance (SG, e.g., a functional domain), a cell (CELL), and a service instance (SI). In addition, for a particular operation, the system specifies the type of environment (ENV). As an example, a target service instance path object may list the identifiers of each data center entity in the path and the environment as follows: In the following description, the environment is also treated as a data center entity. Thus, the target service instance path object is represented as a series of data center entities, with each data center entity corresponding to a level of the hierarchy along the path from the root to the service instance.

number

[0162] The system determines a match object for each version map. The match object specifies the type of match for each attribute of the target service instance path object. If the version map specifies the exact name of the data center entity, the type of match is an exact match ("exact"). For example, if the data center entity SI is "test_service" in the above example and the version match is applicable to the SI "test_service", the SI match is "exact". Another type of match is a regular expression match ("regex") if the version map specifies a regular expression and the data center entity of the target service instance path object matches the regular expression. For example, if the data center entity SI is "test_service" and the version match is applicable to the SI "test_*", the SI match is "regex".

[0163] Another type of match is the default match ("default"). The service owner specifies a default version map to use when there is no exact or regular expression match. If the default version map is used for a particular data center entity, the match is "default".

[0164] Another type of match is first_match: if there is no match, the system takes the first defined target in the best matching version map file.

[0165] Below is an example match object for the example target service instance above.

number

[0166] Other embodiments may use a numerical value for each type of match. The priority of the match version map is determined as a weighted aggregation of the priority values ​​of each data center entity, with the weight depending on the level of the data center entity in the path. Thus, the environment (ENV) has the highest weight, the data center instance (DI) has the next highest weight, the service group (SG) has the next highest weight, the cell (CELL) has the next highest weight, and the service instance (SI) has the lowest weight. Each type of match is assigned a match score such that an exact match has the highest score, a regular expression match has the next highest score, a default match has the next highest score, and the first defined has the lowest score. The respective match scores of the data center entities are weighted and summed as described above to determine the match rating of the version map.

[0167] Below is an example service metadata configuration file that specifies three version map files under the vmfPaths attribute: test-vmf.yaml, stage-vmf.yaml, and prod-vmf.yaml.

number

[0168] Below is an example version map file for use in a test environment.

number

[0169] The above example provides the pipeline version and artifact version for the exact datacenter entity name. Below is another example of a version map for use in a production environment:

number

[0170] Thus, the embodiments provide several advantages for performing operations on a cloud platform. Rather than requiring users to manually create new template version maps, the system automates the process of selecting the correct target when generating a version map, saving time and reducing the chance of error. The system allows users to pass in their existing version maps, eliminating the need for users to manage two copies of the same information, resulting in only one source of truth. Additionally, storing only one copy of the version map improves storage efficiency. The system allows services with complex multiple version map structures to make changes to a portion of their versions without affecting other version maps for different targets. Thus, the system enables modular development of version maps and related code for deployment and provisioning using a cloud platform. Computer Architecture

[0171] 20 is a high level block diagram illustrating a functional view of a typical computer system for use as one of the entities depicted in the environment 100 of FIG. 1, according to one embodiment. Shown is at least one processor 2002 coupled to a chipset 2004. Also coupled to the chipset 2004 are memory 2006, a storage device 2008, a keyboard 2010, a graphics adapter 2012, a pointing device 2014, and a network adapter 2016. A display 2018 is coupled to the graphics adapter 2012. In one embodiment, the functionality of the chipset 2004 is provided by a memory controller hub 2020 and an I / O controller hub 2022. In another embodiment, the memory 2006 is coupled directly to the processor 2002 instead of the chipset 2004.

[0172] The storage device 2008 is a non-transitory computer-readable storage medium such as a hard drive, a compact disc read only memory (CD-ROM), a DVD, or a solid-state memory device. The memory 2006 holds instructions and data used by the processor 2002. The pointing device 2014 may be a mouse, a trackball, or other type of pointing device and is used in combination with the keyboard 2010 to input data into the computer system 2000. The graphics adapter 2012 displays images and other information on the display 2018. The network adapter 2016 couples the computer system 2000 to a network.

[0173] As is known in the art, computer 2000 may have different and / or other components than those shown in Figure 20. Additionally, computer 2000 may lack certain illustrated components. For example, a computer system 2000 functioning as multi-tenant system 110 may lack keyboard 2010 and pointing device 2014. Additionally, storage device 2008 may be local and / or remote from computer 2000 (such as embodied in a storage area network (SAN)).

[0174] The computer 2000 is adapted to execute computer modules for providing the functionality described herein. As used herein, the term "module" refers to computer program instructions and other logic for providing a specified functionality. A module may be implemented in hardware, firmware, and / or software. A module may include one or more processes and / or may be provided by only a portion of a process. A module is typically stored in a storage device 2008, loaded into memory 2006, and executed by the processor 2002.

[0175] The type of computer system 2000 used by the entities of the system environment may vary depending on the embodiment and the processing power used by the entities. For example, a client device may be a mobile phone with limited processing power and a small display 2018, and may lack a pointing device 2014. A multi-tenant system or cloud platform, in contrast, may comprise multiple blade servers that work together to provide the functionality described herein. Further considerations

[0176] The particular naming of components, capitalization of terms, attributes, data structures, or any other programming or structural aspects are not required or important, and mechanisms for implementing the described embodiments may have different names, formats, or protocols. Furthermore, the system may be implemented through a combination of hardware and software, or entirely within hardware elements, as described. Also, the particular division of functionality between various system components described herein is merely exemplary and not required. Functions performed by a single system component may be instead performed by multiple components, and functions performed by multiple components may be instead performed by a single component.

[0177] Some portions of the above description are presented in terms of algorithms and symbolic representations of operations on information. These algorithmic descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. It will be understood that these operations, while described in functional or logical terms, are implemented by computer programs. Further, it has proven convenient at times to refer to the arrangement of these operations as modules or by functional names, without loss of generality.

[0178] As is apparent from the above description, unless otherwise indicated, throughout the description, descriptions utilizing terms such as "processing" or "computing" or "calculating" or "determining" or "displaying" will be understood to refer to the actions and processes of a computer system or similar electronic computing device that manipulates and transforms data represented as physical (electronic) quantities in the computer system memory or registers or other such information storage, transmission, or display device.

[0179] Certain embodiments described herein include process steps and instructions that are described in the form of an algorithm. It should be noted that the process steps and instructions of the embodiments may be embodied in software, firmware, or hardware, and if embodied in software, may be downloaded to reside on and be operated from different platforms used by a real-time network operating system.

[0180] The described embodiments also relate to an apparatus for performing the operations herein. The apparatus may be specially constructed for the required purposes or may comprise a general-purpose computer selectively activated or reconfigured by a computer program stored on a computer-readable medium accessible by the computer. Such computer programs may be stored on non-transitory computer-readable storage media, such as, but not limited to, any type of disk including floppy disks, optical disks, CD-ROMs, magneto-optical disks, read-only memory (ROM), random access memory (RAM), EPROM, EEPROM, magnetic or optical cards, application specific integrated circuits (ASICs), or any type of medium suitable for storing electronic instructions, each coupled to a computer system bus. Furthermore, the computers referred to herein may include a single processor or may be architectures using multiple processor designs to increase computing power.

[0181] The algorithms and operations presented herein are not inherently related to any particular computer or other apparatus. Various general-purpose systems may also be used with programs according to the teachings herein, or it may prove convenient to construct more specialized apparatus to perform the required method steps. The required structure for a variety of these systems, along with equivalent variations, will be apparent to those skilled in the art. In addition, the present embodiments are not described with reference to any particular programming language. It will be understood that a variety of programming languages ​​can be used to implement the teachings of the embodiments described herein.

[0182] Embodiments are well suited to a wide variety of computer network systems across numerous topologies, including configuration and management of large networks including storage devices and computers communicatively coupled to different computers and storage devices over a network such as the Internet.

[0183] Finally, it should be noted that the language used herein has been chosen primarily for ease of reading and educational purposes, and may not have been chosen to delineate or limit the subject matter of the invention. Accordingly, the disclosure of the embodiments is intended to be illustrative, not limiting.< / version> < / service>

Claims

1. 1. A computer-implemented method for configuring a data center on a cloud platform, comprising: receiving a cloud platform agnostic declarative specification for creating a data center on a cloud platform, the data center including a hierarchy of data center entities, each data center entity including one or more of: (1) services; or (2) one or more other data center entities; generating an aggregation pipeline comprising a hierarchy of pipelines, the hierarchy of pipelines comprising pipelines for creating data center entities of the data center, the aggregation pipelines configured to create the data center; generating an aggregate deployment version map relating data center entities of the data center to versions of software artifacts subject to deployment on the data center entities; aggregating a set of software artifacts according to the aggregate deployment version map, the software artifacts being associated with data center entities of the data center in which they are being created; executing the aggregation pipeline in conjunction with the aggregate deployment version map to create the data center according to the cloud platform agnostic declarative specification, the aggregation pipeline configuring services based on the set of software artifacts; 23. A computer-implemented method comprising:

2. The declarative specification describes dependencies between a particular service and one or more other services that are required for execution of the particular service, and the method further comprises: determining an order of execution of pipelines associated with services based on the dependencies between services specified in the declarative specification, wherein the aggregation pipeline arranges the pipelines according to the determined order. The computer-implemented method of claim 1 , further comprising:

3. performing a topological sort of services within a data center entity based on dependencies between the services, wherein an order of pipelines within the aggregation pipeline is determined based on the order determined by the topological sort; The computer-implemented method of claim 2 , further comprising:

4. assigning an identifier to each pipeline stage included in the aggregation pipeline; storing a status of execution of a stage of a pipeline, said status of a stage being associated with a stage identifier of said stage; The computer-implemented method of claim 1 , further comprising:

5. determining a failure of execution of the aggregation pipeline; resuming execution of the aggregation pipeline, the resuming including skipping execution of a stage of the pipeline in response to determining, based on the stored status of the stage, that the execution of the stage was previously completed; The computer-implemented method of claim 4 , further comprising:

6. Providing users with access to the services running in the data center deployed on the cloud platform. The computer-implemented method of claim 4 , further comprising:

7. The step of generating an aggregate deployment version map includes: receiving one or more service deployment version maps, the service deployment version maps being associated with a service; determining the aggregated deployment version map for the data center by aggregating the one or more service deployment version maps; The computer-implemented method of claim 1 , comprising:

8. 2. The computer-implemented method of claim 1 , wherein the cloud platform independent declarative specification includes a definition of one or more data center instances, the data center instances including one or more service groups, the service groups including a set of services.

9. 1. A non-transitory computer-readable storage medium for storing instructions that, when executed by a computer processor, cause the computer processor to perform steps for deploying software artifacts for services that run in a data center configured on a cloud platform, the steps comprising: receiving a cloud platform agnostic declarative specification for creating a datacenter on a cloud platform, the datacenter including a hierarchy of datacenter entities, each datacenter entity including one or more of: (1) services; or (2) one or more other datacenter entities; generating an aggregation pipeline including a hierarchy of pipelines, the hierarchy of pipelines including a pipeline for creating a data center entity of the data center, the aggregation pipeline configured to create the data center; generating an aggregate deployment version map relating data center entities of the data center to versions of software artifacts subject to deployment on the data center entities; collecting a set of software artifacts according to the aggregate deployment version map, the software artifacts being associated with data center entities of the data center in which they are being created; executing the aggregation pipeline in conjunction with the aggregate deployment version map to create the data center according to the cloud platform agnostic declarative specification, the aggregation pipeline configuring services based on the set of software artifacts; A non-transitory computer readable storage medium comprising:

10. The declarative specification describes dependencies between a particular service and one or more other services that are necessary for execution of the particular service, and the instructions further cause the processor to: determining an order of execution of pipelines associated with services based on the dependencies between services specified in the declarative specification, wherein the aggregation pipeline arranges the pipelines according to the determined order.

10. The non-transitory computer-readable storage medium of claim 9, further comprising:

11. The instructions further cause the processor to: performing a topological sort of services within a data center entity based on dependencies between the services, wherein an order of pipelines within the aggregation pipeline is determined based on the order determined by the topological sort. The non-transitory computer-readable storage medium of claim 10, further comprising:

12. The instructions further cause the processor to: assigning an identifier to each pipeline stage included in the aggregation pipeline; storing a status of execution of a stage of a pipeline, the status of a stage being associated with a stage identifier of the stage; 10. The non-transitory computer-readable storage medium of claim 9, further comprising:

13. The instructions further cause the processor to: determining a failure of execution of the aggregation pipeline; resuming execution of the aggregation pipeline, the resuming including skipping execution of a stage of the pipeline in response to determining, based on the stored status of the stage, that the execution of the stage was previously completed; 13. The non-transitory computer-readable storage medium of claim 12, further comprising:

14. The instructions further cause the processor to: Providing users with access to the services running in the data centers deployed on the cloud platform.

10. The non-transitory computer-readable storage medium of claim 9, further comprising:

15. The instructions for generating the aggregate deployment version map may include instructions for the processor to: receiving one or more service deployment version maps, the service deployment version maps being associated with a service; determining the aggregated deployment version map for the data center by aggregating the one or more service deployment version maps; 10. The non-transitory computer-readable storage medium of claim 9, further comprising:

16. 10. The non-transitory computer-readable storage medium of claim 9, wherein the cloud platform independent declarative specification includes a definition of one or more data center instances, each data center instance including one or more service groups, each service group including a set of services.

17. 1. A computer system comprising: A computer processor; a non-transitory computer-readable storage medium for storing instructions; the instructions, when executed by the computer processor, cause the computer processor to perform steps for configuring a data center in a cloud platform, the steps comprising: receiving a cloud platform agnostic declarative specification for creating a datacenter on a cloud platform, the datacenter including a hierarchy of datacenter entities, each datacenter entity including one or more of: (1) services; or (2) one or more other datacenter entities; generating an aggregation pipeline including a hierarchy of pipelines, the hierarchy of pipelines including a pipeline for creating a data center entity of the data center, the aggregation pipeline configured to create the data center; generating an aggregate deployment version map relating data center entities of the data center to versions of software artifacts subject to deployment on the data center entities; collecting a set of software artifacts according to the aggregate deployment version map, the software artifacts being associated with data center entities of the data center in which they are being created; executing the aggregation pipeline in conjunction with the aggregate deployment version map to create the data center according to the cloud platform agnostic declarative specification, the aggregation pipeline configuring services based on the set of software artifacts; 2. A computer system comprising:

18. The declarative specification describes dependencies between a particular service and one or more other services that are necessary for execution of the particular service, and the instructions further cause the processor to: determining an order of execution of pipelines associated with services based on the dependencies between services specified in the declarative specification, wherein the aggregation pipeline arranges the pipelines according to the determined order.

20. The computer system of claim 17, configured to perform the steps including:

19. The instructions further cause the processor to: performing a topological sort of services within a data center entity based on dependencies between the services, wherein an order of pipelines within the aggregation pipeline is determined based on the order determined by the topological sort.

20. The computer system of claim 18, configured to perform the steps comprising:

20. The instructions further cause the processor to: assigning an identifier to each pipeline stage included in the aggregation pipeline; storing a status of execution of a stage of a pipeline, the status of a stage being associated with a stage identifier of the stage; 20. The computer system of claim 17, configured to perform the steps including:

21. 1. A computer-implemented method for performing operations on a cloud platform, comprising: receiving a request to perform an operation on a data center configured on the cloud platform, the data center associated with a set of services that run on the cloud platform; receiving, for a particular service from the set of services, a number of version maps, each version map providing version information for a particular context associated with the data center; creating an aggregate version map for the requested operation, the creating step including selecting a particular version map from the plurality of version maps for the particular service, the selection being based on relevance of the particular version map to a current context for the requested operation; generating an aggregation pipeline based on the aggregation version map; collecting a set of software artifacts according to the aggregate version map; executing the aggregation pipeline to perform the requested operation for the data center, the aggregation pipeline performing the requested operation based on the set of software artifacts; 23. A computer-implemented method comprising:

22. receiving a declarative specification describing a target configuration of data center entities for the data center; accessing said data center to extract information describing data center entities currently available at said data center; comparing the information extracted from the data center with the declarative specification to identify the data center entities on which the action is requested to be performed; 22. The computer-implemented method of claim 21, further comprising:

23. determining a failure of execution of the aggregation pipeline; resuming execution of the aggregation pipeline, the resuming including skipping execution of a stage of the pipeline in response to determining, based on the extracted information of the data center, that execution of the stage was previously completed; 23. The computer-implemented method of claim 22, further comprising:

24. The particular context is Target environment, Target Data Center Entity, or Target actions to be performed on the cloud platform 22. The computer-implemented method of claim 21, further comprising specifying one or more of:

25. The action requested to be performed on the data center is: provisioning software artifacts on said cloud platform; Deploying software artifacts on said cloud platform; or Patching software artifacts on said cloud platform.

22. The computer-implemented method of claim 21, wherein:

26. 22. The computer-implemented method of claim 21, wherein the version map specifies one or more of a version of a pipeline to be used and a version of a software artifact to be used for the particular context, the particular context being represented using one or more of a target environment, a target data center entity, and a target action to be performed on the cloud platform.

27. 22. The computer-implemented method of claim 21, wherein the data center includes a hierarchy of data center entities, each data center entity including one or more of: (1) a service; or (2) one or more other data center entities.

28. performing a topological sorting of services in the data center entity based on dependencies; configuring the data center on a cloud platform based on a cloud platform independent declarative specification; generating an aggregation pipeline including a hierarchy of pipelines, the hierarchy of pipelines configured to execute the operations on a data center configured on the cloud platform; 22. The computer-implemented method of claim 21, further comprising:

29. 1. A non-transitory computer-readable storage medium for storing instructions that, when executed by a computer processor, cause the computer processor to perform steps for deploying software artifacts for services that run in a data center configured on a cloud platform, the steps comprising: receiving a request to perform an operation on a data center configured on the cloud platform, the data center being associated with a set of services that execute on the cloud platform; receiving, for a particular service from the set of services, a plurality of version maps, each version map providing version information for a particular context associated with the data center; creating an aggregate version map for the requested operation, the creating including selecting a particular version map from the plurality of version maps for the particular service, the selection being based on a relevance of the particular version map to a current context for the requested operation; generating an aggregation pipeline based on the aggregation version map; aggregating a set of software artifacts according to the aggregate version map; executing the aggregation pipeline to perform the requested operation for the data center, the aggregation pipeline performing the requested operation based on the set of software artifacts; A non-transitory computer readable storage medium comprising:

30. The instructions further cause the processor to: receiving a declarative specification describing a target configuration of data center entities for the data center; accessing said data center to extract information describing data center entities currently available at said data center; comparing the information extracted from the data center with the declarative specification to identify the data center entities on which the action is requested to be performed; 30. The non-transitory computer-readable storage medium of claim 29, further comprising:

31. The particular context is Target environment, Target Data Center Entity, or Target actions to be performed on the cloud platform 30. The non-transitory computer-readable storage medium of claim 29, further comprising:

32. The action requested to be performed on the data center is: provisioning software artifacts on said cloud platform; Deploying software artifacts on said cloud platform; or Patching software artifacts on said cloud platform.

30. The non-transitory computer readable storage medium of claim 29, wherein:

33. 33. The non-transitory computer-readable storage medium of claim 32, wherein the version map specifies one or more of a version of a pipeline to be used and a version of a software artifact to be used for the particular context, the particular context being represented using one or more of a target environment, a target data center entity, and a target action to be performed on the cloud platform.

34. 30. The non-transitory computer-readable storage medium of claim 29, wherein the data center includes a hierarchy of data center entities, each data center entity including one or more of: (1) a service; or (2) one or more other data center entities.

35. 30. The non-transitory computer-readable storage medium of claim 29, wherein the data center includes a hierarchy of data center entities, each data center entity including one or more of: (1) a service; or (2) one or more other data center entities.

36. The instructions further cause the processor to: performing a topological sort of services within a data center entity based on dependencies; Configuring the data center on a cloud platform based on a cloud platform independent declarative specification; generating an aggregation pipeline including a hierarchy of pipelines, the hierarchy of pipelines configured to execute the operations on a data center configured on the cloud platform; 30. The non-transitory computer-readable storage medium of claim 29, further comprising:

37. 1. A computer system comprising: A computer processor; a non-transitory computer-readable storage medium for storing instructions; the instructions, when executed by the computer processor, cause the computer processor to perform steps for configuring a data center in a cloud platform, the steps comprising: receiving a request to perform an operation on a data center configured on the cloud platform, the data center being associated with a set of services that execute on the cloud platform; receiving, for a particular service from the set of services, a plurality of version maps, each version map providing version information for a particular context associated with the data center; creating an aggregate version map for the requested operation, the creating including selecting a particular version map from the plurality of version maps for the particular service, the selection being based on a relevance of the particular version map to a current context for the requested operation; generating an aggregation pipeline based on the aggregation version map; aggregating a set of software artifacts according to the aggregate version map; executing the aggregation pipeline to perform the requested operation for the data center, the aggregation pipeline performing the requested operation based on the set of software artifacts; 2. A computer system comprising:

38. The instructions further cause the processor to: receiving a declarative specification describing a target configuration of data center entities for the data center; accessing said data center to extract information describing data center entities currently available at said data center; comparing the information extracted from the data center with the declarative specification to identify the data center entities on which the action is requested to be performed; 38. The computer system of claim 37, configured to perform the steps including:

39. The particular context is Target environment, Target Data Center Entity, or Target actions to be performed on the cloud platform 40. The computer system of claim 38, further comprising:

40. The action requested to be performed on the data center is: provisioning software artifacts on said cloud platform; Deploying software artifacts on said cloud platform; or Patching software artifacts on said cloud platform.

38. The computer system of claim 37, wherein:

Images