Flexible continuous integration and continuous delivery system

Abstract

Examples include a system that receives job configuration information for execution of a process including a plurality of stages, each stage including a plurality of sequences, each sequence including a plurality of pipelines, each pipeline including a plurality of operations to be executed sequentially. The plurality of sequences of a first stage are executed in parallel, with each sequence corresponding to a respective resource group, each respective resource group including a copy of resources from a base resource group specified by the job configuration information. Following completion of execution of the plurality of sequences of the first stage, the plurality of sequences of a next stage are executed, such that the plurality of stages are executed one at a time in a specified order until execution of the plurality of stages is completed. Following completion of execution of the stages, results of the process are sent to a computing device.

Description

FLEXIBLE CONTINUOUS INTEGRATION AND CONTINUOUS DELIVERY SYSTEMTECHNICAL FIELD

[0001] This disclosure relates to the technical field of building, testing, and / or deploying software.BACKGROUND

[0002] CI / CD is a combination of continuous integration (CI) and continuous delivery (CD) (also referred to as “continuous deployment”). For example “continuous integration” may include frequent merging of several small changes to a software component into a main branch. Further, “continuous delivery” may include the practice of producing software in short cycles with high frequency so that reliable software can be released at any time, such as via a simple and repeatable deployment process. In some cases, CI / CD may be used by development teams to build and run portions of a software product, such as locally for development and qualification purposes, and / or in a lab environment, such as for qualification and characterization purposes. Using CI / CD as part of software product development may include a number of stages such as initializing the environment, building desired components, testing each component, and / or tearing down the environment. However, conventional CI / CD systems may lack a desired level of flexibility.SUMMARY

[0003] In some implementations, a system may receive job configuration information for execution of a process including a plurality of stages, each stage including a plurality of sequences, each sequence including a plurality of pipelines, each pipeline including a plurality of data processing elements to be executed sequentially. The system executes the process by executing, in parallel, the plurality of sequences of a first stage of the plurality of stages, such that each of the sequences of the first stage corresponds to a respective resource group, each respective resource group including a copy of resources from a base resource group specified by the job configuration information, and where the copy of the resources includes the plurality of pipelines for each sequence. Following completion of execution of the plurality of pipelines of the plurality of sequences of the first stage, the system executes, in parallel, the plurality of sequences of a next stage of the plurality of stages. Thus, the plurality of stages may be executed one at a time in a specified order until execution of the plurality of stages is completed. The system may send a result of the process to a computing device.BRIEF DESCRIPTION OF THE DRAWINGS

[0004] The detailed description is set forth with reference to the accompanying figures. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. The use of the same reference numbers in different figures indicates similar or identical items or features.

[0005] FIG. 1 illustrates an example architecture and logical configuration of a system configured to execute CI / CD in a containerized application environment according to some implementations .

[0006] FIG. 2 illustrates a logical arrangement including details of some portions of the example system of FIG. 1 according to some implementations.

[0007] FIG. 3 illustrates an example of a cleanup process that may be performed according to some implementations.

[0008] FIG. 4 illustrates an example architecture of a computing system in which the examples of FIGS. 1-3 may be realized according to some implementations.

[0009] FIG. 5 illustrates select example components of one or more service computing devices that may be used to implement some of the functionality of the systems described herein.

[0010] FIG. 6 illustrates select example components of a user device according to some implementations .DESCRIPTION OF THE EMBODIMENTS

[0011] Some implementations herein are directed to techniques and arrangements for a flexible CI / CD system capable of handling a development environment as well as handling periodic builds for testing and deployment. Implementations further include the ability to clean up consistently after execution regardless of the outcome of a CI / CD task. In some examples, the CI / CD system may operate within a containerized application framework, such as a KUBERNETES® system. KUBERNETES® is a well-known platform for running containerized applications, which is available from the Cloud Native Computing Foundation of San Francisco, California, USA. Accordingly, a KUBERNETES® platform is an example of a platform on which the build operations herein may take place. Some examples herein provide a flexible and efficient CI / CD system in a KUBERNETES® architecture capable of handling a complex development environment; however, implementations herein are not limited to use with KUBERNETES® systems.

[0012] Some examples herein include a collection of tools that may be used to initialize, build, deploy, test, clean up, and tear down a software product. For instance, some examples include integration of an open-source CI / CD pipeline management program for creating a CI / CD system that enables developers to build, test, and deploy software across cloud providers and on-premise systems. Examples of such a CI / CD pipeline management program may include TEKTON™, an open-source framework for building CI / CD systems, and which is part of the CD Foundation, a LINUX FOUNDATION® project. The CI / CD pipeline management program herein may be used to execute pipeline operations related to building, deploying, testing, cleaning, and tearing down software products, such as applications, application programming interfaces (APIs), or other software subject to CI / CD operations. For example, each pipeline may define a set of custom resources that may be used as building blocks for performing CI / CD operations in a pipeline framework. Each pipeline may be executed as a series of ordered data processing elements (also referred herein to as operations or tasks). A data processing element in a pipeline may typically use the output of a previously executed pipeline data processing element as its input until the end of the pipeline is reached.

[0013] In addition to using one or more open-sourced tools, examples herein include a job execution application that is configured to execute stages of the CI / CD process. The job execution application is run as a deployment within the containerized application framework. Additionally, examples herein may include a client program that is configured as a client to the job execution application 126. The client program may be configured to start processes of the job execution application 126. The client program may be further configured to display, to a user that executes the client program, logs related to the processes run by the job execution application. As one example, the client program may be started and controlled by a client control program (e.g., a wrapper script) and may be executed as a job in the containerized application framework. In addition, a cleanup program may be executed following running of one or more processes by the job execution application. For instance, the cleanup program may be configured to clean up the execution environment, such as by deleting various resources that have been annotated by the job execution application for deletion. The cleanup program may be executed as a job in the containerized application environment and may be scheduled as a repeating job (e.g., a CronJob or the like), or may be executed on demand, such as by a user.

[0014] For discussion purposes, some example implementations are described in the environment of a job execution application that executes CI / CD in a containerized application environment, and that includes the execution of pipelines for building, deploying, testing, cleaning, and tearing down software. However, implementations herein are not limited to theparticular examples provided, and may be extended to other types of computing system architectures, other hardware configurations, other types of software and logical configurations, other types of user interaction, and so forth, as will be apparent to those of skill in the art in light of the disclosure herein.

[0015] FIG. 1 illustrates an example architecture and logical configuration of a system 100 configured to execute CI / CD in a containerized application environment according to some implementations. The system 100 includes a plurality of service computing devices 102 that are able to communicate with one or more client computing devices 104, such as through one or more networks 106. In some examples, the service computing devices 102 and / or the client computing device(s) 104 are also able to communicate over the network(s) 106 with one or more user devices 108. For example, the user device 108 may be used for communicating with and configuring the service computing devices 102 and / or the client computing device(s) 104, such as for executing CI / CD jobs. The user devices 108 may be any of various types of computing devices, as discussed additionally below. A user 112 such as a client user and / or an administrative user may be associated with a respective user device 108, such as through a user application 114 or the like. In some examples, the user application 114 may be a browser, while in other examples, the user application 1 14 may be a proprietary application configured to communicate with the client computing device(s) 104 and / or the service computing devices 102. Further, in some examples, the client computing device(s) 104 may be the same device as a user device 108.

[0016] The one or more networks 106 may include any suitable network, including a wide area network, such as the Internet; a local area network (LAN), such as an intranet; a wireless network, such as a cellular network, a local wireless network, such as Wi-Fi, and / or short-range wireless communications, such as BLUETOOTH®; a wired network including Fibre Channel, fiber optics, Ethernet, or any other such network, a direct wired connection, or any combination thereof. Accordingly, the one or more networks 106 may include both wired and / or wireless communication technologies. Components used for such communications can depend at least in part upon the type of network, the environment selected, or both. Protocols for communicating over such networks are well known and will not be discussed herein in detail. Implementations herein are not limited to any particular type of network as the networks 106.

[0017] FIG. 1 further includes an example flow of the system 100, which may start with a client control program 116 that may be executed on the client computing device(s) 104 for invoking execution of a CI / CD process on the service computing devices 102. For instance, the client control program 116 on the client computing device(s) 104 may be a wrapper script, web app, or the like, for communicating with and instructing a client program 118 that is executableon the service computing devices 102. The client control program 116 may send one or more communications to initiate a new CI / CD job that will be executed on the service computing devices 102. For example, the client control program 116 may initiate the new CI / CD job in response to receiving an instruction from one of the user devices 108, or based on some other triggering event, such as receipt of software 120 from one or more user devices 108, receipt of software 120 or an indication of software for processing from a repository computing device (not shown in FIG. 1), or so forth. In addition to sending, to the service computing devices 102, an instruction 122 to start a CI / CD job, the client control program 116 may also send job configuration information 124 that provides information related to the CI / CD process that the client control program 116 is requesting to be executed. Example details of the job configuration information 124 sent by the client control program 116 to the client program 118 are described additionally below.

[0018] In response to receiving, from the client computing device 104, the one or more communications including the instruction 122 to start the CI / CD job and the job configuration information 124, the client program 118 may send an API call, such as a CI / CD job call 128, to the job execution application 126 (or the client program 118 may send another type of communication to achieve the same effect). The CI / CD job call 128 may include, or may otherwise provide access to, the job configuration information 124 received from the client computing device 104, and may instruct the job execution application 126 to start a CI / CD job on the service computing devices 102 according to the job configuration information 124.

[0019] After the client program 118 sends the CI / CD job call 128 and the job configuration information 124 as a request to the job execution application 126, the job execution application 126 may start a new CI / CD process 130, details of which are illustrated in FIG. 1 as a CI / CD job run 132. The job execution application 126 may start the CI / CD job run 132 and may then report back to the client program 1 18 that the CI / CD job run 132 has started. Alternatively, if an error occurs, the CI / CD job run 132 stops and / or does not start. An example of an error that might prevent the CI / CD job run 132 from running is that the job configuration information 124 refers to a pipeline that does not exist in a base resource group (rsrc grp) 134 specified by the job configuration information 124.

[0020] After the CI / CD job run 132 has been initiated, in some examples, the client program 118 may send status and logging information 136 to the client control program 116 on the client computing device 104 and / or to the user device 108. For instance, the client program 118 may begin to record the status and logging information 136 after the CI / CD job run 132 has started. Additionally, as one example, the client control program 116 may provide or may present thestatus and logging information 136 to a user 112 who caused the client control program 116 to initiate the CI / CD job. This also enables the user 112 to be informed as to when the CI / CD job run 132 has ended execution.

[0021] When the job execution application 126 starts the CI / CD job run 132, the job execution application 126 first validates the job configuration information 124 provided as part of the CI / CD job call 128. The validation performed by the job execution application 126 may include verifying existence of the base resource group 134 specified in the job configuration information 124, and may further include verifying existence of each pipeline 140 specified in the job configuration information 124. If either of those conditions is false, an error is returned and the CI / CD job run 132 stops. Otherwise, if both conditions are true, an acknowledgement is sent to the client program 118, and execution of the CI / CD job run 132 proceeds.

[0022] After the client program 1 18 receives the acknowledgement that the CI / CD job run 132 has started, the client program 118 may begin monitoring the CI / CD job run 132 using a status endpoint provided by the job execution application 126. One of the response elements of the status endpoint is an indication of which pipeline 140 is currently running. The client program 118 may follow and / or store the logs for the current pipeline and may output the logs as part of the status and logging information 136 sent to the client computing device 104 and / or to the user device 108.

[0023] In the illustrated example, the base resource group 134 includes the pipelines 140 that can be used with the base resource group 134, and further includes one or more of credentials, tokens, configuration data, and / or other resources 142. Examples of credentials and tokens may include database credentials, repository credentials, and the like, and tokens that may be used during execution of a pipeline, such as for accessing data or other information. One example of a token may include a token that is used for interacting with an external test tracking system that can be used to publish test results, e.g., by sending information out of a run. Numerous other types of tokens and credentials will be apparent to those of skill in the art having the benefit of the disclosure herein. Furthermore, the configuration data may include the configuration and arrangement of data to be processed, data locations, data repositories, or the like.

[0024] As indicated at 144, select resources of the base resource group 134, such as pipelines 140, credentials, tokens, configuration data, and other resources 142, may be copied into run resource groups 146 that are used in the CI / CD job run 132. In this example, the CI / CD job run 132 may be executed as a series of stages 150, each of which may include one or more sequences 152 of pipeline runs 154. For example, as indicated at 156, a CI / CD pipeline management program 158, such as TEKTON™, may manage the pipelines 140 to generate the pipeline runs 154 for the respective sequences 152. Additionally, the CI / CD job run 132 may use one or more resources forperforming the CI / CD process, including one or more of storage space, authorizations, and / or other resources 160. For example, each pipeline run 154 may include execution of a plurality of ordered data processing elements, and the data processing elements of a pipeline run 154 may use one or more storage spaces, such as for sharing a filesystem, or the like, which may include receiving, storing, and providing data used by the data processing elements of a pipeline run 154.

[0025] Once the CI / CD job run 132 executed by the job execution application 126 is no longer active (e.g., has completed, failed, etc.), the client program 118 may stop following / recording the pipeline logs. Further, the client program 118 may report the overall result of the CI / CD job run 132 to the client control program 116 and / or the user device 108 based on the status endpoint output provided by the job execution application 126.

[0026] FIG. 2 illustrates a logical arrangement 200 including details of some portions of the example system 100 of FIG. 1 according to some implementations. As mentioned above, the job configuration information 124 may be used by the job execution application 126 to start the CI / CD job run 132. The base resource group 134 is a source of the pipelines 140 and other related configuration information and resources. When the job execution application 126 creates a new resource group 146 to run as part of the CI / CD job run 132, the job execution application 126 copies data processing elements, pipelines, tagged credentials, tokens, and tagged configuration data from the base resource group 134 into the run resource group 146.

[0027] Additionally, the CI / CD job run 132 may include execution of a series of stages 150 in a prescribed sequence. For example, a later stage 150 is generally not started until a previous stage 150 has completed execution. The order in which the stages 150 are defined in the job configuration information 124 defines an order in which the stages 150 will be executed during execution of the CI / CD job run 132.

[0028] In addition, each stage 150 may contain a set of sequences 152. The sequences 152 included within the same stage 150 may be executed in parallel. Consequently, the order in which sequences 152 are defined within a particular stage 150 does not correspond to an order of execution. Additionally, each sequence 152 may correspond to one or more pipelines 140 that may be executed as pipeline runs 154. Pipeline runs 154 within a sequence 152 may be run in the order in which the pipelines are declared in the job configuration information 124. A pipeline run 154 declared within the job configuration information 124 refers to an existing pipeline 140 within the base resource group 134. If a specified pipeline does not exist in the base resource group 134, the CI / CD job run 132 typically may return an error and will not start until the error is resolved. In some examples, a pipeline 140 may be labeled to control aspects of how that pipeline 140 is executed in a sequence 152.

[0029] The environment may be used to control aspects of how the pipeline runs 154 are executed. For example, user and group identifiers (IDs) to use when running pipelines may be specified as part of the environment of the CI / CD job run 132 and may differ between the CI / CD job runs 132 initiated by a human user as opposed to this initiated by an automated process.

[0030] Furthermore, run parameters 202 may be used in some examples to specify parameters to pipeline runs 154 if the default values for those pipeline runs 154 are not appropriate. A typical use for the run parameters 202 may be to configure a data processing element in a pipeline run 154 as part of a build step, such as for configuring how the data processing element behaves when the data processing element is executed.

[0031] When running the stages 150, the sequences 152 defined for a particular stage 150 can all be run in parallel. After all the sequences 152 for a particular stage 150 have completed running, the stage 150 is complete. If any sequence 152 in a stage 150 experiences an error, then the stage 150 ends execution, no further stages 150 are run, and the status of the corresponding CI / CD job run 132 may be set to “failure” or equivalent. On the other hand, if all sequences 152 in a particular stage 150 are completed successfully, then the stage 150 completes and the CI / CD job run 132 then moves on to execution of the next stage 150, repeating in this manner until all stages 150 have been executed. After the final stage 150 is completed successfully, then the overall result of the CI / CD job run 132 is set to “success” or equivalent, and the CI / CD job run 132 ends.

[0032] After the CI / CD job run 132 ends, any storage space requests created by the sequences 152 in the run may be annotated or otherwise marked for cleanup. When a storage space request is created by a sequence 152, the storage space may be expanded upon in the running of the sequence 152 and may be used, for example, by pipeline data processing elements of pipelines run in the sequence 152, such as for storing or passing data.

[0033] When running each sequence 152, the pipeline runs 154 defined in that sequence 152 may be run one at a time according to a specified order. After all the pipeline runs 154 have completed for a particular sequence 152, the particular sequence 152 is completed. If any pipeline run 154 experiences an error, then the corresponding sequence 152 completes with an error. Other sequences 152 running concurrently with the sequence that experienced the error may continue to run to completion. If all pipeline runs 154 for a sequence 152 are completed successfully, then the sequence 152 completes successfully. After all the sequences 152 for a particular stage 150 have completed successfully, that stage 150 is complete. As indicated at 204, when running a sequence 152, a corresponding run resource group 146 may be created in which all the pipeline runs 154 in that sequence 152 will be run. As indicated at 206, as part of the run resource group creation, resources such as pipelines 140, pipeline data processing elements, configuration data, credentials,tokens, and other resources 142 may be copied from the base resource group 134 specified in the job configuration information 124 into the new run resource group 146. In some examples, only configuration data and credentials, tokens that are labeled to be copied will be copied to the run resource group 146. Additionally, in some examples, a service account with an authorizationbinding giving that service account cluster-administrator privileges may also be created. This service account may be used by one or more pipelines to perform any necessary operations as part of the pipeline runs 154.

[0034] In some examples, a run resource group 146 might not always be created. Pipeline runs 154 may be labelled to indicate that the pipelines require certain declared workspaces to be shared. In cases where workspaces need to be shared, the pipeline runs 154 which share that workspace may run in a common resource group. These pipeline runs may then be run in the base resource group. Further, the workspaces may be marked as shared, such as by using a label which contains a comma-separated list of workspace names that are shared, or through other suitable techniques. In examples herein, a workspace may be a feature that allows a pipeline run 154 to be configured to create a space that can be shared across data processing elements and pipelines. When a pipeline run 154 declares a workspace, the workspace may be either shared or not shared. If the workspace is shared, then the job execution application 126 may check to determine whether the workspace has already created a storage space for that workspace. If a storage space has already been created, then the job execution application 126 may reuse that storage space. Otherwise, if a storage space has not already been created, the job execution application 126 may create a new storage space by generating a storage space request. Alternatively, if the workspace is not shared, then a storage space, such as an empty directory volume may be created.

[0035] When running a pipeline run 154, the environment and run parameters 202 specified in the job configuration information 124 may be applied to the pipeline run 154. The run parameters 202 allow a user to specify the user ID and group ID with which the pipeline runs. For instance, when running a pipeline run 154 as part of an engineer’s workflow this enables avoidance of populating the engineer's environment with incorrectly owned files and directories. Additionally, some run parameters 202 may be set on the pipeline run 154, thereby allowing the user to alter how the pipeline runs.

[0036] FIG. 3 illustrates an example of a cleanup process 300 that may be performed according to some implementations. For example, following execution of one or more CI / CD job runs 132, the resources created by the job execution application 126 runs may be cleaned up. As one example, some or all resources that are created by the job execution application 126 may be marked with an annotation so that a cleanup program 302, knows which resources to examine fordetermining whether to delete those resources. If a resource is not marked with the annotation of job execution application 126, then the cleanup program 302 will ignore that resource. Resources that typically are annotated or otherwise marked for cleanup may include resource groups, cluster authorization-bindings, and storage space requests. For instance, run resource groups 146 may be marked for cleanup as soon as the sequence 152 that created the resource group completes its run, whether the sequence succeeded or failed. When a run resource group 146 has been marked for cleanup, the run resource group 146 is eligible for deletion, and the resource group 146 may be marked for cleanup using an annotation.

[0037] Additionally, cluster authorization-bindings may be created with a label indicating which resource group contains the service account to which the authorization is bound. When the indicated resource group is deleted or is terminating, the corresponding cluster authorization binding is eligible for deletion, and may be annotated or otherwise marked for deletion. Similarly, requested storage spaces may be marked for cleanup after a full CI / CD job run 132 has completed, regardless of whether the run succeeded or failed.

[0038] As indicated in FIG. 3, the job execution application 126 may annotate or otherwise mark the used resources discussed above for deletion during or following the CI / CD job run 132. In some examples, the cleanup program 302 may be executed by a scheduler program 304, or the like (e.g., as a CronJob in the case of a KUBERNETES® system). For example, an API call, such as a cleanup job call 306 may be issued periodically or by any other triggering event to invoke the cleanup program 302. The cleanup program 302 may execute one or more cleaner processes 308 that may examine each of the resource types discussed above to determine whether the resources are annotated resources 310 or non-annotated resources 312. As indicated at 314, non-annotated resources 312 are ignored. On the other hand, as indicated at 316, if any annotated resources 310 are found that are eligible for deletion, the cleaner process 308 deletes them.

[0039] FIG. 4 illustrates an example architecture of a computing system 400 in which the above-discussed examples of FIGS. 1-3 may be realized according to some implementations. In the example of FIG. 4, the service computing devices 102 include a management computing device 404 in communication via the one or more networks 106 with a plurality of worker computing devices 406(1 )-406(N) that together make up the computing system 400 for providing the CI / CD services described herein to the user devices 108. In this example, the management computing device 404 includes a system control program 428, an API server 430, and system configuration data 434, in addition to the client program 118, the job execution application 126, the status and logging information 136, the cleanup program 302, and the scheduling program 304 discussed above.

[0040] Further, in this example, a repository computing device 410 is able to communicate over the one or more networks 106 with at least one of the user devices 108, the client computing devices 104 and / or the service computing devices 102. For example, the repository computing device(s) 410 may include software 412 for integration that may be initialized, built, deployed, and tested using the CFCD techniques described herein. As one example, multiple users 112 may collaborate on a software 412 that is maintained in a repository provided by the repository computing device 410. Examples of such software 412 may include APIs, applications, and the like.

[0041] Each worker computing device 406(l)-406(N) may also be referred to as a “worker node”, and includes a respective node management application 436(1)-436(N) and a respective routing application 438(1)-438(N). Further, each worker computing device 406 may include at least one container group 440 (abbreviated as “CG” in FIG. 4). Each container group 440 may include one or more containers 442. In the illustrated example, the first worker computing device 406(1) includes at least a first container group 440(la) and a second container group 440(lb), each of which includes at least one respective container 442. Similarly, the Nth worker computing device 406(N) includes at least a first container group 440(Na) and a second container group 440(Nh), each of which includes at least one respective container 442. In some examples, each respective container 442 may be used to execute a separate process, such as a separate pipeline, sequence, or the like. One example of a computing system 400 for providing and managing a large number of containerized processes is a KUBERNETES® system, as mentioned above; however, implementations herein are not limited to use with KUBERNETES® systems.

[0042] The management computing device 404 serves as the primary controlling unit of the service computing devices 102 in the computing system 400, and may manage the workload and direct communications across the computing system 400. In some examples, the system configuration data 434 may include the configuration data for the service computing devices 102, which may represent the overall state of the computing system 400 at any given point in time. Furthermore, the API server 430 may process and validate API requests and may update the state of API objects in the configuration data 434 to enable configuration of workloads and containers 442 across the worker nodes 406. Additionally, the scheduling program 304 may track the resource usage on each respective worker node 406 to ensure that the scheduled respective workload does not exceed the available resources on the respective worker node 406. Accordingly, the scheduling program 304 may be configured to match the workload demand with the available resources of the worker nodes 406. In addition, the system control program 428 may be executedto motivate the computing system 400 toward a desired state, such as by communicating with the API server 430 to create, update, and / or delete resources of the worker nodes 406.

[0043] The worker computing devices 406 may each include a respective instance of the node management application 436 that manages starting, stopping, and maintaining the containers 442 organized into the respective container groups 440, and as directed by the management computing device 404. In some examples, each worker computing device 406 may relay its own status periodically (e.g., every second, every couple of seconds, etc.) to the management computing device 404, such as via a heartbeat message. If the management computing device 404 detects failure of a worker computing device 406, such as due to a missing heartbeat, the management computing device 404 may relaunch the container group(s) 440 from the failed worker computing device 406 on one or more different worker computing devices 406.

[0044] The container groups 440 may each contain one or more containers 442 that are colocated on the same worker computing device 406. As one non-limiting example, each container group 440 may be assigned a unique IP address within the computing system 400, which may allow applications to use ports without the risk of conflicts. Within a respective container group 440, all the containers 442 within the same container group 440 may be able to reference each other. In this example, for a first container 442 within a first container group 440 to access a second container 442 within a second container group 440, the first container may use the IP address of the second container group 440.

[0045] An individual container 442 may contain all of the resources for running a respective process. Accordingly, a container 442 may be the lowest level of a process or service, and may contain, e.g., an executable application or process, libraries used by the executable application or process, and their dependencies. The containers 442 may be lightweight, standalone, executable software packages that include everything needed to run an application or process, e.g., executable code, runtime, system tools, system libraries, and settings. Multiple containers 442 can run on the same worker computing device 406 and share an OS kernel with other containers 442, each running as isolated processes in user space. Selected ones of the containers 442 may be executed, and respective containers 442 may call other respective containers 442, as needed, such as for executing a plurality of sequences 152 in parallel, each sequence 152 including sequential execution of one or more pipeline runs 154, as discussed above.

[0046] Each user device 108 may be any suitable type of computing device such as a desktop, laptop, tablet computing device, mobile device, smart phone, wearable device, terminal, and / or any other type of computing device able to send data over a network. Client users 112 may be associated with client device(s) 108 such as through a respective user account, user logincredentials, or the like. Furthermore, the client device(s) 108 may be configured to communicate with the client computing device(s) 104 through the one or more networks 106, through separate networks, or through any other suitable type of communication connection. Numerous other variations will be apparent to those of skill in the art having the benefit of the disclosure herein.

[0047] In some implementations, each user device 108 may include a respective instance of the client application 114 that may execute on the user device 108, such as for communicating with the service computing devices 102, the client computing device(s) 104 and / or the repository computing device(s) 410. In some cases, the application 114 may include a browser or may operate through a browser, while in other cases, the application 114 may include any other type of application having communication functionality enabling communication with other computing devices over the one or more networks 106. For example, the user device 108 may send instructions for configuring and managing the service computing devices 102. In addition, the user device 108 may send instructions for configuring and managing the client computing device(s) 104. In some cases, the management computing device 404 and / or the client computing device(s) 104 may include a management web application (not shown in FIG. 1) or other application to enable the user device 108 to configure operations performed by the management computing device 404 and / or the client computing device(s) 104. Numerous other variations will be apparent to those of skill in the art having the benefit of the disclosure herein.

[0048] FIG. 5 illustrates select example components of one or more service computing devices 102 that may be used to implement some of the functionality of the systems described herein. The service computing devices(s) 102 may include one or more servers or other types of computing devices that may be embodied in any number of ways. For instance, in the case of a server, the programs, other functional components, and data may be implemented on a single server, a cluster of servers, a server farm or data center, a cloud-hosted computing service, and so forth, although other computer architectures may additionally or alternatively be used. Multiple service computing devices(s) 102 may be located together or separately, and organized, for example, as servers, virtual servers, server banks, and / or server farms. The described functionality may be provided by the servers of a single entity or enterprise, or may be provided by the servers and / or services of multiple different entities or enterprises.

[0049] In the illustrated example, the service computing device 102 includes, or may have associated therewith, one or more processors 502, one or more computer-readable media 504, and one or more communication interfaces 506. Each processor 502 may be a single processing unit or a number of processing units, and may include single or multiple computing units, or multiple processing cores. The processor(s) 502 can be implemented as one or more central processingunits, microprocessors, microcomputers, microcontrollers, digital signal processors, system-on- chip processors, artificial intelligence processors, graphics processing units, state machines, logic circuitries, and / or any devices that manipulate signals based on operational instructions. As one example, the processor(s) 502 may include one or more hardware processors and / or logic circuits of any suitable type specifically programmed or configured to execute the algorithms and processes described herein. The processor(s) 502 may be configured to fetch and execute computer-readable instructions stored in the computer-readable media 504, which may program the processor(s) 502 to perform the functions described herein.

[0050] The computer-readable media 504 may include volatile and nonvolatile memory and / or removable and non-removable media implemented in any type of technology for storage of information, such as computer-readable instructions, data structures, program modules, or other data. For example, the computer-readable media 504 may include, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, optical storage, solid state storage, magnetic tape storage, magnetic disk storage, network or cloud storage, array storage, network attached storage, a storage area network, or any other medium that can be used to store the desired information and that can be accessed by a computing device. Depending on the configuration of the service computing devices(s) 102, the computer-readable media 504 may be a tangible non- transitory medium to the extent that, when mentioned, non-transitory computer-readable media exclude media such as energy, carrier signals, electromagnetic waves, and / or signals per se. In some cases, the computer-readable media 504 may be at the same location as the service computing devices(s) 102, while in other examples, the computer-readable media 504 may be partially remote from the service computing devices(s) 102.

[0051] The computer-readable media 504 may be used to store any number of functional components that are executable by the processor(s) 502. In many implementations, these functional components comprise instructions or programs that are executable by the processor(s) 502 and that, when executed, specifically program the processor(s) 502 to perform the actions attributed herein to the service computing devices(s) 102. Functional components stored in the computer-readable media 504 may include the client program 118, the job execution application 126, the cleanup program 302, the scheduling program 304, the system control program 428, and the API server 430, each of which may include one or more computer programs, applications, executable code, or portions thereof. Further, while these programs are illustrated together in this example, during use, some or all of these programs may be executed on separate service computing devices(s) 102.

[0052] In addition, the computer-readable media 504 may store data, data structures, and other information used for performing the functions and services described herein. For example, the computer-readable media 504 may store the status and logging information 136, and the system configuration data 434. The service computing devices 102 may also include or maintain other functional components and data, which may include programs, drivers, etc., and the data used or generated by the functional components. Further, the service computing devices 102 may include many other logical, programmatic, and physical components, of which those described above are merely examples that are related to the discussion herein.

[0053] The one or more communication interfaces 506 may include one or more software and hardware components for enabling communication with various other devices, such as over the one or more network(s) 106. For example, the communication interface(s) 506 may enable communication through one or more of a LAN, the Internet, cable networks, cellular networks, wireless networks (e.g., Wi-Fi) and wired networks (e.g., Fibre Channel, fiber optic, Ethernet), direct connections, as well as close-range communications such as BLUETOOTH®, and the like, as additionally enumerated elsewhere herein.

[0054] In addition, the client computing device(s) 104 and / or the repository computing device(s) 410 may include hardware configurations and components similar to those discussed above for the service computing devices 102, but with different functional components and data, e.g., as discussed above. Thus, the client computing device(s) 104 and / or the repository computing device(s) 410 may each include at least one or more processors 502, one or more computer- readable media 504, and one or more communication interfaces 506.

[0055] FIG. 6 illustrates select example components of a user device 108 according to some implementations. As mentioned above, the user device 108 may include any of a number of different types of computing devices such as a desktop, laptop, tablet computing device, mobile device, smart phone, wearable device, terminal, workstation, server, and / or any other type of computing device able to send and receive data over a network.

[0056] In the example of FIG. 6, the user device 108 includes components such as at least one processor 602, one or more computer-readable media 604, one or more communication interfaces 606, and one or more input / output (I / O) devices 608. Each processor 602 may itself comprise one or more processors or processing cores. For example, the processor(s) 602 can be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, system-on-chip processors, artificial intelligence processors, graphics processors, state machines, logic circuitries, and / or any devices that manipulate signals based on operational instructions. In some cases, the processor(s) 602 may be one or morehardware processors and / or logic circuits of any suitable type specifically programmed or otherwise configured to execute the algorithms and processes described herein. The processor(s) 602 can be configured to fetch and execute computer-readable processor-executable instructions stored in the computer-readable media 604.

[0057] Depending on the configuration of the user device 108, the computer-readable media 604 may be an example of tangible non-transitory computer storage media and may include volatile and nonvolatile memory and / or removable and non-removable media implemented in any type of technology for storage of information such as computer-readable processor-executable instructions, data structures, program modules or other data. The computer-readable media 604 may include, but is not limited to, RAM, ROM, EEPROM, flash memory, solid-state storage, magnetic disk storage, optical storage, and / or other computer-readable media technology. Further, in some cases, the user device 108 may access external storage, such as storage arrays, network attached storage, storage area networks, cloud storage, or any other medium that can be used to store information and that can be accessed by the processor(s) 602 directly or through another computing device or network. Accordingly, the computer-readable media 604 may be computer storage media able to store instructions, modules or components that may be executed by the processor(s) 602. Further, when mentioned, non-transitory computer-readable media exclude media such as energy, carrier signals, electromagnetic waves, and signals per se.

[0058] The computer-readable media 604 may be used to store and maintain any number of functional components that are executable by the processor 602. In some implementations, these functional components comprise instructions or programs that are executable by the processor 602 and that, when executed, implement operational logic for performing the actions and services attributed above to the user device 108. Functional components of the user device 108 stored in the computer-readable media 604 may include the user application 114, as discussed above, which may enable the user device 108 to interact with the service computing devices 102 and / or the client computing device(s) 104.

[0059] In addition, the computer-readable media 604 may also store data, data structures and the like, that are used by the functional components. Depending on the type of the user device 108, the computer-readable media 604 may also optionally include other functional components and data, which may include applications, programs, drivers, etc., and the data used or generated by the functional components. Further, the user device 108 may include many other logical, programmatic and physical components, of which those described are merely examples that are related to the discussion herein.

[0060] The communication interface(s) 606 may include one or more interfaces and hardware components for enabling communication with various other devices, such as over the network(s) 106 or directly. For example, communication interface(s) 606 may enable communication through one or more of the Internet, cable networks, cellular networks, wireless networks (e.g., Wi-Fi) and wired networks, as well as close-range communications such as BLUETOOTH®, BLUETOOTH® low energy, and the like, as additionally enumerated elsewhere herein.

[0061] The user device 108 may further include the one or more I / O devices 608. The TO devices 608 may include a display, which may include a touchscreen as an input device. The TO devices 608 may further include speakers, a microphone, a camera, and various user controls (e.g., buttons, a joystick, a keyboard, a keypad, touchpad, mouse, etc.), a haptic output device, and so forth. Additionally, the user device 108 may include various other components that are not shown, examples of which include removable storage, a power source, such as a battery and power control unit, and so forth. Further, in some examples, the client computing device(s) 104 and / or the repository computing device(s) 410 may include hardware structures and components similar to those described for the user device 108, but with one or more different functional components, e.g., as discussed above.

[0062] The example processes described herein are only examples of processes provided for discussion purposes. Numerous other variations will be apparent to those of skill in the art in light of the disclosure herein. Additionally, while the disclosure herein sets forth several examples of suitable frameworks, architectures and environments for executing the processes, implementations herein are not limited to the particular examples shown and discussed. Furthermore, this disclosure provides various example implementations, as described and as illustrated in the drawings. However, this disclosure is not limited to the implementations described and illustrated herein, but can extend to other implementations, as would be known or as would become known to those skilled in the art.

[0063] Various instructions, methods, and techniques described herein may be considered in the general context of computer-executable instructions, such as computer programs and applications stored on computer-readable media, and executed by the processor(s) herein. Generally, the terms program and application may be used interchangeably, and may include instructions, routines, scripts, modules, objects, components, data structures, executable code, etc., for performing particular tasks or implementing particular data types. These programs, applications, and the like, may be executed as native code or may be downloaded and executed, such as in a virtual machine or other just-in-time compilation execution environment. Typically, the functionality of the programs and applications may be combined or distributed as desired invarious implementations. An implementation of these programs, applications, and techniques may be stored on computer storage media or transmitted across some form of communication media.

[0064] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as example forms of implementing the claims.

Claims

CLAIMS1. A system comprising: one or more processors configured by executable instructions to perform operations comprising: receiving, by the one or more processors, job configuration information for execution of a process including a plurality of stages, each stage including a plurality of sequences, each sequence including a plurality of pipelines, each pipeline including a plurality of data processing elements to be executed sequentially; executing, by the one or more processors, the process by executing, in parallel, the plurality of sequences of a first stage of the plurality of stages, wherein each of the sequences of the first stage corresponds to a respective resource group, each respective resource group including a copy of resources from a base resource group specified by the job configuration information, the copy of resources including the plurality of pipelines for each sequence; following completion of execution of the plurality of pipelines of the plurality of sequences of the first stage, executing, by the one or more processors, in parallel, the plurality of sequences of a next stage of the plurality of stages, wherein the plurality of stages are executed one at a time in a specified order until execution of the plurality of stages is completed; and sending, by the one or more processors, to a computing device, a result of the process.

2. The system as recited in claim 1, wherein the plurality of pipelines for each sequence are executed sequentially according to a specified order.

3. The system as recited in claim 1, further comprising: recording logging information related to execution of the process; and sending the logging information over a network to a computing device.

4. The system as recited in claim 1, the operations further comprising: receiving, by the one or more processors, over a network from a client computing device, the job configuration information and an instruction to execute the process.

5. The system as recited in claim 5, the operations further comprising: in response to receiving the job configuration information and the instruction to execute the process, sending an application programming interface call to an application executing on the one or more processors to start execution of the process.

6. The system as recited in claim 1, wherein the one or more processors are configured in a computing system including a plurality worker computing devices, each worker computing device including a plurality of separate containers for enabling the parallel execution of the plurality of sequences for each stage of plurality of stages.

7. The system as recited in claim 1, wherein the process is at least one of a continuous integration or continuous delivery process.

8. The system as recited in claim 1, the operations further comprising: annotating one or more resources used by the process as being eligible for deletion; executing a cleanup program that examines resources for annotations; and deleting the one or more resources determined to include the annotations.

9. The system as recited in claim 1 , wherein: during execution of the next stage, the pipelines of at least one sequence of the plurality of sequences of the next stage are labeled to indicate that the pipelines utilize a workspace that is shared; and the pipelines that share the workspace are run in the base resource group as a common resource group.

10. The system as recited in claim 1, wherein the process is executed for at least one of testing or updating a software product,11. The system as recited in claim 10, further comprising receiving at least a portion of software of the software product from at least one of a client computing device, a user computing device, or a repository computing device.

12. A method comprising: receiving, by one or more processors, job configuration information for execution of a process including a plurality of stages, each stage including a plurality of sequences, each sequence including a plurality of pipelines, each pipeline including a plurality of data processing elements to be executed sequentially; executing, by the one or more processors, the process by executing, in parallel, the plurality of sequences of a first stage of the plurality of stages, wherein each of the sequences of the first stage corresponds to a respective resource group, each respective resource group including a copy of resources from a base resource group specified by the job configuration information, the copy of resources including the plurality of pipelines for each sequence; following completion of execution of the plurality of pipelines of the plurality of sequences of the first stage, executing, by the one or more processors, in parallel, the plurality of sequences of a next stage of the plurality of stages, wherein the plurality of stages are executed one at a time in a specified order until execution of the plurality of stages is completed; and sending, by the one or more processors, to a computing device, a result of the process.

13. The method as recited in claim 12, wherein the plurality of pipelines for each sequence are executed sequentially according to a specified order.

14. One or more non-transitory computer-readable media storing instructions executable by one or more processors to configure the one or more processors to perform operations comprising: receiving job configuration information for execution of a process including a plurality of stages, each stage including a plurality of sequences, each sequence including a plurality of pipelines, each pipeline including a plurality of data processing elements to be executed sequentially; executing the process by executing, in parallel, the plurality of sequences of a first stage of the plurality of stages, wherein each of the sequences of the first stage corresponds to a respective resource group, each respective resource group including a copy of resources from a base resource group specified by the job configuration information, the copy of resources including the plurality of pipelines for each sequence; following completion of execution of the plurality of pipelines of the plurality of sequences of the first stage, executing, in parallel, the plurality of sequences of a next stage of the plurality of stages, wherein the plurality of stages are executed one at a time in a specified order until execution of the plurality of stages is completed; andsending, to a computing device, a result of the process.

15. The one or more non- transitory computer-readable media as recited in claim 14, wherein the plurality of pipelines for each sequence are executed sequentially according to a specified order.

Images