Method and system for development of scalable parallel software

Abstract

A method for developing scalable parallel software may include receiving a request to execute multiple instances of a code block and dispatching tasks to compute nodes for parallel and distributed execution of instances of the code block, wherein input to the tasks includes respective portions of an input dataset and output from the tasks includes respective portions of an output dataset. The method may further include storing in a datastore, for each task, runtime states of the compute node to which the respective task is dispatched at the start and the end of execution of the respective task, the portion of the input dataset processed by the respective task, and the portion of the output dataset produced by the respective task.

Description

[0001] METHOD AND SYSTEM FOR DEVELOPMENT OF SCALABLE PARALLEL SOFTWARE

[0002] RELATED APPLICATION(S)

[0003] This application claims the benefit under 35 U.S.C. § 119(e) of U.S. Provisional Application Serial No. 63 / 666,830, titled “Method and System for Collaborative Software Development” and filed on July 2, 2024, which is hereby incorporated by reference herein in its entirety.

[0004] FIELD

[0005] This disclosure relates to software development tools, and more particularly to tools for the development of parallel software at scale.

[0006] BACKGROUND

[0007] Developing software in which compute-intensive portions of a program can be carried out by many tasks executing in parallel on many processors is often crucial for harnessing the full computational power of modem high-performance computing systems, which can have thousands to millions of processors. As scientific, engineering, and data-driven applications grow in complexity and data volume, scalable parallel programming enables the efficient execution of tasks that likely would be infeasible or prohibitively slow on single-processor systems or systems lacking massively parallel hardware. The development of scalable parallel software is particularly important for advancing research in fields such as computational biology, genomics, drug discovery, artificial intelligence, machine learning, and physics simulations, where datasets can be massive and the demand for timely results can be omnipresent. Moreover, scalable parallelism can significantly improve the cost-effectiveness of resource utilization.

[0008] Integrated Development Environments (IDEs) allow individual software developers to experiment and prototype in isolation from other developers so as to prevent contaminating production code with defective experimental code. Once code is sufficiently tested and polished, it can be committed (e.g., uploaded and / or integrated) to a shared repository and accessed by other developers. Some IDEs provide limited support for the development and debugging of scalable parallel programs. SUMMARY

[0009] The present disclosure describes techniques and tools for the development of scalable parallel software. In some aspects, the techniques described herein relate to a method including: receiving, by a computing device, a request to execute a workflow, wherein the workflow includes a code block, wherein executing the workflow includes executing a plurality of instances of the code block, wherein input to the plurality of instances of the code block includes a plurality of portions of an input dataset, and wherein output of the plurality of instances of the code block includes a plurality of portions of an output dataset; dispatching, by the computing device, a plurality of tasks to a plurality of respective compute nodes for parallel and distributed execution by the plurality of compute nodes, wherein each task of the plurality of tasks corresponds to a respective instance of the plurality of instances of the code block, and wherein each task is configured to process a respective portion of the plurality of portions of the input dataset and produce a respective portion of the plurality of portions of the output dataset; and storing, in a datastore, for each task of the plurality of tasks, a first runtime state of the compute node to which the respective task is dispatched at a start of execution of the respective task, a second runtime state of the compute node to which the respective task is dispatched at an end of execution of the respective task, the portion of the input dataset processed by the respective task, and the portion of the output dataset produced by the respective task.

[0010] In some aspects, the techniques described herein relate to a method, wherein dispatching the plurality of tasks includes: initializing a plurality of runtime environments on the plurality of compute nodes based on a runtime configuration associated with the code block.

[0011] In some aspects, the techniques described herein relate to a method, wherein dispatching the plurality of tasks further includes: causing the plurality of runtime environments to execute the plurality of respective instances of the code block.

[0012] In some aspects, the techniques described herein relate to a method, wherein the plurality of runtime environments execute the plurality of respective instances of the code block in parallel.

[0013] In some aspects, the techniques described herein relate to a method, wherein the plurality of compute nodes include one or more serverless compute nodes.

[0014] In some aspects, the techniques described herein relate to a method, wherein the plurality of compute nodes include a first compute node of a first cloud computing environment and a second compute node of a second cloud computing environment.

[0015] In some aspects, the techniques described herein relate to a method, wherein each instance of the code block includes one or more iterations of a loop.

[0016] In some aspects, the techniques described herein relate to a method, wherein the plurality of tasks include a first task corresponding to a first instance of the code block, wherein the first instance of the code block includes a plurality of iterations of a loop, and wherein the method further includes storing, in the datastore, for each iteration of the plurality of iterations of the loop, a runtime state of the compute node to which the first task is dispatched at a start of execution of the respective iteration.

[0017] In some aspects, the techniques described herein relate to a method, wherein the computing device is a first computing device, the method further including: by the first computing device, aggregating the plurality of portions of the output dataset and sending the output dataset to a second computing device of a user.

[0018] In some aspects, the techniques described herein relate to a method, wherein the plurality of instances of the code block is a first plurality of instances of the code block, wherein executing the workflow includes executing a second plurality of instances of the code block, and wherein the method further includes: detecting, by the computing device, completion of a first task of the plurality of tasks, the first task corresponding to a first instance of the first plurality of instances of the code block; and dispatching, by the computing device, a second task corresponding to a second instance of the second plurality of instances of the code block.

[0019] In some aspects, the techniques described herein relate to a system including: a computing device; and a computer-readable storage medium storing instructions which, when executed by the computing device, cause the computing device to perform operations including: receiving, by the computing device, a request to execute a workflow, wherein the workflow includes a code block, wherein executing the workflow includes executing a plurality of instances of the code block, wherein input to the plurality of instances of the code block includes a plurality of portions of an input dataset, and wherein output of the plurality of instances of the code block includes a plurality of portions of an output dataset; dispatching, by the computing device, a plurality of tasks to a plurality of respective compute nodes for parallel and distributed execution by the plurality of compute nodes, wherein each task of the plurality of tasks corresponds to a respective instance of the plurality of instances of the code block, and wherein each task is configured to process a respective portion of the plurality of portions of the input dataset and produce a respective portion of the plurality of portions of the output dataset; and storing, in a datastore, for each task of the plurality of tasks, a first runtime state of the compute node to which the respective task is dispatched at a start of execution of the respective task, a second runtime state of the compute node to which the respective task is dispatched at an end of execution of the respective task, the portion of the input dataset processed by the respective task, and the portion of the output dataset produced by the respective task.

[0020] In some aspects, the techniques described herein relate to a system, wherein dispatching the plurality of tasks includes: initializing a plurality of runtime environments on the plurality of compute nodes based on a runtime configuration associated with the code block.

[0021] In some aspects, the techniques described herein relate to a system, wherein dispatching the plurality of tasks further includes: causing the plurality of runtime environments to execute the plurality of respective instances of the code block.

[0022] In some aspects, the techniques described herein relate to a system, wherein the plurality of runtime environments execute the plurality of respective instances of the code block in parallel.

[0023] In some aspects, the techniques described herein relate to a system, wherein the plurality of compute nodes include one or more serverless compute nodes.

[0024] In some aspects, the techniques described herein relate to a system, wherein the plurality of compute nodes include a first compute node of a first cloud computing environment and a second compute node of a second cloud computing environment.

[0025] In some aspects, the techniques described herein relate to a system, wherein the plurality of tasks include a first task corresponding to a first instance of the code block, wherein the first instance of the code block includes a plurality of iterations of a loop, and wherein the operations further include storing, in the datastore, for each iteration of the plurality of iterations of the loop, a runtime state of the compute node to which the first task is dispatched at a start of execution of the respective iteration.

[0026] In some aspects, the techniques described herein relate to a system, wherein the computing device is a first computing device, the operations further including: by the first computing device, aggregating the plurality of portions of the output dataset and sending the output dataset to a second computing device of a user. In some aspects, the techniques described herein relate to a system, wherein the plurality of instances of the code block is a first plurality of instances of the code block, wherein executing the workflow includes executing a second plurality of instances of the code block, and wherein the operations further include: detecting, by the computing device, completion of a first task of the plurality of tasks, the first task corresponding to a first instance of the first plurality of instances of the code block; and dispatching, by the computing device, a second task corresponding to a second instance of the second plurality of instances of the code block.

[0027] In some aspects, the techniques described herein relate to a computer-readable storage medium storing instructions that, when executed by a computing device, cause the computing device to perform operations including: receiving, by the computing device, a request to execute a workflow, wherein the workflow includes a code block, wherein executing the workflow includes executing a plurality of instances of the code block, wherein input to the plurality of instances of the code block includes a plurality of portions of an input dataset, and wherein output of the plurality of instances of the code block includes a plurality of portions of an output dataset; dispatching, by the computing device, a plurality of tasks to a plurality of respective compute nodes for parallel and distributed execution by the plurality of compute nodes, wherein each task of the plurality of tasks corresponds to a respective instance of the plurality of instances of the code block, and wherein each task is configured to process a respective portion of the plurality of portions of the input dataset and produce a respective portion of the plurality of portions of the output dataset; and storing, in a datastore, for each task of the plurality of tasks, a first runtime state of the compute node to which the respective task is dispatched at a start of execution of the respective task, a second runtime state of the compute node to which the respective task is dispatched at an end of execution of the respective task, the portion of the input dataset processed by the respective task, and the portion of the output dataset produced by the respective task.

[0028] BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Many additional features and advantages will become apparent to those skilled in the art upon reading the following description, when considered in conjunction with the accompanying drawings, wherein:

[0030] FIG. 1 is a block diagram of a system for partitioning code into blocks, executing code blocks, and storing the output to enable stable collaboration, in accordance with one or more embodiments;

[0031] FIG. 2 is a flow diagram of a method for executing a plurality of code blocks, in accordance with one or more embodiments;

[0032] FIG. 3 is a flow diagram of a method for exporting and replaying a plurality of code blocks, in accordance with one or more embodiments;

[0033] FIG. 4 is a block diagram of an example of code blocks in a workflow, in accordance with one or more embodiments;

[0034] FIG. 5 is a flow diagram of a method for executing code blocks in parallel, in accordance with one or more embodiments; and

[0035] FIG. 6 is a block diagram of an example of a computing device, in accordance with one or more embodiments.

[0036] DETAILED DESCRIPTION

[0037] Introduction

[0038] The development and use of artificial intelligence (Al) tools, such as machine learning (ML) models, have become increasingly complex, involving a myriad of specialized tools and platforms. Software developers, tasked with managing an entire Al workflow, often find themselves navigating a fragmented landscape of IDEs, data analysis tools, model training platforms, and deployment services. Each stage of development - from data preparation and analysis (e.g., using Jupyter Notebooks on local devices) to model design and training (e.g., leveraging cloud-based Al platforms with GPU acceleration) - may involve different sets of tools, leading to a cumbersome, multi-faceted development process. Despite the sophistication of individual tools, the lack of seamless integration between them hinders the overall efficiency, scalability, and reliability of Al development.

[0039] The use of disjointed Al development and operations tools may result in significant challenges. Specifically, the absence of a unified, integrated platform may lead to poor reproducibility of results, as subtle differences in environment setups and data handling can introduce inconsistencies. Efficiency may suffer due to inadequate support for orchestration of workflows across multiple tools, while the scalability of these workflows may be constrained by the limitations of individual components. Furthermore, the resilience of Al workflows may be compromised, as failures in one tool can have a cascading impact on the entire process. Ultimately, such pain points prolong the time-to-market for Al-driven solutions, increase development costs, and undermine the overall confidence in Al model reliability, prompting the desire for a more cohesive, collaborative, and scalable solution.

[0040] These challenges are particularly acute in the context of developing workflows (e.g., workflows for Al applications) with compute-intensive code designed to execute in parallel and at scale. For example, in a generative Al workflow, a generative model (e.g., a large language model (LLM), an image generator, a video generator, etc.) may be invoked many times to process many distinct data objects. For example, the generative model may be invoked over and over to generate responses to different queries or prompts. To avoid lengthy delays in the processing of such data sets, the Al workflow may process such queries or prompts in parallel by dispatching a large number of tasks to a large set of compute nodes, with each task configured to process a subset (or “slice”) of the queries or prompts.

[0041] The development and debugging of workflows capable of executing in parallel at scale can be challenging, and existing software development tools do not address the challenges that arise when developing and debugging such workflows. In particular, existing tools do not automatically scale the execution of such workflows and / or are not capable of tracing, reproducing, or replaying the execution of parallel tasks in such workflows. Thus, there is a need for improved tools for development and debugging of workflows with code capable of executing in parallel at scale.

[0042] Some embodiments of such improved tools are described herein. In some embodiments, these tools solve technical challenges that arise specifically in the realm of computational systems (e.g., technical challenges associated with developing and debugging parallel software). The development and debugging of parallel code can be difficult because the execution of independent parallel tasks on distributed compute nodes tends to be non-deterministic, making it difficult to reproduce, trace, or replay portions of the workflow that exhibit intermittent bugs or performance problems. The present disclosure describes some embodiments of tools that address this challenge.

[0043] Embodiments of the present disclosure may include a graph-based workflow management paradigm, leveraging directed acyclic graphs (DAGs) to orchestrate complex software development and Al workflows. Some embodiments described herein enable decoupling of compute and storage resources, allowing for dynamic allocation of heterogeneous compute nodes (e.g., local or remote CPUs, GPUs, neural processing units (NPUs), tensor processing units (TPUs), etc.) to individual tasks (e.g., code blocks) within the workflow. This decoupling facilitates fine-grained resource optimization so that each task may be executed with suitable resources and software tools, thereby enhancing computational efficiency (e.g., reduced latency, improved throughput, parallel execution of tasks in the absence of data dependences between the tasks, etc.) and scalability (e.g., support for large, distributed workflows).

[0044] Embodiments may also include checkpointing mechanisms with very fine granularity (e.g., task-level or even instruction-level granularity) to provide a robust fault-tolerance framework, enabling rapid recovery from task failures by reinstating the workflow from previously persisted states. Checkpoint mechanisms may also support reproducibility, as the inputs / outputs (e.g., data and runtime state) for each task may be explicitly captured and reused (e.g., for the same or a different task).

[0045] Embodiments may also allow the orchestrating of tasks in a workflow to be performed by local and / or remote compute nodes (e.g., via APIs to cloud resources or SKDs with local resources). By providing a unified, programmable interface for orchestrating entire workflows (e.g., Al pipelines), some embodiments of the subject technology allow developers to focus on software development and refinement rather than infrastructure management, thus accelerating the software development cycle.

[0046] Some embodiments may provide a method, system, and / or computer program product for executing a code block (e.g., in a data science notebook), and storing output and intermediate state in a persistence store with multi-user synchronization for stable collaboration. In some embodiments, execution can resume or independently replay from configurable, highly granular code points for reproducibility and traceability.

[0047] Example Systems and Methods for Software Development

[0048] Referring to FIG. 1, an example system 100 for software development is shown. The system 100 may include one or more clients 102. A client 102 may include a software agent running on any manner of computing device (e.g., a mobile phone, tablet, laptop, personal computer, virtual machine (VM), cloud instance, Docker container, Open Container Initiative (OCI) container, Kubernetes pod, etc., and variations thereof). The software agents of the client 102 can be implemented as standalone applications, standalone or embedded Web applications, software libraries, software development kits (SDKs), application programming interface (API) calls, or any other software distribution package.

[0049] A client 102 may connect to a canvas service 104 (e.g., via a web browser). The canvas service 104 may provide services to support different visual representations of code. For example, a canvas may be configured to receive code as user input and transform the received code into a distinct code block graphical element on the canvas. In some embodiments, connections between clients 102 and the canvas service 104 are persistent bidirectional communications channels (e.g., using WebSockets, gRPC bidirectional streaming, or any other suitable protocol) where clients 102 may make requests and the canvas service 104 may provide asynchronous responses and updates. In some embodiments, connections between clients 102 and the canvas service 104 are initiated by clients 102 with synchronous responses from the canvas service 104. In such cases, updates may be requested by clients 102 through a polling mechanism. As shown in FIG. 1, the canvas service 104 is represented in the system 100 as a single service, but it should be understood that the canvas service 104 may include multiple services. For example, individual functions of the canvas service 104 may be separated into independent services or actors (e.g., using a microservice architecture).

[0050] In some embodiments, the canvas service 104 authenticates clients 102 through a user service 112. A client 102 may provide user credentials or an authentication token to the canvas service 104, which may then be forwarded to the user service 112. In some embodiments, the user service 112 stores and retrieves user credentials from a user credential and permissions store 114. The user service 112 may also use the user credential and permissions store 114 to validate the user credentials or authentication token provided by clients 102. In some embodiments, the user service 112 is implemented as one or more services (e.g., microservice) inside a cluster (e.g., Kubemetes cluster) that accepts requests (e.g., Hypertext Transfer Protocol (HTTP) requests) from the canvas service 104.

[0051] In some embodiments, the user service 112 forwards the user credentials or authentication token provided by clients 102 to an external identity provider 134, which may validate user credentials or authentication tokens and respond to user service 112 accordingly. In some embodiments, the external identity provider provides single sign-on (SSO) functionality. Some non-limiting examples of providers of SSO functionality include Google, Okta, and GitHub.

[0052] The user credential and permissions store 114 may include persistent storage for storing user credentials, permissions, configuration (e.g., feature enablement flags, compute resource limits, group associations, user / administrative roles, canvas read / write access permissions, etc.), etc. The user credential and permissions store 114 may be implemented with a relational database (e.g., a SQL database like Postgres, MariaDB, MySQL, Oracle, AWS RDB, AWS Aurora, AWS Athena, GCP Cloud SQL, Azure SQL, etc.). In some embodiments, the user credential and permissions store 114 is implemented with a custom permissions database (e.g., Google Zanzibar, OryKeto, etc.). In some embodiments, the user credential and permissions store 114 is implemented with a document database (e.g., AWS DynamoDB, MongoDB, etc.), a flat file, or any other persistent storage medium.

[0053] In one embodiment, the canvas service 104 authorizes requests from clients 102 through user service 112. The user service 112 stores and retrieves user permissions associated with the user of the client, for example, to determine whether the use is authorized to access, modify, and / or execute a workflow. In some embodiments, the user service 112 returns a bundle of all or a subset of permissions associated with the user to the canvas service 104 so that the canvas service 104 can cache the permissions and use them for fine-grained orchestration of different canvas service 104 functions. In some embodiments, the user service 112 looks up a specific permission and responds to the canvas service 104 with an indication (e.g., Boolean values) of whether the user has permission.

[0054] The canvas service 104 may store and retrieve code from the code store 116. The code store 116 may include one or more computer programs written in a human- or computer-readable programming language, also referred to herein as “code.” A computer program may be partitioned (e.g., by a user) into one or more blocks of code (referred to herein as code blocks 130), which may include one or more lines of the computer program (e.g., a function, method, library, or any set of instructions of the computer program). For example, the canvas service 104 may provide a graphical user interface (e.g., a canvas) for the user, and the user may interact with the graphical user interface to create a code block interface element representing a code block 130. Once the code block interface element is created, the user may input one or more lines of code into the code block interface element. The collection of the code in the code block interface elements may collectively represent a computer program. In some embodiments, the code in a code block may represent a computer program.

[0055] The plurality of code blocks 130 may have inter-dependencies with respect to a runtime state 128, including but not limited to program inputs (e.g., data sets, machine learning (ML) models, configuration flags, etc.) and outputs (e.g., data tables, graphs, plots, images, etc.), method inputs and outputs, variable states, stack and heap memory states, etc. In some embodiments, the code in the code store 116 is custom code (e.g., proprietary source code) provided by a user (e.g., a client 102) of the system 100.

[0056] A runtime state 128 may also be dependent upon a runtime configuration 124, which includes but is not limited to operating system type (e.g., Linux kernel version, processor architecture, instruction set extensions, etc.), resource requirements (e.g., virtual CPUs, random access memory (RAM), network bandwidth, etc.), additional hardware requirements (e.g., graphics processing units (GPUs), ephemeral block storage, persistent block storage, security isolation, etc.), software compiler, interpreter, SDK version and configuration (e.g., Java virtual machine (JVM) parameters, Java SDK version, Python interpreter version, Ruby interpreter version, Node.js version, etc.), third party software package dependencies, etc.

[0057] In addition to the plurality of code blocks 130, the code store 116 may store a transaction history 126, which may include a history of software development transactions related to the code blocks (e.g., a “version history”) and a history of runtime transactions involving the code blocks (e.g., instances of execution of versions of the code blocks in runtime environments) (e.g., a “runtime history”).

[0058] A version history of a code block 130 may include information about various versions of the code block 130, such as who changed the code, when the code was changed, what code was changed, how the code was changed, etc. In some embodiments, the version history may include additional information about the most recent software development transactions for a code block 130 (e.g., code changes pending commit, code changes queued to be processed, any errors processing a code change, indications that code changes are complete and stable, etc.).

[0059] A runtime history of a code block 130 may include information about various instances in which the code block 130 was executed (e.g., a “runtime data”), such as who executed the code, when was the code executed, where was the code executed, how was the code executed, etc. In some embodiments, the runtime history may include data associating the code block 130 to runtime environments, such as pointers to the runtime configurations 124 used to initialize runtime environments for runtime transactions as well as pointers to runtime states 128 associated with (e.g., generated by, used to initialize, etc.) the runtime environments during the runtime transactions. In some examples, a previous version (e.g., of a code block 130, runtime configuration 124, runtime state 128, workflow, etc.) may be referred to as a “historical version.” In some embodiments, the code store 116 is implemented with a relational database (e.g., a SQL database like Postgres, MariaDB, MySQL, Oracle, AWS RDB, AWS Aurora, AWS Athena, GCP Cloud SQL, Azure SQL, etc.). In some embodiments, the code store 116 is implemented with a document database (e.g., AWS DynamoDB, MongoDB, etc.). In some embodiments, the code store 116 is implemented as a flat file, or any other persistent storage medium. In some embodiments, the code store 116 is a remote source code repository (e.g., GitHub, GitLab, Bitbucket, etc.).

[0060] Connecting to a code store 116 may require credentials. The canvas service 104 may retrieve credentials from the credential service 120 to provide to the code store 116. The credential service 120 may store credentials (e.g., encrypted credentials) per user and per service in a persistent credential store 122. In some embodiments, the credential store 122 is a remote credential store (e.g., HashiCorp Vault, AWS KMS, Azure Key Vault, GCP Secret Manager, etc.). In some embodiments, the credential store 122 is implemented with a relational database (e.g., a SQL database like Postgres, MariaDB, MySQL, Oracle, AWS RDB, AWS Aurora, AWS Athena, GCP Cloud SQL, Azure SQL, etc.), a document database (e.g., AWS DynamoDB, MongoDB, etc.), a flat file, or any other persistent storage medium.

[0061] For the canvas service 104 to independently process a plurality of code blocks 130, a runtime configuration 124 and a runtime state 128 may be utilized. A runtime configuration 124 may specify how one or more code blocks 130 are to be run. For example, the runtime configuration 124 for a code block 130 may specify an amount of memory, a type of Al accelerator, a type of processor, etc. The runtime configuration 124 for each code block 130 may be stored in a configuration store 118. In some embodiments, the runtime configuration 124 is stored as a collection of files in a configuration store 118 implemented in cloud storage (e.g., in AWS S3, Azure Blob Storage, Google Storage, etc.). In some embodiments, the runtime configuration 124 for each code block 130 is stored as a collection of files in a configuration store 118 implemented on a storage area network (SAN), network attached storage (NAS), or local disk. In some embodiments, the runtime configuration 124 for each code block 130 is stored in a configuration store 118 implemented in a relational database (e.g., a SQL database like Postgres, MariaDB, MySQL, Oracle, AWS RDB, AWS Aurora, AWS Athena, GCP Cloud SQL, Azure SQL, etc.) or a document database (e.g., AWS DynamoDB, MongoDB, etc.).

[0062] The runtime state 128 associated with a code block 130 (e.g., a version of a code block 130) may be stored in a runtime store 110. A runtime state 128 may be a record of the conditions of a runtime environment at any point throughout a workflow (e.g., the execution of one or more code blocks 130). For example, the runtime state 128 may include a set of running processes, persistent data (e.g., input data to a process, output data from a process), temporary data (e.g., caches), etc. In some embodiments, the runtime state 128 is stored as a collection of files in a runtime store 110, which may be implemented in cloud storage (e.g., in AWS S3, Azure Blob Storage, Google Storage, etc.). In some embodiments, the runtime state 128 for each code block 130 is stored as a collection of files in a runtime store implemented on a storage area network (SAN), network attached storage (NAS), or local disk. In some embodiments, runtime state(s) 128 for each code block 130 is stored in a runtime store 110 implemented in a relational database (e.g., a SQL database like Postgres, MariaDB, MySQL, Oracle, AWS RDB, AWS Aurora, AWS Athena, GCP Cloud SQL, Azure SQL, etc.) or a document database (e.g., AWS DynamoDB, MongoDB, etc.).

[0063] Though the code store 116, configuration store 118, and runtime store 110 are logically represented as separate entities in system 100, it should be understood that any combination of the code store 116, configuration store 118, and / or runtime store 110 could reside in the same physical or logical persistence store or be further subdivided into separate physical or logical persistence stores. It should also be understood that the functionality described herein is not dependent on the underlying storage architecture.

[0064] The canvas service 104 may dispatch tasks for processing one or more code blocks 130 by sending a task request to the executor service 106. The task request may include the code blocks 130 (e.g., retrieved from the code store 116), the runtime configuration 124 (e.g., retrieved from the configuration store 118) to create a runtime environment, and pointers (e.g., retrieved from the code store 116) to the runtime state 128 in the runtime store 110 to initialize the runtime environment.

[0065] The executor service 106 may manage compute nodes 108. In some embodiments, compute nodes 108 are ephemeral, serverless compute nodes (e.g., AWS Lambda, Azure Functions, GCP Cloud Functions, Kubemetes Knative functions, etc.). In some embodiments, compute nodes 108 may be dedicated compute nodes (e.g., AWS EC2, Azure Virtual Machines, GCP GCE, bare metal servers, etc.).

[0066] The executor service 106 may map task requests to compute nodes 108. The executor service 106 may also manage the creating and tearing down of compute nodes 108 (e.g., to meet demand), limit the number of compute nodes 108 that can be created, monitor the state of tasks running on compute nodes 108, and terminate compute nodes 108 that are stalled or otherwise unhealthy.

[0067] In order to execute a code block 130, the runtime environment may first be provided. A runtime environment may include a set of resources and services that a computer program may utilize when running, such as operating systems, libraries, dependencies, and configuration. For example, a runtime environment may include an operating system (e.g., Windows or Linux), a virtual machine (e.g., VMware or Virtual Box), a container (e.g., Docker or Kubemetes), a cloud platform (e.g., AWS or Azure), etc., running on a compute node 108.

[0068] A runtime environment may be provided from a pool of existing runtime environments (e.g., idle or running on a set of compute nodes 108) having a runtime configuration 124 and set to a runtime state 128. Additionally or alternatively, a runtime environment may be provided by first creating a new runtime environment (e.g., by an available compute node 108) using the runtime configuration 124 of the task request and initialized using the runtime state 128 (e.g., as pointed to by the runtime state 128 pointers of the task request). In some embodiments, the executor service 106 may first determine whether there is a suitable runtime environment available (e.g., matching the runtime configuration 124 in the task request) and, if not, assign a compute node 108 to create a new runtime environment (e.g., a virtual machine) with the runtime configuration 124 in the task request.

[0069] Once a runtime environment is available on a compute node 108, the executor service 106 may initialize the runtime by sending a configuration message to the compute node 108. The configuration message may include the runtime state 128 retrieved from the runtime store 110 (e.g., as pointed to by the runtime state 128 pointers provided in the task request). The runtime state 128 may be the start of a workflow (e.g., for running the entire workflow) or an intermediate state of the workflow (e.g., for resuming a previously incomplete execution of the workflow or otherwise executing a subset of the workflow). Once the runtime environment is initialized, the executor service 106 initiates processing of the plurality of code blocks 130. For example, the executor service 106 may send a run message to the compute node 108, which may include the plurality of code blocks 130 to be processed.

[0070] In some embodiments, task requests contain custom data retrieval tasks as part of the runtime creation (e.g., retrieving custom software package dependencies including ML models) and / or runtime initialization (e.g., retrieving input datasets from external databases or data services like Snowflake, Databricks, AWS RDB, AWS Aurora, AWS Athena, GCP Cloud SQL, Azure SQL, AWS DynamoDB, MongoDB, AWS S3, Azure Blob Storage, Google Storage, etc.). In these cases, the canvas service 104 may retrieve any credentials necessary to access data from external databases, data warehouses, or other services (e.g., Databricks, Snowflake, etc.) from the credential service 120 to include with the custom data retrieval task.

[0071] In some embodiments, the executor service 106 assigns task requests to execute in parallel if sufficient compute nodes 108 are available. If there are insufficient compute nodes 108 to execute a task request, the executor service 106 may notify the canvas service 104 of a failure to execute the task request.

[0072] The canvas service 104 may enforce dependencies between task requests. For example, if a plurality of code blocks A, B, and C are to be run in serial, the canvas service 104 may confirm B does not run before A and may confirm C does not run before B. As another example, if two users request to change and process the same code block 130, the canvas service 104 may resolve which user has priority (e.g., the first user to request, the user with higher permissions, etc.).

[0073] The canvas service 104 may also or instead synchronize inter-dependent task request completion. For example, if a code block C depends on both code blocks A and B, but A and B are independent of each other, the canvas service 104 may confirm that C is not executed until both A and B have completed and a coherent runtime state for C is created.

[0074] The canvas service 104 may also or instead retry failed task requests. The canvas service 104 may also or instead enforce compute limits on task execution based on user permissions (e.g., user license, user role, user group, subscription level, etc.) retrieved from the user service 112.

[0075] The compute nodes 108 may execute their assigned code blocks 130 in their configured and initialized runtime environment. As the plurality of code blocks 130 execute, the corresponding compute node 108 may track the runtime state 128, which may include program outputs (e.g., data tables, graphs, plots, images, etc.), method inputs and outputs, variable states, stack and heap memory state, runtime errors and exceptions, stack traces, core dumps, log messages, program metrics, host metrics (e.g., CPU usage, memory usage, network usage, etc.), runtime, etc. The compute node 108 may write the runtime state 128 to the runtime store 110, which may index runtime states 128 by task. In some embodiments, where the runtime store 110 is implemented as cloud storage, the runtime state 128 for a given task is stored as a collection of files. In some embodiments, where the runtime store 110 is implemented as a SAN, NAS, or local disk, the runtime state 128 for a given task is stored as a collection of files. In some embodiments, where the runtime store 110 is implemented as a transactional database, relational database, or document database, the runtime state 128 for a given task is stored in a database. The runtime state 128 may be used as a checkpoint to repeat the execution a code block 130 at a particular point in time.

[0076] In some embodiments, the runtime store 110 is a remote storage service. In some embodiments, connecting to the runtime store 110 requires credentials. If credentials are used to connect to the runtime store 110, the canvas service 104 may retrieve any credentials necessary to write data to the remote storage service from the credential service 120 and may include the credentials in the task request to the executor service 106. The executor service 106 may then provide the credentials to the compute node 108 executing the task so that the compute node 108 can write the runtime state 128 to the runtime store 110 (e.g., remote storage service). In some embodiments, compute nodes 108 write task execution progress to the runtime store 110. In some embodiments, compute nodes 108 publish periodic notifications with execution progress to the executor service 106. In some embodiments, the executor service 106 periodically polls compute nodes 108 for execution progress. In some embodiments, the executor service 106 periodically publishes task execution status reports to the canvas service 104, and the task execution status reports may be used to update transaction histories 126 associated with the code blocks 130 in the code store 116. In some embodiments, the canvas service 104 periodically polls the executor service 106 for task execution status, and the task execution status may be used to update transaction histories 126 associated with the code blocks in the code store 116.

[0077] In some embodiments, the canvas service 104 sends real-time status updates to clients 102. For example, the canvas service 104 may send runtime state 128 updates to clients 102 upon task completion. In some embodiments, clients 102 retrieve runtime state 128 for completed tasks from the canvas service 104. In some embodiments, the canvas service 104 redirects clients 102 (e.g., via HTTP redirect) to download a runtime state 128 directly from the runtime store 110.

[0078] In some embodiments, the canvas service 104 keeps track of all client 102 connections associated with a given user group and sends real-time status updates to all clients 102 associated with a given user group. In some embodiments, the canvas service 104 keeps track of all client 102 connections associated with a given plurality of code blocks 130 and sends real-time status updates to all clients 102 associated with a given plurality of code blocks 130. Clients 102 may be grouped in a variety of ways (e.g., by user group or by code blocks 130), and so it should be understood that the canvas service 104 is capable of supporting any and all potential groupings of clients 102 for the purposes of sending status updates (e.g., real-time or requested status updates).

[0079] FIG. 2 is a flow diagram of a process 200 for executing a plurality of code blocks 130, according to some embodiments. The process 200 may be performed by the canvas service 104 of FIG. 1. The canvas service 104 may process requests from a plurality of clients 102, each of which may request operations on the plurality of code blocks 130. As the process 200 relates to the canvas service 104, the following discussion of FIG. 2 may be made with references to FIG. 1.

[0080] At step 202 of the process 200, a request may be received by the canvas service 104 from a client 102.

[0081] At step 232, the request may be authenticated and / or authorized. The canvas service 104 may authenticate the client 102 through the user service 112. The client 102 may provide user credentials or an authentication token to the canvas service 104, which the canvas service 104 may forward to the user service 112 for authentication. In some embodiments, the user service 112 stores and retrieves user credentials from a user credential and permissions store 114, and the user service 112 may use the credential and permissions store 114 to validate the user credentials or authentication token provided by the client 102. In some embodiments, the user service 112 forwards the user credentials or authentication token provided by the client 102 to an external identity provider 134 to validate the user credentials or authentication token provided by the client 102. The canvas service 104 may also authorize requests from the client 102 through user service 112. The user service 112 may store and retrieve (e.g., from the user credential and permissions store 114) user permissions associated with the user associated with the request.

[0082] If either authentication or authorization fails, the process 200 may proceed to step 234 where the canvas service 104 rejects the request. Otherwise, the process 200 may proceed to steps 204, 206, or 208. If the request relates to creating or modifying a code block 130, the process 200 may proceed to step 204. If the request relates to the processing of a plurality of code blocks 130, the process 200 may proceed to step 206. If the request relates to updating the runtime configuration 124 for a plurality of code blocks 130, the process 200 may proceed to step 208.

[0083] At step 204, the client 102 may request to create or modify a code block 130. The code block 130 may be part of a larger computer program in the code store 116. Creating a new code block 130 may include receiving one or more lines of code for a computer program in the code store 116. Modifying a code block 130 may include changing one or more lines of code in the code block 130, splitting an existing code block 130 into one or more code blocks 130, combining two or more code blocks 130 into a smaller set of code blocks 130, or deleting one or more code blocks 130.

[0084] At step 206, the client 102 may request to process one or more code blocks 130 of a computer program. Processing may include running (e.g., executing) a code block 130 that has never been run before (and therefore has no prior runtime state 128). Processing may also or instead include running a code block 130 that has been run previously and has not been modified since the previous run. In this case, the code block 130 may have an existing runtime state 128 that can be reused. Processing may also or instead include running a code block 130 that has been run previously but has been modified since the previous run. In this case, the code block 130 may have an existing runtime state 128 that may be re-initialized or modified for use.

[0085] At step 208, the client 102 may request to update the runtime configuration 124 for one or more code blocks 130. Updating the runtime configuration 124 may include creating a runtime configuration 124 for a new code block 130, which has never before been processed. Updating the runtime configuration 124 may also include deleting a runtime configuration 124 for an existing code block 130. Updating the runtime configuration 124 may also include modifying the runtime configuration 124 for an existing code block 130, when the existing code block 130 would run in a new runtime environment (e.g., while being processed at step 206). In some embodiments, the client 102 request may include a combination of runtime configuration 124 updates, code block 130 modifications, and / or code block 130 processing.

[0086] The process 200 may proceed from steps 204, 206, and 208 to step 210.

[0087] At step 210, it is determining whether the request can be processed immediately. The canvas service 104 enforces dependencies between task requests. For example, the canvas service 104 may enforce a particular order for code blocks 130 to run. As another example, if multiple users request changes to a code block, the canvas service 104 may resolve conflicts between the users (e.g., by prioritizing one user over another and executing their requests in serial). The canvas service 104 may also synchronize inter-dependent task request completion. For example, if a code block C depends on both code blocks A and B, but A and B are independent of each other, the canvas service 104 may verify that C is not executed until both A and B have completed and a coherent runtime state for C is created. The canvas service 104 may also manage the retrying of failed task requests.

[0088] If the request is dependent upon one or more existing requests in the transaction history 126, the process 200 may continue to step 212 where the request is queued for future processing. At step 212, the canvas service 104 may create a new entry in the transaction history 126 to track the request. Once the existing requests on the queue have been completed, the current request may be re-evaluated at step 210.

[0089] If the request is not dependent upon one or more existing requests in the transaction history 126, the process 200 may proceed to step 214.

[0090] Although steps 214, 216, and 218 are presented sequentially, steps 214, 216, and 218 may be performed in any order, sequentially or in parallel.

[0091] At step 214, if the request includes creating or modifying a code block 130, the process 200 may continue to step 222 before proceeding to step 216; otherwise, the process 200 may continue directly to step 216.

[0092] At step 222, code blocks 130 are added, removed, and / or modified in the code store 116 according to the request. In some embodiments, the code store 116 is a remote source code repository (e.g., GitHub, GitLab, Bitbucket, etc.). In some embodiments, connecting to the code store 116 (e.g., remote source code repository) requires credentials. The canvas service 104 may retrieve credentials from the credential service 120 to provide to the code store 116. In some embodiments, when code blocks 130 are added to the code store 116 according to the request, runtime configurations 124 corresponding to each new code block 130 may also be created in the configuration store 118. In some embodiments, when code blocks 130 are removed from the code store 116 according to the request, runtime configurations 124 corresponding to each code block 130 may also be deleted from the configuration store 118.

[0093] Code block 130 changes (e.g., addition, removal, and / or modification to some or all of a code block 130) may be reflected by an entry in the transaction history 126. In some embodiments, sequential code block 130 changes may be aggregated into a single transaction history 126 entry. In some embodiments, sequential code block 130 changes are each given separate transaction history 126 entries.

[0094] At step 216, if the request includes a runtime configuration 124 update, the process 200 may continue to step 224 before proceeding to step 218; otherwise, the process 200 may continue directly to step 218.

[0095] At step 224, one or more runtime configurations 124 for code blocks 130 specified by the request are modified in the configuration store 118 according to the request. In some embodiments, runtime configuration 124 changes may be sent by the canvas service 104 to the executor service 106 for creating new runtime environments, prior to a request for processing of a plurality of code blocks 130.

[0096] At step 218, if the request includes code block 130 processing, the process 200 may continue to step 226 before proceeding to step 220; otherwise, the process 200 may continue directly to step 220.

[0097] At step 226, the canvas service 104 may create a task request to send to the executor service 106. The task request may include the plurality of code blocks 130 retrieved from the code store 116, the runtime configuration 124 used to create the runtime environment (e.g., retrieved from the configuration store 118), and the runtime state 128 in the runtime store 110, all of which may be used to initialize the runtime environment. In some embodiments, the task request may also include custom data retrieval tasks as part of the runtime creation (e.g., retrieving custom software package dependencies including ML models) and / or runtime initialization (e.g., retrieving input datasets from external databases or data service like Snowflake, Databricks, AWS RDB, AWS Aurora, AWS Athena, GCP Cloud SQL, Azure SQL, AWS DynamoDB, MongoDB, AWS S3, Azure Blob Storage, Google Storage, etc.). If the task request includes custom data retrieval tasks, the canvas service 104 may retrieve credentials from the credential service 120 to access data from external databases or services and include the credentials with the custom data retrieval task. Once the task request is dispatched to the executor service 106, the process 200 may continue to step 220.

[0098] In some embodiments, the executor service 106 assigns task requests to execute in parallel as long as there are sufficient compute nodes 108 available. If there are insufficient compute nodes 108 to execute a task request, the executor service 106 may notify the canvas service 104 of a failure to execute the task request. In some embodiments, the executor service 106 periodically publishes task execution status reports to the canvas service 104, which may be used to update transaction histories 126 in the code store 116. In some embodiments, the canvas service 104 periodically polls the executor service 106 for a task execution status, which may be used to update transaction histories 126 in the code store 116.

[0099] In step 220, the canvas service 104 sends a response to the client 102 for the client request received in step 202. In some embodiments, the canvas service 104 sends periodic, asynchronous real-time status updates to the client 102. In some embodiments, the canvas service 104 sends runtime state 128 updates to the client 102 upon task completion. In some embodiments, the client 102 retrieves runtime state 128 for completed tasks from the canvas service 104. In some embodiments, the canvas service 104 redirects clients 102 (e.g., via HTTP redirect) to download a runtime state 128 directly from the runtime store 110. In some embodiments, the canvas service 104 keeps track of all client 102 connections associated with a given user group and sends real-time status updates to all clients 102 associated with the user group. In some embodiments, the canvas service 104 keeps track of all client 102 connections associated with a given plurality of code blocks 130 and sends real-time status updates to all clients 102 associated with the plurality of code blocks 130. It should be understood that clients may be grouped in a variety of ways and that the canvas service 104 is capable of supporting any and all potential groupings of clients 102 for the purposes of sending status updates (e.g., realtime status updates).

[0100] FIG. 3 is a flow chart of a process 300 for exporting and replaying a plurality of code blocks. The process 300 may be performed by the canvas service 104 of FIG. 1. The canvas service 104 may export a plurality of code blocks 130 with corresponding runtime states 128 and / or runtime configurations 124 to reproduce a task execution from the transaction history 126. As the process 300 relates to the canvas service 104, the following discussion of FIG. 3 may be made with references to FIG. 1.

[0101] At step 302 of the process 300, an export request may be received by the canvas service 104 (e.g., from one of a plurality of clients 102). The export request may identify, for example, the code blocks 130 to be exported (e.g., the versions of the code blocks 130 to be exported). In such a case, the runtime states and / or runtime configurations associated with the requested code blocks may be identified based on the records of the transaction history 126 associated with the indicated code blocks. In some examples, the export request identifies a runtime transaction for which a runtime environment (or an interface to the runtime environment) is to be exported. In such examples, the runtime states and / or runtime configurations associated with the identified runtime transaction may be identified by the record of the runtime transaction in the transaction history 126. In some examples, the export request indicates that the code blocks and runtime states are to be exported as program code (e.g., source code) suitable for compilation and execution. In some examples, the export request indicates that a runtime environment or an interface to a runtime environment (e.g., a runtime environment wrapper, a Docker image, a Docker container, etc.) is to be exported, such that a suitable runtime environment (e.g., a runtime environment conforming to the runtime configuration(s), initialized with the runtime state(s), and loaded with the code block(s)) can be accessed via the interface.

[0102] At step 316, the request may be authenticated and / or authorized. The canvas service 104 may authenticate the client 102 through the user service 112. The client 102 may provide user credentials or an authentication token to the canvas service 104, which the canvas service 104 may forward to the user service 112 for authentication. In some embodiments, the user service 112 stores and retrieves user credentials from a user credential and permissions store 114, and the user service 112 may use the credential and permissions store 114 to validate the user credentials or authentication token provided by the client 102. In some embodiments, the user service 112 forwards the user credentials or authentication token provided by the client 102 to an external identity provider 134 to validate the user credentials or authentication token provided by the client 102.

[0103] The canvas service 104 may also authorize requests from the client 102 through user service 112 (e.g., to determine whether the canvas service 104 may facilitate the export request for the client 102). The user service 112 may store and retrieve (e.g., from the user credential and permissions store 114) user permissions associated with the user associated with the request.

[0104] If either authentication or authorization fails, the process 300 may proceed to step 318 where the canvas service 104 rejects the request. Otherwise, the process 300 may proceed to step 304.

[0105] At step 304, the canvas service 104 may collect (e.g., access, retrieve, download, etc.) the requested set of code blocks 130 (e.g., from the code store 116), the associated set of runtime configurations 124 (e.g., from the configuration store 118 as pointed to by the relevant record(s) of the transaction history 126), and the associated set of runtime states 128 (e.g., from the runtime store 110 as pointed to by the relevant record(s) of the transaction history 126). In some embodiments, the export includes a runtime environment generated from a runtime configuration 124 and runtime state 128, both of which may be associated with one or more code blocks 130 (e.g., as pointed to by the relevant record of the code block (e.g., the latest version of the code block) in the transaction history 126). In some embodiments, the export request specifies a particular version of the code block in the transaction history 126, for which specific historical versions of the code blocks 130 are retrieved from the code store 116, specific historical versions of the runtime configuration 124 are retrieved from the configuration store 118, and specific historical versions of the runtime state 128 are retrieved from the runtime store 110. In some embodiments, the export request specifies a particular runtime state of the workflow associated with the requested set of code blocks 130. For example, the export request may specify an intermediate runtime state associated with a previous incomplete execution of the workflow (e.g., that did not execute each code block 130). This way, the requester can resume the execution of the workflow from a particular point of the workflow.

[0106] In some examples, servicing the export request includes exporting the plurality of code blocks 130 as part of a runtime environment for offline execution. For example, the exported code blocks may be used for offline debugging, demonstration purposes, etc. In some examples, servicing the export request includes exporting the plurality of code blocks 130 with runtime state 128 converted to source code. For example, the exported source code may be used as part of a unit test, integration test, smoke test, canary test, etc.

[0107] At step 306, the canvas service 104 may determine whether the export request includes a request to export to source code. If the export request includes a request to export to source code, the process 300 may continue to step 308 before proceeding to step 310; otherwise, the process 300 may continue directly to step 310.

[0108] At step 308, the canvas service 104 may combine the plurality of code blocks 130 and associated runtime state 128 (e.g., as pointed to by the transaction history 126) into source code files in a programming language (e.g., Java, Python, C, Scala, Ruby, Go, R, etc.) specified by the export request. In some embodiments, each code block 130 is implemented as a method, the runtime state 128 (including the inputs to the method implementing the code block 130) is implemented as static variables, and a test framework corresponding to the programming language (e.g., JUnit, pytest, CUnit, RSpec, etc.) is used to associate the runtime state 128 with the code blocks 130 to create an independently runnable (e.g., executable) source code program suitable for reproducible execution of code blocks 130.

[0109] At step 310, the canvas service 104 may determine whether the export request includes a request to export a runtime environment or an interface to a runtime environment (e.g., a runtime environment wrapper, a Docker image, a Docker container, etc.). If the export request includes a request to export a runtime environment or an interface to a runtime environment, the process 300 may continue to step 312 to configure the runtime environment and / or generate an interface to the runtime environment before proceeding to step 314; otherwise, the process 300 may continue directly to step 314.

[0110] At step 312, the canvas service 104 may request a runtime environment and / or an interface thereto. The runtime environment may conform to the runtime configuration 124 associated with the code block(s) 130. The runtime environment may be initialized using the runtime state(s) 128 associated with the code block(s) 130 (e.g., by the executor service 106). The runtime environment may include the code block(s) 130 (e.g., may be loaded with the code block(s) 130 by the executor service 106). If requested, a runtime wrapper (e.g., a script capable of modifying parameters of the runtime environment) may be generated (e.g., by the executor service 106).

[0111] The executor service 106 may determine whether a suitable runtime environment is available for export. If a suitable runtime environment is not available, the executor service 106 may assign a compute node 108 to generate a new runtime environment (e.g., build a Docker container) with the runtime configuration 124. Once the runtime environment is available, the executor service 106 may initialize the runtime environment using the runtime state 128, load the code block(s) 130 into the runtime environment, and return the initialized runtime environment (e.g., the virtual machine or Docker container) or an interface thereto.

[0112] In some embodiments, the runtime environment (or interface thereto) is saved to the runtime store 110, and the executor service 106 may respond to the request (e.g., via the canvas service 104) with a pointer to the runtime environment (or interface thereto) in the runtime store 110. In some embodiments, the canvas service 104 may retrieve (e.g., access, obtain, download, etc.) the runtime environment (or interface thereto) from the runtime store 110 to return to the client 102 in response to the request. In some embodiments, the canvas service 104 stores only the pointer to the runtime environment (or interface thereto) in the runtime store 110 to return to the client in response to the request. The pointer to the runtime environment (or interface thereto) in the runtime store 110 may also or instead be stored in the transaction history 126. In some embodiments, a runtime wrapper may be or may be part of a runtime environment.

[0113] Though steps 306 and 310 are presented sequentially, steps 306 and 310 may be performed in any order, sequentially or in parallel.

[0114] At step 314, the canvas service 104 may return the exported runtime environment (or interface thereto) to the client 102 in response to the request. The exported runtime environment (or interface thereto) may be used for subsequent execution of the code blocks 130 (e.g., by the requester). In embodiments, the export includes an archive file with all the raw code blocks 130 and associated runtime state 128 (e.g., as pointed to by the transaction history 126). In some embodiments, the export includes an application container (e.g., a Docker container, an OCI container, etc.) containing the code blocks 130 built into a runtime environment initialized with the associated runtime state 128 (e.g., as pointed to by the transaction history 126). In some embodiments, the export includes an archive file with the generated source code files including the code blocks 130 and the associated runtime state 128 (e.g., as pointed to by the transaction history 126). It should be understood that there may be many possible export formats, and any suitable archive format can be generated at step 314.

[0115] Distributed Computing Workflows

[0116] Some examples of a canvas service 104 have been described. In some embodiments, the canvas service 104 stores and retrieves code blocks 130 (e.g., from a code store 116) of a workflow; stores and retrieves credential information (e.g., from a credential store 122) used to access data processed by code blocks 130 of a workflow; stores scheduled times to run code blocks 130; stores data regarding invocations (e.g., in runtime store 110) of a workflow and of the steps of a workflow; retrieves results (e.g., runtime state 128) corresponding to runs of a code block 130 in the workflow; communicates with an executor service 106 to send code blocks 130 for execution on computational resources (e.g., compute nodes 108), runtime configuration 124 data indicating the runtime configuration 124 in which the code blocks 130 run (e.g., the computational resources used to run the code blocks 130), and runtime state 128 data indicating the initial state of the code block 130 (e.g., inputs to the code block) and the location where the final state of the code block 130 (e.g., outputs from the code block) are to be stored; and communicates with the executor service 106 to obtain the results of a run of a code block 130, to orchestrate the next task to run, etc. In some embodiments, a code block 130 may implement a data preparation operation, a modeling operation (e.g., an ML model, one or more layers of an ML model, one or more portions of an ML model, etc.), or any other Al operation.

[0117] A workflow may include a plurality of code blocks 130 arranged (e.g., connected) as nodes on a canvas (e.g., a visual representation of code) to form a graph (e.g., a dataflow graph). In some embodiments, the workflow graph can be directed and / or acyclic, indicating an order of execution.

[0118] In some examples, a workflow or one or more code blocks (e.g., parallel code blocks) of a workflow may access or implement a model (e.g., a generative model, a predictive model, etc.). Generative models can analyze existing content, identify patterns in the content, and combine or modify the identified patterns to generate new content. The new content can include text, images, video, music, or any other suitable type of content. Some non-limiting examples of generative models include generative adversarial networks (GANs), variational autoencoders (VAEs), autoregressive models (e.g., large language models (LLMs)), recurrent neural networks (RNNs), transformer-based models, reinforcement learning models for generative tasks, etc. Transformerbased models generally have an encoder-decoder architecture, use an attention mechanism (e.g., scaled dot-product attention, multi-head attention, masked attention, etc.) to model the relationships between different elements in a sequence of content, and perform well when processing long sequences of content. Some non-limiting examples of transformer-based models include Generalized Pre-trained Transformer 4 (GPT-4), DALL-E3, etc. Other examples of generative models with text-processing capability include Jurassic- 1, Command, and Paradigm.

[0119] The term “generative model” as used herein may generally refer to a type of machine learning model that is trained on existing data to enable the generative model to generate, based on an input or prompt, new data that shares characteristics similar to that of the training data. In some examples, a generative model may handle text. In these examples, the generative model may accept text prompts and produce text outputs.

[0120] Predictive models can analyze historical data, identify patterns in that data, and make inferences (e.g., produce predictions or forecast outcomes) based on the identified patterns. Some non-limiting examples of predictive models include neural networks (e.g., deep neural networks (DNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), learning vector quantization (LVQ) models, etc.), regression models (e.g., linear regression models, logistic regression models, linear discriminant analysis (LDA) models, etc.), decision trees, random forests, support vector machines (SVMs), naive Bayes models, classifiers, etc.

[0121] Some examples of an executor service 106 (or “execution service”) have been described. In some embodiments, the executor service 106 submits code blocks 130 to run as tasks on computing resources (e.g., compute nodes 108). In some embodiments, the executor service 106 also provides information that can be used to run the tasks correctly on the computing resources (e.g., runtime configuration 124 data, runtime state 128 data, etc.). In some embodiments, the executor service 106 determines when a task corresponding to a code block 130 has finished executing and communicates the results to other components of the system 100 (e.g., to the canvas service 104). In some embodiments, the executor service 106 is implemented as a microservice.

[0122] Some examples of runtime configuration 124 data have been described. In some embodiments, runtime configuration 124 data may correspond to one or more code blocks 130 (e.g., a plurality of code blocks 130 arranged in a workflow graph). Such runtime configuration 124 data may indicate dependencies (e.g., control dependencies, data dependencies, etc.) between or among code blocks 130 in a workflow. In some embodiments, runtime configuration 124 data indicates attributes of the computational resources on which each (or all) of the code blocks 130 will run. For example, the runtime configuration 124 data may indicate that a code block 130 will run on one or more central processing units (CPUs), one or more graphics processing units (GPUs), a processing device with at least a specified amount of internal RAM, a compute node of a particular cloud computing environment, etc. In some embodiments, runtime configuration 124 data identifies the language of the source code contained in each of the code blocks 130. In some embodiments, runtime configuration 124 data identifies a type of distributed computing operation associated with (e.g., performed by) a code block 130. Some non-limiting examples of distributed computing operations may include data processing, data shuffling, data aggregation, validating the condition of a conditional loop, etc.

[0123] Some examples of a runtime store 110 have been described. In some embodiments, the runtime store 110 has an object storage architecture, which can be implemented in short-term storage (e.g., in RAM) or in long-term storage (e.g., solid-state drives, flash memory, hard drives, etc.). In some embodiments, the runtime store 110 provides a hierarchical directory structure in which the data associated with each project is stored. For example, the directory structure may include the following directories:

[0124] • ‘canvases / ’

[0125] • ‘canvases / {canvas_id} / ’

[0126] • ‘canvases / {canvas_id} / blocks / ’

[0127] • ‘canvases / {canvas_id} / blocks / {block id} / ’

[0128] • ‘canvases / {canvas_id} / blocks / {block_id} / fleet_run / ’, and

[0129] • ‘canvases / {canvas_id} / blocks / {block_id} / fleet_run / {fleet_run_id} ’ .

[0130] The ‘canvases’ directory may include a sub-directory ‘{canvas id}’ corresponding to each unique canvas (also referred to herein as a “project”). A canvas may include two or more code blocks 130 arranged to form one or more workflows. For each code block 130 in a canvas, the corresponding canvas directory may include a sub-directory ‘blocks / {block_id}’ corresponding to the code block. For each unique execution (or “fleet run”) of a workflow (each unique instance in which the workflow is executed), the directories of the code blocks of the workflow may include a sub-directory ‘fleet_run / {fleet_run_id}’ corresponding to that instance of the workflow. The runtime state 128 of the workflow upon arriving at a code block 130 and / or upon exiting the code block 130 may be stored in the code block’s fleet run directory, ‘blocks / {block_id} / fleet_run / {fleet_run_id} ’ .

[0131] A runtime state 128 may include (1) the data processed by a unique instance of a code block 130 (e.g., any inputs to that instance of the code block 130, including register reads, memory reads, and the runtime state 128 when that instance of the code block 130 starts executing), and (2) the data generated by that instance of the code block 130 (e.g., any outputs produced by that instance of the code block 130, including register writes, memory writes, and the runtime state when that instance of the code block 130 stops executing). In some embodiments, the runtime state 128 corresponding to an instance of a code block 130 also includes metadata indicating computational performance characteristics of an instance of the code block 130 (e.g., resource utilization, runtime, etc.), memory addresses to which the code block 130 instance stored data, indications of whether the code block 130 instance caused any error conditions, and / or any other suitable data associated with an instance of the code block 130.

[0132] The runtime state 128 for an instance of a code block 130 can include metadata (e.g., a JSON metafile) identifying memory addresses (e.g., project variables) to which the instance of the code block 130 writes information. Likewise, the runtime state 128 for an instance of a code block 130 can include the values of those memory addresses when the instance of the code block 130 finishes running. In some embodiments, the canvas service 104 tracks and enforces the control flow and data flow dependencies between instances of code blocks 130 in a project. For example, if an instance of a first code block 130 stores a value to a memory address Al and an instance of a dependent code block 130 subsequently loads the value at that memory address Al, the canvas service 104 can delay the execution of the instance of the dependent code block 130 until the instance of the first code block 130 has finished executing so that the correct value of the memory address Al is available before the instance of the dependent code block 130 attempts to load it. Likewise, when the dependent code block 130 begins executing, the canvas service 104 can initialize the runtime state 128 of the dependent code block 130 such that the value stored at the memory address Al within the runtime state 128 of the dependent code block 130 when the dependent code block 130 begins executing is the value stored to the memory address Al by the instance of the first code block 130. Alternatively, the canvas service 104 and executor service 106 can support “lazy loading,” such that the canvas service 104 provides the value written to the memory address Al by the instance of the first code block 130 to the instance of the dependent code block 130 when the instance of the dependent code block 130 first attempts to load the value of the memory address Al, rather than when the instance of the dependent code block 130 begins executing.

[0133] When a workflow of a project includes a loop (e.g., a conditional loop), such that one or more code blocks 130 of the workflow are executed multiple times, the system 100 can treat each loop iteration as a distinct fleet run with distinct instances of the one or more code blocks 130, such that runtime state 128 at the beginning and / or end of each iteration of each code block 130 is stored in the runtime store 110. In another approach, the system 100 can treat all iterations of the loop as a single fleet run, such that the runtime state 128 at the beginning of the first loop iteration and at the end of the final loop iteration are stored in the runtime store 110. The former approach can use considerably more storage than the latter approach but facilitates fine-grained replay of individual loop iterations of interest, which enhances the system’s tracing and debugging capabilities and can enhance the reproducibility of events of interest (e.g., runtime bugs). The latter approach can use considerably less storage than the former approach at the expense of providing a more coarse-grained replay of the loop, which can make debugging more difficult, make tracing more time-consuming, and reduce the reproducibility of events of interest.

[0134] Parallel processing of compute-intensive code blocks can substantially enhance the overall performance of software, including Al software. Thus, in some scenarios, software developers may wish to collaboratively develop software (e.g., Al software) in which at least some code blocks can be executed in parallel on distributed computing resources. In some embodiments, the system 100 supports collaborative development, debugging, tracing, and replaying of workflows incorporating code blocks configured to execute in parallel on distributed computing resources. Such code blocks may be referred to herein as “parallel code blocks.”

[0135] In some examples, users may expressly configure the workflows or code blocks to execute in parallel using parallel programming constructs. In some examples, users may configure workflows or code blocks to execute sequentially (e.g., using sequential programming constructs), and the system 100 may automatically execute the workflows or code blocks in parallel using automatic parallelization techniques. For example, the system 100 may convert sequential iterations of a loop into parallel instances of the code block within the loop. As another example, the system 100 may convert code that processes dimensioned data (e.g., lists, dictionaries, arrays, tables, vectors, etc.) into parallel instances of a code block that processes the elements of the dimensioned data. The foregoing examples are not limiting; system 100 may support automatic parallelization of any suitable sequential programming constructs.

[0136] To support collaborative development of parallel code blocks, the directory structure of the runtime store 110 may further include the following directories:

[0137] • ‘canvases / {canvas_id} / blocks / {block_id} / fleet_run / {fleet_run_id} / slice / ’, and

[0138] • ‘canvases / {canvas_id} / blocks / {block_id} / fleet_run / {fleet_run_id} / slice / {slide_id} / ’ . As used herein, “slice” may refer to a portion of a dataset processed by an instance of a parallel code block. When execution of a workflow reaches a parallel code block and the canvas service 104 orchestrates execution of two or more instances of the parallel code block on different slices of the dataset, the runtime states of those instances of the parallel code block may be stored in sub-directories corresponding to the slices processed by those instances of the parallel code block, ‘blocks / {block_id} / fleet_run / {fleet_run_id} / slice / {slice_id}’. The disaggregated data (e.g., runtime state) corresponding to the distinct slices can be aggregated at any stage in the workflow.

[0139] Scalable and reproducible. In some embodiments, the system 100 supports the development (e.g., collaborative, scalable, and / or reproducible development) of workflows (e.g., “parallel workflows”) using the techniques described below.

[0140] Roles. In some embodiments, user(s) create workflows by creating code blocks 130 and configuring the code blocks 130 on a canvas. In some embodiments, the canvas service 140 orchestrates the workflows, and the executor service 106 dispatches and tracks individual tasks. In some embodiments, the compute nodes 108 run (e.g., execute) the code, read data from the runtime store 110, and write data to the runtime store 110.

[0141] Process. In some embodiments, a process for the development of distributed workflows (e.g., workflows with parallel code) includes one or more of the following steps.

[0142] 1) A user may create an account on the system 100 (or with a service provided by an operator of the system 100). A reference may be created for the user in the user service 112. For each request submitted by the user, the system 100 may determine (e.g., via the user credentials and permissions store 114) whether the user has permission to perform the requested task or access the requested resource. Access control may be applied at the canvas level. In some embodiments, more fine-grained permissions are provided to control access to runtime configuration and usage.

[0143] 2) The user may create a canvas, which may be stored by the canvas service 104 (e.g., in the code store 116). The canvas may have a unique identifier that may be used as a common reference across all the stores.

[0144] 3) The user may create a set of code blocks 130 through the user interface (UI), APIs, SDKs, etc. For example, the user may interact with the UI to create a code block interface element in the UI representing a code block 130, and then the user may input one or more lines of code into the code block interface element. 4) The user may create dependencies between code blocks 130. Referring to FIG. 4, these dependencies may be visually represented as connections 402. The user may create the connections 402 through the user interface, APIs, SDKs, etc. For example, the user may draw a connection 402 in the UI between two code block interface elements 404 to create a connection between the two code blocks 130 represented by the two code block interface elements 404. Connected code blocks 130 may have access to the data created from upstream code blocks 130 (e.g., code blocks 130 executed to their left in the graph 400). Connections 402 may inform the canvas service 104 which code blocks 130 to execute as part of a workflow and in what order to execute the code blocks 130.

[0145] 5) The user may configure the compute node 108 that each code block 130 runs on and indicate whether each code block 130 is used to split input data, shuffle input data, aggregate input data, or act as a logical gate or conditional loop within the workflow. Input data may refer to data provided as parameters when invoking the workflow or the runtime state 128 saved in the runtime store 110.

[0146] 6) The user may submit a workflow (e.g., a reference to a workflow) to the canvas service 104 for the canvas service 104 to run one or more code blocks 130 of the workflow. The user may provide input data for the workflow, which may become data available to the first code blocks 130 in the graph 400.

[0147] 7) The canvas service 104 may create a fleet_run_id and one or more database entries (e.g., sub-directories in the directory structure of the runtime store 110) to track the progress of the workflow’s execution. a. The system (e.g., the canvas service) may assign each invocation of the workflow a fleet run id, which may be used to separate the data in the runtime store and to orchestrate different invocations of the same workflow simultaneously. b. Similarly, the system (e.g., the canvas service) may associate slice id’s with batches of data that are processed in parallel within each fleet run. In some examples, the same code block(s) run independently across each slice of data.

[0148] 8) The canvas service 104 may obtain (e.g., receive, download, access, etc.) the code blocks 130 (e.g., from the code store 116) for the workflow.

[0149] 9) The canvas services 104 may obtain data indicating the dependencies between the code blocks 130 and may build the graph (e.g., a control-flow and / or data-flow dependence graph).

[0150] 10) The canvas service 104 may request the executor service 106 to run the code blocks 130 (e.g., code blocks not waiting for dependencies to be satisfied) as one or more tasks. For example, each task may be associated with a request to run a code block 130. Each task provided to the executor service 106 may include:

[0151] 10a) Data identifying the code blocks 130 to run;

[0152] 10b) Data identifying the programming language(s) in which code blocks 130 are programmed;

[0153] 10c) The runtime configuration on which each code block 130 is executed; lOd) Data identifying the connections 402 between code blocks 130 (so the executor service 106 can correctly load the data); lOe) Input data (if provided);

[0154] 1 Of) The slide_id’s, fleet_run_id, block_id and canvas id, so that the correct storage paths can be constructed;

[0155] 10g) The semantics of the code block 130 within the context of the workflow (e.g., a conditional step, a data aggregation operation, etc.); and lOh) Any authentication information suitable for running the workflow within a cloud environment.

[0156] 11) The executor service 106 may use the programming language and runtime configuration information to trigger the code block 130 to run in the correct compute node 108 (e.g., a compute node 108 configured to interpret and execute the code block 130). Compute resources (e.g., the compute node 108) may be created or allocated at the time of execution if necessary.

[0157] 12) The compute node 108 may receive information regarding:

[0158] 12a) The code block 130 to run;

[0159] 12b) The canvas_id, block id, fleet_run_id, and slice id of the instance of the code block 130;

[0160] 12c) Temporary credentials for reading and writing to the runtime store 110;

[0161] 12d) Other code blocks that may be connected to the code block 130 (e.g., by data or control flow dependencies); and

[0162] 12e) The semantics of the code block 130 in the context of the workflow. 13) The compute node 108 may run the code block 1 0, analyze the instance of the code block 130, and store the results and metadata in the runtime store 110 (e.g., as a runtime state 128). In some embodiments, the system 100 may then shut down the compute node 108. In other examples, the compute node 108 is reused to execute similar code blocks 130.

[0163] 14) The executor service 106 may monitor the progress of each compute node 108. When a compute node 108 completes (e.g., an execution of a code block 130), the executor service 106 may communicate the completion with the canvas service 104, including information such as:

[0164] 14a) The fleet run id associated with the task;

[0165] 14b) The block id associated with code block;

[0166] 14c) The slice_id associated with the instance of the code block (if applicable); and 14d) Whether the task ran successfully.

[0167] 15) The canvas service 104 may perform one of the following operations:

[0168] 15a) Retry the task if it failed, when the canvas service 104 is configured to retry failed tasks;

[0169] 15b) Exit the workflow and mark the fleet_run as being unsuccessful; or

[0170] 15c) Save the status of the task (e.g., as being successful), save the state (e.g., a complete or intermediate state) of the runtime (e.g., in a runtime store 110), and call the executor service 106 to run the next task(s).

[0171] 16) Steps 10-16 may be repeated until the process is complete. Repeating steps 10-16 may include looping over the workflow conditionally.

[0172] 17) Mark the fleet run as having been successful.

[0173] Throughout the process the user can request information on the status, intermediate results, resource utilization, data created, time taken, etc.

[0174] Throughout the process the user can request information on the status, intermediate results, resource utilization, data created, time taken, etc.

[0175] In some embodiments, the system 100 improves upon conventional software development systems in one or more of the following ways:

[0176] 1) Users can define compute resources at the granularity of the code block 130 rather than relying on the coarse-grained autoscaling logic used by many providers of cloud computing services.

[0177] 2) Automatic reconstruction of variable spaces (e.g., memory address spaces) from connected steps (e.g., code blocks 130) in the graph (e.g., graph 400).

[0178] 3) Saving intermediate states (e.g., runtime state 128) allows code blocks 130 to be re-run from a particular stage (e.g., point in time) and inspected for debugging purposes.

[0179] 4) The format / protocol for handling the metadata produced means that distributed computing can happen across compute nodes that are not directly connected in the network but can access the runtime store 110. Thus, the system 100 enables compute nodes 108 in workflows to work across cloud platforms (e.g., Spark on Azure, GPU in AWS).

[0180] 5) Multi-language support and interoperability within the same workflow.

[0181] 6) Support for running functional code and non-functional code. For example, normal Python code can be written without adhering to a specific framework. This capability is useful as the initial stage of development can be scaled without re-writing code.

[0182] 7) Support for full traceability and reproducibility and for replaying a workflow from the beginning of any instance of a code block 130.

[0183] 8) Support for sharing data across tasks (e.g., instances of code blocks 130) without relying on in-memory storage of the shared data.

[0184] FIG. 5 is a flow diagram of a method 500 for executing code blocks in parallel, in accordance with some embodiments. In some examples, the method 500 is performed by one or more components of a system for software development (e.g., system 100).

[0185] At step 510, a computing device (e.g., a server implementing a canvas service 104) receives a request to execute a workflow. The workflow may include a code block. Executing the workflow may include executing multiple instances of the code block. Input to the instances of the code block may include respective portions of an input dataset. Output of the instances of the code block may include respective portions of an output dataset.

[0186] At step 520, the computing device dispatches tasks to respective compute nodes (e.g., compute nodes 108) for parallel and distributed execution by the compute nodes. Each task may correspond to a respective instance of the code block. Each task may be configured to process a respective portion of the input dataset and produce a respective portion of the output dataset.

[0187] In some examples, dispatching the tasks includes initializing runtime environments on the compute nodes based on a runtime configuration associated with the code block. In some examples, dispatching the tasks includes causing the runtime environments to execute the instances of the code block. In some examples, the runtime environments execute the respective instances of the code block in parallel. In some examples, the compute nodes include one or more serverless compute nodes. In some examples, the compute nodes include a first compute node of a first cloud computing environment and a second compute node of a second cloud computing environment. In such examples, the first and second compute nodes may both be configured to access a remote datastore, which may be hosted in a third cloud computing environment.

[0188] At step 530, data corresponding to each task are stored in a datastore. The data corresponding to a respective task may include a first runtime state of the compute node to which the task is dispatched at a start of execution of the task (or at a start of execution of an instance of the code block), a second runtime state of the compute node to which the task is dispatched at an end of execution of the task (or at an end of execution of an instance of the code block), the portion of the input dataset processed by the task, and the portion of the output dataset produced by the task.

[0189] In some examples, each instance of the code block comprises one or more iterations of a loop. In some examples, the tasks include a first task corresponding to a first instance of the code block which includes multiple iterations of a loop, and the method further comprises storing, in the datastore, for each iteration of the multiple iterations of the loop, a runtime state of the compute node to which the first task is dispatched at a start of execution of the respective iteration.

[0190] In some examples, the computing device is a first computing device, and the method further includes aggregating the portions of the output dataset and sending the output dataset to a second computing device of a user. The aggregating and sending may be performed by the first computing device.

[0191] In some examples, the instances of the code block are first instances of the code block, and executing the workflow includes executing second instances of the code block. In some examples, the method further includes detecting, by the computing device, completion of a first task corresponding to one of the first instances of the code block, and dispatching, by the computing device, a second task corresponding to one of the second instances of the code block.

[0192] Examples of Computing Devices

[0193] FIG. 6 illustrates an exemplary implementation of a computing device 600 that may be used in a system (e.g., system 100) implementing the techniques described herein. In some examples, a compute node 108 may be a computing device or include one or more computing devices. In some examples, a computing device configured to implement one or more services described herein. It should be understood that FIG. 6 is intended neither to be a depiction of necessary components for a computing device to operate in accordance with the principles described herein nor a comprehensive depiction.

[0194] The computing device 600 may include one or more processors 602, network adapters 604, and computer-readable storage media 606. The computing device 600 may be, for example, a desktop or laptop personal computer, a personal digital assistant (PDA), a smart mobile phone, a server, a wireless access point or other networking element, or any other suitable computing device.

[0195] The network adapter 604 may be any suitable hardware and / or software to enable the computing device 600 to communicate wired and / or wirelessly with any other suitable computing device over any suitable computing network. The computing network may include wireless access points, switches, routers, gateways, and / or other networking equipment as well as any suitable wired and / or wireless communication medium or media for exchanging data between two or more computers, including the Internet.

[0196] The computer-readable storage medium 606 may be adapted to store data to be processed and / or instructions to be executed by one or more processors 602. Processor 602 enables the processing of data and execution of instructions. The data and instructions may be stored on the computer-readable storage medium 606. The data and instructions stored on computer-readable storage medium 606 may comprise computer-executable instructions implementing techniques that operate according to the principles described herein. In the example of FIG. 6, the computer- readable storage medium 606 may store computer-executable instructions implementing various facilities and storing various information as described above. The computer-readable storage medium 606 may store a client facility 612 configured to perform the operations of client 102, a canvas service facility 614 configured to perform the operations of canvas service 104, an executor service facility 616 configured to perform the operations of executor service 106, a user service facility 622 configured to perform the operations of user service 112, and / or a credential service facility 630 configured to perform the operations of credential service 120.

[0197] While not illustrated in FIG. 6, a computing device may additionally have one or more components and peripherals, including input and output devices. Output devices can be used, among other things, to present a user interface and input devices can be used, among other things, to interact with the user interface. Examples of output devices that can be used to provide a user interface include printers or display screens for visual presentation of output and speakers or other sound-generating devices for audible presentation of output. Examples of input devices that can be used for interacting with a user interface include keyboards and pointing devices (e.g., mice, touch pads, and digitizing tablets). As another example, a computing device 600 may receive input information through speech recognition or in other audible format.

[0198] Terminology

[0199] Terms such as “coupled” and “connected,” along with their derivatives, may be used in this disclosure, the claims, or both. “Coupled” may be used to indicate that two or more elements, which may or may not be in direct physical or electrical contact with each other, cooperate or interact with each other. “Connected” may be used to indicate the establishment of communication (e.g., a communicative relationship) between two or more elements that are coupled with each other. Further, in some embodiments set forth herein, an element, component, or module may be configured to perform a function if it is capable of performing or otherwise structurally arranged to perform that function.

[0200] A “network element” or “node” may include one or more pieces of service network equipment, including hardware and software that communicatively interconnects other equipment on a network (e.g., other network elements, end stations, etc.) and is adapted to host one or more applications or services with respect to one or more subscribers. As such, some network elements may be disposed in a wireless / wireline telecommunications network or a cable provider network, whereas other network elements may be disposed in a public packet-switched network infrastructure (e.g., a local area network, wide area network, the internet, etc.), private packet-switched network infrastructures (e.g., intranets and enterprise networks), as well as service provider network infrastructures (e.g., cloud platforms), any of which may span or involve a variety of access networks and core networks in a hierarchical arrangement. Accordingly, some network elements may comprise multiple services network elements that provide support for multiple network-based functions. “Software” and “code” may be understood to refer to the literary representation of the computer program. The literary representation of a computer program may include high-level languages (e.g., C, C++, Java, Kotlin, Python, R, Go, Scala, etc.) and / or low-level languages (e.g., assembly code, binary, or byte code representations). “Software” and “code” may also be understood to refer to any literary work that is syntactically compatible with a software compiler or interpreter for the execution of a computer program represented by the literary work.

[0201] Techniques operating according to the principles described herein may be implemented in any suitable manner. Included in the discussion above are flow charts showing the steps and acts of various processes that facilitate collaborative software development. The processing and decision blocks of the flow charts above represent steps and acts that may be included in algorithms that carry out these various processes. Algorithms derived from these processes may be implemented as software integrated with and directing the operation of one or more single- or multi-purpose processors. Processors 602 may be implemented as integrated circuits, such as a Digital Signal Processing (DSP) circuit, Field Programmable Gate Array (FPGA), an Application-Specific Integrated Circuit (ASIC), or any other suitable manner. It should be understood that the flow charts included herein do not depict the syntax or operation of any particular circuit or programming language. Rather, the flow charts illustrate the information one of ordinary skill in the art may use to fabricate circuits or to implement computer software algorithms to perform the techniques described herein. It should also be understood that, unless otherwise indicated herein, the particular sequence of steps and / or acts described in each flow chart is merely illustrative and not limiting.

[0202] Accordingly, in some embodiments, the techniques described herein may be embodied in computer-executable instructions implemented as software, including application software, system software, firmware, middleware, embedded code, or any other type of software. Such computer-executable instructions may be written using any suitable programming languages and / or programming or scripting tools, and also may be compiled as executable machine language code or intermediate code that is executed on a computer (e.g., a physical or logical computing device).

[0203] When techniques described herein are embodied as computer-executable instructions, the computer-executable instructions may be implemented in any suitable manner, including as a number of functional facilities, each providing one or more operations to complete execution of algorithms operating according to these techniques. A “functional facility,” however instantiated, may be a structural component of a computer system that, when integrated with and executed by one or more computers, may cause the one or more computers to perform a specific operational role. A functional facility may be a portion of or an entire software element. For example, a functional facility may be implemented as a function of a process, a discrete process, or as any other suitable unit of processing. If techniques described herein are implemented as multiple functional facilities, each functional facility may be implemented in its own way and need not be implemented the same way. Additionally, the functional facilities may be executed in parallel and / or serially, as appropriate, and may pass information between one another using a shared memory on the computer(s) on which they are executing (e.g., by using a message-passing protocol, or in any other suitable way).

[0204] Functional facilities may include routines, programs, objects, components, data structures, etc., that perform particular tasks or implement particular abstract data types. The functionality of the functional facilities may be combined or distributed as desired in the systems in which they operate. In some implementations, one or more functional facilities carrying out techniques herein may together form a complete software package (e.g., computer program). The functional facilities may, in some embodiments, be adapted to interact with others (e.g., unrelated functional facilities and / or processes) to implement a software program application.

[0205] Some exemplary functional facilities are described herein for carrying out one or more tasks. It should be understood that the functional facilities and division of tasks described are merely illustrative of the type of functional facilities that may implement embodiments described herein and that embodiments are not limited to being implemented in any specific number, division, or type of functional facilities. In some embodiments, all functionalities described herein may be implemented in a single functional facility. It should also be appreciated that, in some embodiments, some of the functional facilities described herein may be implemented together with or separately from others (e.g., as a single unit or separate units), or some of these functional facilities may not be implemented.

[0206] Computer-executable instructions implementing the techniques described herein (e.g., when implemented as one or more functional facilities or in any other manner) may, in some embodiments, be encoded on one or more computer-readable media to provide functionality to the media. Computer-readable media may include magnetic media such as a hard disk drive, optical media such as a Compact Disk (CD) or a Digital Versatile Disk (DVD), a persistent or non-persistent solid-state memory (e.g., flash memory, RAM, etc.), or any other suitable storage media. A computer-readable medium may be implemented in any suitable manner, including as a computer-readable storage medium 606 of FIG. 6 (e.g., as a portion of a computing device 600) or as a stand-alone, separate storage medium.

[0207] As used herein, “computer-readable medium” (also called “computer-readable storage medium”) may refer to one or more tangible storage mediums. Tangible storage mediums may be non-transitory and may have at least one physical, structural component. In a “computer- readable medium,” as used herein, at least one physical, structural component has at least one physical property that may be altered in some way during a process of creating the medium with embedded information, a process of recording information thereon, or any other process of encoding information onto the medium. For example, a magnetization state of a portion of a physical structure of a computer-readable medium may be altered during a recording process for storing data onto the medium.

[0208] Further, some techniques described above comprise acts of storing information (e.g., data and / or instructions) in certain ways for use by these techniques. In some implementations of these techniques — such as implementations where the techniques are implemented as computerexecutable instructions — the information may be encoded on a computer-readable storage medium. Where specific structures are described herein as advantageous formats in which to store this information, these structures may be used to impart a physical organization of the information when encoded on the storage medium. These advantageous structures may then provide functionality to the storage medium by affecting the operations of one or more processors interacting with the information (e.g., by increasing the efficiency of computer operations performed by the processor(s)).

[0209] In some embodiments in which the techniques described herein may be embodied as computer-executable instructions, the instructions may be executed on one or more suitable computing device(s) operating in any suitable computer system (e.g., the system 100), or one or more computing devices (e.g., one or more processors 602 of one or more computing devices 600) may be programmed to execute the computer-executable instructions. A computing device 600 or processor 602 may be programmed to execute instructions when the instructions are stored in a manner accessible to the computing device / processor, such as in a local memory (e.g., an on-chip cache or instruction register, a computer-readable storage medium accessible via a bus, a computer-readable storage medium accessible via one or more networks and accessible by the device / processor, etc.). Functional facilities that comprise these computer-executable instructions may be integrated with and direct the operation of a single multi-purpose programmable digital computer apparatus, a coordinated system of two or more multi-purpose computer apparatuses sharing processing power, a single computer apparatus or coordinated system of computer apparatuses (e.g., co-located or geographically distributed), one or more FPGAs, or any other suitable system.

[0210] Embodiments have been described where the techniques are implemented in circuitry and / or computer-executable instructions. It should be understood that some embodiments may be in the form of a method, of which at least one example has been provided. The acts performed as part of the method may be ordered in any suitable way. Accordingly, embodiments may be constructed in which acts are performed in an order different than illustrated, which may include performing some acts simultaneously, even though shown as sequential acts in illustrative embodiments.

[0211] Various aspects of the embodiments described above may be used alone, in combination, or in a variety of arrangements not specifically discussed in the embodiments described in the foregoing and are therefore not limited in their application to the details and arrangement of components set forth in the foregoing description or illustrated in the drawings. For example, aspects described in one embodiment may be combined in any manner with aspects described in other embodiments.

[0212] Use of ordinal terms such as “first,” “second,” “third,” etc., in the claims to modify a claim element does not by itself connote any priority, precedence, or order of one claim element over another or the temporal order in which acts of a method are performed, but are used merely as labels to distinguish one claim element having a certain name from another element having a same name (but for use of the ordinal term) to distinguish the claim elements.

[0213] Also, the phraseology and terminology used herein are for the purpose of description and should not be regarded as limiting. The use of “including,” “comprising,” “having,” “containing,” “involving,” and variations thereof herein, is meant to encompass the items listed thereafter and equivalents thereof as well as additional items. The phrase “and / or,” as used in the specification and in the claims, should be understood to mean “either or both” of the elements so conjoined, i.e., elements that are conjunctively present in some cases and disjunctively present in other cases. Multiple elements listed with “and / or” should be construed in the same fashion, i.e., “one or more” of the elements so conjoined. Other elements can optionally be present other than the elements specifically identified by the “and / or” clause, whether related or unrelated to those elements specifically identified. Thus, as a non-limiting example, a reference to “A and / or B”, when used in conjunction with open-ended language such as “comprising” can refer, in one embodiment, to A only (optionally including elements other than B); in another embodiment, to B only (optionally including elements other than A); in yet another embodiment, to both A and B (optionally including other elements); etc.

[0214] The word “exemplary” is used herein to mean serving as an example, instance, or illustration. Any embodiment, implementation, process, feature, etc., described herein as exemplary should therefore be understood to be an illustrative example and should not be understood to be a preferred or advantageous example unless otherwise indicated.

[0215] Having thus described several aspects of at least one embodiment, it is to be appreciated that various alterations, modifications, and improvements will readily occur to those skilled in the art. Such alterations, modifications, and improvements are intended to be part of this disclosure and are intended to be within the spirit and scope of the principles described herein. Accordingly, the foregoing description and drawings are by way of example only.

Claims

CLAIMSWhat is claimed is:

1. A method comprising : receiving, by a computing device, a request to execute a workflow, wherein the workflow includes a code block, wherein executing the workflow includes executing a plurality of instances of the code block, wherein input to the plurality of instances of the code block includes a plurality of portions of an input dataset, and wherein output of the plurality of instances of the code block includes a plurality of portions of an output dataset; dispatching, by the computing device, a plurality of tasks to a plurality of respective compute nodes for parallel and distributed execution by the plurality of compute nodes, wherein each task of the plurality of tasks corresponds to a respective instance of the plurality of instances of the code block, and wherein each task is configured to process a respective portion of the plurality of portions of the input dataset and produce a respective portion of the plurality of portions of the output dataset; and storing, in a datastore, for each task of the plurality of tasks, a first runtime state of the compute node to which the respective task is dispatched at a start of execution of the respective task, a second runtime state of the compute node to which the respective task is dispatched at an end of execution of the respective task, the portion of the input dataset processed by the respective task, and the portion of the output dataset produced by the respective task.

2. The method of claim 1 , wherein dispatching the plurality of tasks includes: initializing a plurality of runtime environments on the plurality of compute nodes based on a runtime configuration associated with the code block.

3. The method of claim 2, wherein dispatching the plurality of tasks further includes: causing the plurality of runtime environments to execute the plurality of respective instances of the code block.

4. The method of claim 3, wherein the plurality of runtime environments execute the plurality of respective instances of the code block in parallel.

5. The method of claim 4, wherein the plurality of compute nodes comprise one or more serverless compute nodes.

6. The method of claim 4, wherein the plurality of compute nodes include a first compute node of a first cloud computing environment and a second compute node of a second cloud computing environment.

7. The method of claim 1, wherein each instance of the code block comprises one or more iterations of a loop.

8. The method of claim 1, wherein the plurality of tasks include a first task corresponding to a first instance of the code block, wherein the first instance of the code block includes a plurality of iterations of a loop, and wherein the method further comprises storing, in the datastore, for each iteration of the plurality of iterations of the loop, a runtime state of the compute node to which the first task is dispatched at a start of execution of the respective iteration.

9. The method of claim 1, wherein the computing device is a first computing device, the method further comprising: by the first computing device, aggregating the plurality of portions of the output dataset and sending the output dataset to a second computing device of a user.

10. The method of claim 1, wherein the plurality of instances of the code block is a first plurality of instances of the code block, wherein executing the workflow includes executing a second plurality of instances of the code block, and wherein the method further comprises: detecting, by the computing device, completion of a first task of the plurality of tasks, the first task corresponding to a first instance of the first plurality of instances of the code block; and dispatching, by the computing device, a second task corresponding to a second instance of the second plurality of instances of the code block.

11. A system comprising: a computing device; anda computer-readable storage medium storing instructions which, when executed by the computing device, cause the computing device to perform operations including: receiving, by the computing device, a request to execute a workflow, wherein the workflow includes a code block, wherein executing the workflow includes executing a plurality of instances of the code block, wherein input to the plurality of instances of the code block includes a plurality of portions of an input dataset, and wherein output of the plurality of instances of the code block includes a plurality of portions of an output dataset; dispatching, by the computing device, a plurality of tasks to a plurality of respective compute nodes for parallel and distributed execution by the plurality of compute nodes, wherein each task of the plurality of tasks corresponds to a respective instance of the plurality of instances of the code block, and wherein each task is configured to process a respective portion of the plurality of portions of the input dataset and produce a respective portion of the plurality of portions of the output dataset; and storing, in a datastore, for each task of the plurality of tasks, a first runtime state of the compute node to which the respective task is dispatched at a start of execution of the respective task, a second runtime state of the compute node to which the respective task is dispatched at an end of execution of the respective task, the portion of the input dataset processed by the respective task, and the portion of the output dataset produced by the respective task.

12. The system of claim 11, wherein dispatching the plurality of tasks includes: initializing a plurality of runtime environments on the plurality of compute nodes based on a runtime configuration associated with the code block.

13. The system of claim 12, wherein dispatching the plurality of tasks further includes: causing the plurality of runtime environments to execute the plurality of respective instances of the code block.

14. The system of claim 13, wherein the plurality of runtime environments execute the plurality of respective instances of the code block in parallel.

15. The system of claim 14, wherein the plurality of compute nodes comprise one or more serverless compute nodes.

16. The system of claim 14, wherein the plurality of compute nodes include a first compute node of a first cloud computing environment and a second compute node of a second cloud computing environment.

17. The system of claim 11, wherein the plurality of tasks include a first task corresponding to a first instance of the code block, wherein the first instance of the code block includes a plurality of iterations of a loop, and wherein the operations further comprise storing, in the datastore, for each iteration of the plurality of iterations of the loop, a runtime state of the compute node to which the first task is dispatched at a start of execution of the respective iteration.

18. The system of claim 11, wherein the computing device is a first computing device, the operations further comprising: by the first computing device, aggregating the plurality of portions of the output dataset and sending the output dataset to a second computing device of a user.

19. The system of claim 11, wherein the plurality of instances of the code block is a first plurality of instances of the code block, wherein executing the workflow includes executing a second plurality of instances of the code block, and wherein the operations further comprise: detecting, by the computing device, completion of a first task of the plurality of tasks, the first task corresponding to a first instance of the first plurality of instances of the code block; and dispatching, by the computing device, a second task corresponding to a second instance of the second plurality of instances of the code block.

20. A computer-readable storage medium storing instructions that, when executed by a computing device, cause the computing device to perform operations including: receiving, by the computing device, a request to execute a workflow, wherein the workflow includes a code block, wherein executing the workflow includes executing a pluralityof instances of the code block, wherein input to the plurality of instances of the code block includes a plurality of portions of an input dataset, and wherein output of the plurality of instances of the code block includes a plurality of portions of an output dataset; dispatching, by the computing device, a plurality of tasks to a plurality of respective compute nodes for parallel and distributed execution by the plurality of compute nodes, wherein each task of the plurality of tasks corresponds to a respective instance of the plurality of instances of the code block, and wherein each task is configured to process a respective portion of the plurality of portions of the input dataset and produce a respective portion of the plurality of portions of the output dataset; and storing, in a datastore, for each task of the plurality of tasks, a first runtime state of the compute node to which the respective task is dispatched at a start of execution of the respective task, a second runtime state of the compute node to which the respective task is dispatched at an end of execution of the respective task, the portion of the input dataset processed by the respective task, and the portion of the output dataset produced by the respective task.

Images