Method and system for collaborative software development

Abstract

A tool for collaborative software development may perform a method including receiving a request to execute a second code block of a workflow, wherein the second code block is associated with a runtime configuration and a first runtime state that includes data produced by a first code block of the workflow; initializing a runtime environment based on the runtime configuration and the first runtime state; causing the runtime environment to execute the second code block and to generate a second runtime state based on the execution of the second code block; and storing the second runtime state and data associating the second runtime state with the second code block. The first runtime state can include results of a data analysis process, and the second runtime state can include results of applying the modeling process to at least a portion of the results of the data analysis process.

Description

METHOD AND SYSTEM FOR COLLABORATIVE SOFTWARE DEVELOPMENTCROSS-REFERENCE TO RELATED APPLICATION(S)

[0001] This application claims the benefit of U.S. Patent App. No. 18 / 499,184, titled “Method and System for Collaborative Software Development” and filed on October 31, 2023, subsequently converted to U.S. Provisional App. No.which is hereby incorporated by reference herein to the maximum extent permitted by applicable law.FIELD

[0002] This disclosure relates to software development tools and, more particularly, to such tools enabling real-time collaboration among software developers.BACKGROUND

[0003] Integrated Development Environments (IDEs) allow individual software developers to experiment and prototype in isolation from other developers so as to prevent contaminating production code with defective experimental code. Once code is sufficiently tested and polished, it can be committed (e.g., uploaded and / or integrated) to a shared repository and accessed by other developers. However, IDEs generally do not allow developers to share their code with other developers before it is committed.SUMMARY

[0004] In some aspects, the techniques described herein relate to a method including: receiving, by a first computing device and from a second computing device, a first request to execute a workflow, wherein the workflow includes a set of code blocks, and each code block of the set of code blocks is associated with a respective set of runtime configurations; initializing, by the first computing device, a first runtime environment based on the set of runtime configurations associated with a first code block of the set of code blocks; causing, by the first computing device, the first runtime environment to execute the first code block and to generate afirst runtime state based on the execution of the first code block; initializing, by the first computing device, a second runtime environment based on the first runtime state and based on the set of runtime configurations associated with a second code block of the set of code blocks; causing, by the first computing device, the second runtime environment to execute the second code block and to generate a second runtime state based on the execution of the second code block; and storing, by the first computing device, the first runtime state, the second runtime state, and data associating the first runtime state and the second runtime state with the first code block and the second code block, respectively.

[0005] In some aspects, the techniques described herein relate to a method, wherein: the first code block causes the first runtime environment to perform a data analysis process; the first runtime state includes a set of results of the data analysis process; the second code block causes the second runtime environment to perform a modeling process; and the second runtime states includes a set of results of the modeling process.

[0006] In some aspects, the techniques described herein relate to a method, wherein the first computing device causes the second code block to execute after the first code block executes in response to the second code block having one or more data dependences on the first code block.

[0007] In some aspects, the techniques described herein relate to a method, wherein the second runtime environment is identical to the first runtime environment, in response to the set of runtime configurations associated with the second code block being identical to the set of runtime configurations associated with the first code block.

[0008] In some aspects, the techniques described herein relate to a method, further including providing, by the first computing device to the second computing device, a user interface for presenting to the user via the second computing device, wherein the first request is received by the first computing device from the second computing device via the user interface.

[0009] In some aspects, the techniques described herein relate to a method, further including, providing, by the first computing device and to the second computing device, an indication that the first code block and the second code block have successfully executed, in response to storing the first runtime state and the second runtime state.

[0010] In some aspects, the techniques described herein relate to a method, further including: receiving, by the first computing device and from a third computing device, a second request toexecute the workflow; initializing, by the first computing device, a third runtime environment based on the set of runtime configurations associated with the first code block of the set of code blocks; causing, by the first computing device, the third runtime environment to execute the first code block and to generate a third runtime state based on the execution of the first code block; initializing, by the first computing device, a fourth runtime environment based on the third runtime state and based on the set of runtime configurations associated with the second code block of the set of code blocks; causing, by the first computing device, the fourth runtime environment to execute the second code block and to generate a fourth runtime state based on the execution of the second code block; and storing the third runtime state, the fourth runtime state, and data associating the third runtime state and the fourth runtime state with the first code block and the second code block, respectively.

[0011] In some aspects, the techniques described herein relate to a method, wherein the first runtime environment and the third runtime environment are different and the second runtime environment and the fourth runtime environment are different, in response to the first request being different from the second request.

[0012] In some aspects, the techniques described herein relate to a method, wherein: the second request includes a modification to the set of runtime configurations associated with the first code block; and the first runtime environment is different from the third runtime environment based on the modification to the set of runtime configurations associated with the first code block.

[0013] In some aspects, the techniques described herein relate to a method, wherein: the second request includes a modification to the first code block; and the third runtime state is different from the first runtime state based on the modification to the first code block.

[0014] In some aspects, the techniques described herein relate to a method, wherein the third and fourth runtime environments are initialized after the first and second runtime environments, in response to the second request being queued after the first request.

[0015] In some aspects, the techniques described herein relate to a computing device including: a processor; and a memory storing instructions that, when executed by the processor, cause the computing device to perform operations including: receiving, by the computing device, a first request to execute a workflow, wherein the workflow includes a set of code blocks, andeach code block of the set of code blocks is associated with a respective set of runtime configurations; initializing, by the computing device, a first runtime environment associated with a code block of the set of code blocks and based on the set of runtime configurations associated with the code block; causing, by the computing device, the first runtime environment to execute the code block and to generate a first runtime state based on the execution of the code block; storing, by the computing device, the first generated runtime state and data associating the first generated runtime state with the code block; receiving, by the computing device, a second request to execute the code block; initializing, by the computing device, a second runtime environment based on the first generated runtime state associated with the code block and based on the set of runtime configurations associated with the code block; causing, by the computing device, the second runtime environment to execute the code block and to generate a second runtime state based on the execution of the code block; and storing the second generated runtime state and data associating the second generated runtime state with the code block.

[0016] In some aspects, the techniques described herein relate to a computing device, further including providing, by the computing device to another computing device, a user interface for presenting to the user via the other computing device, wherein the first request is received by the first computing device from the second computing device via the user interface.

[0017] In some aspects, the techniques described herein relate to a computing device, wherein: the computing device includes a plurality of historical versions of the workflow; and the workflow is a historical version of the plurality of historical versions specified by the first request.

[0018] In some aspects, the techniques described herein relate to a computing device, further including providing, by the computing device and to another computing device, an indication that the code block has successfully executed, in response to storing the second generated runtime state.

[0019] In some aspects, the techniques described herein relate to a computing device, wherein: the second request includes a modification to the set of runtime configurations associated with the code block; and the first generated runtime environment is different from the second generated runtime environment based on the modification to the set of runtime configurations associated with the first code block.

[0020] In some aspects, the techniques described herein relate to a computing device, wherein: the second request includes a modification to the code block; and the second generated runtime state is different from the first generated runtime state based on the modification to the code block.

[0021] In some aspects, the techniques described herein relate to a non-transitory computer- readable storage medium storing instructions that, when executed by a processor of a computing device, causes the computing device to perform operations including: receiving, by the computing device, a first request to execute a workflow, wherein the workflow includes a set of code blocks, and each code block of the set of code blocks is associated with a respective set of runtime configurations; for each respective code block of the set of code blocks: initializing, by the computing device, a respective runtime environment based on the respective set of runtime configurations associated with the respective code block of the set of code blocks; and causing, by the computing device, the respective runtime environment to execute the respective code block and to generate a respective runtime state based on the execution of the respective code block; receiving, by the computing device, a second request to export the workflow; generating, by the computing device, a set of runtime environments based on the set of runtime configurations of each code block of the set of code blocks and based on the generated set of runtime states; and providing, by the computing device, the generated set of runtime environments and the set of code blocks to another computing device for subsequent execution of the set of code blocks with the generated set of runtime environments.

[0022] In some aspects, the techniques described herein relate to a non-transitory computer- readable storage medium, wherein the generated set of runtime environments and the set of code blocks are provided to the other computing device as an application container.

[0023] In some aspects, the techniques described herein relate to a non-transitory computer- readable storage medium, the operations further including: receiving, by the computing device, a third request to export the workflow; generating, by the computing device, a second set of runtime environments based on the set of runtime configurations of each code block of the set of code blocks and based on the generated set of runtime states; and providing, by the computing device, the second generated set of runtime environments and the set of code blocks to another computing device for subsequent execution of the set of code blocks with the second generatedset of runtime environments on the other computing device.

[0024] In some aspects, the techniques described herein relate to a non-transitory computer- readable storage medium, wherein: the generated set of runtime environments is a first generated set of runtime environments; the third request includes a modification to a runtime configuration of a set of runtime configurations associated with a code block of the set of code blocks; and the second generated set of runtime environments is different from the first generated set of runtime environments based on the modification to the runtime configuration.

[0025] In some aspects, the techniques described herein relate to a method including: receiving, by a first computing device and from a second computing device, a first request to execute a second code block of a workflow, wherein the second code block is associated with a runtime configuration and a first runtime state, and wherein the first runtime state includes data produced by a first code block of the workflow; initializing, by the first computing device, a runtime environment based on the runtime configuration and the first runtime state associated with the second code block; causing, by the first computing device, the runtime environment to execute the second code block and to generate a second runtime state based on the execution of the second code block; and storing, by the first computing device, the second runtime state and data associating the second runtime state with the second code block.

[0026] In some aspects, the techniques described herein relate to a method, wherein: the first code block includes instructions of a data analysis process; the first runtime state includes a set of results of the data analysis process; the second code block includes instructions of a modeling process; and the second runtime state includes a set of results of applying the modeling process to at least a portion of the set of results of the data analysis process.BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Additional features and advantages will become apparent to those skilled in the art upon reading the following description when considered in conjunction with the accompanying drawings wherein:

[0028] FIG. 1 is a block diagram of a system for partitioning code into blocks, executing code blocks, and storing the output to enable stable collaboration, in accordance with one or more embodiments.

[0029] FIG. 2 is a flow diagram of a method for executing a plurality of code blocks, in accordance with one or more embodiments.

[0030] FIG. 3 is a flow diagram of a method for exporting and replaying a plurality of code blocks, in accordance with one or more embodiments.

[0031] FIG. 4 is a block diagram of an example of code blocks in a workflow, in accordance with one or more embodiments.

[0032] FIG. 5 is a flow diagram of a process for executing code blocks, in accordance with one or more embodiments.

[0033] FIG. 6 is a flow diagram of a process for collaboratively executing one or more code blocks, in accordance with one or more embodiments.

[0034] FIG. 7 is a flow diagram of a process for exporting a workflow, in accordance with one or more embodiments.

[0035] FIG. 8 is a flow diagram of a process for executing at least one code block, in accordance with one or more embodiments.

[0036] FIG. 9 is a block diagram of an example of a computing device, in accordance with one or more embodiments.DETAILED DESCRIPTION

[0037] The development and utilization of artificial intelligence (Al) tools, such as machine learning (ML) models, have become increasingly complex, involving a myriad of specialized tools and platforms. Software developers, tasked with managing an entire Al workflow, often find themselves navigating a fragmented landscape of IDEs, data analysis tools, model training platforms, and deployment services. Each stage of development — from data preparation and analysis (e.g., using Jupyter Notebooks on local devices) to model design and training (e.g., leveraging cloud-based Al platforms with GPU acceleration) — may involve different sets of tools, leading to a cumbersome, multi-faceted development process. Despite the sophistication of individual tools, the lack of seamless integration between them hinders the overall efficiency, scalability, and reliability of Al development.

[0038] The use of disjointed Al development and operations tools may result in significant challenges. Specifically, the absence of a unified, integrated platform may lead to poorreproducibility of results, as subtle differences in environment setups and data handling can introduce inconsistencies. Efficiency may suffer due to inadequate support for orchestration of workflows across multiple tools, while the scalability of these workflows may be constrained by the limitations of individual components. Furthermore, the resilience of Al workflows may be compromised, as failures in one tool can have a cascading impact on the entire process. Ultimately, such pain points prolong the time-to-market for Al-driven solutions, increase development costs, and undermine the overall confidence in Al model reliability, prompting the desire for a more cohesive, collaborative, and scalable solution.

[0039] Embodiments of the present disclosure may include a graph-based workflow management paradigm, leveraging directed acyclic graphs (DAGs) to orchestrate complex software development and Al workflows. Some embodiments described herein enable decoupling of compute and storage resources, allowing for dynamic allocation of heterogeneous compute nodes (e.g., local or remote CPUs, GPUs, neural processing units (NPUs), tensor processing units (TPUs), etc.) to individual tasks (e.g., code blocks) within the workflow. This decoupling facilitates fine-grained resource optimization so that each task may be executed with suitable resources and software tools, thereby enhancing computational efficiency (e.g., reduced latency, improved throughput, parallel execution of tasks in the absence of data dependences between the tasks, etc.) and scalability (e.g., support for large, distributed workflows).

[0040] Embodiments may also include checkpointing mechanisms at node-level granularity (e.g., task-level or even instruction-level) to provide a robust fault- tolerance framework, enabling rapid recovery from task failures by reinstating the workflow from previously persisted states. Checkpoint mechanisms may also support reproducibility, as the inputs / outputs (e.g., data and runtime state) for each task may be explicitly captured and reused (e.g., for the same or a different task).

[0041] Embodiments may also include a DAG visualization and editing interface to help streamline workflow authoring and debugging, which reduces the cognitive overhead associated with managing intricate, multi-stage pipelines. The interface may also be used by multiple users (e.g., via the internet) to facilitate collaboration across the workflow.

[0042] Embodiments may also allow the orchestrating of each task in a workflow to be performed by local and / or remote compute nodes (e.g., via APIs to cloud resources or SKDs withlocal resources). By providing a unified, programmable interface for orchestrating entire workflows (e.g., Al development pipelines), some embodiments of the subject technology allow developers to focus on software development and refinement rather than infrastructure management, thus accelerating the software development cycle.

[0043] Referring to FIG. 1, an example system 100 for software development is shown. The system 100 may include one or more clients 102. A client 102 may include a software agent running on any manner of computing device (e.g., a mobile phone, tablet, laptop, personal computer, virtual machine (VM), cloud instance, Docker container, Open Container Initiative (OCI) container, Kubernetes pod, etc., and variations thereof). The software agents of the client 102 can be implemented as standalone applications, standalone or embedded Web applications, software libraries, software development kits (SDKs), application programming interface (API) calls, or any other software distribution package.

[0044] A client 102 may connect to a canvas service 104 (e.g., via a web browser). The canvas service 104 may provide services to support different visual representations of code. For example, a canvas may be configured to receive code as user input and transform the received code into a distinct code block graphical element on the canvas. In some embodiments, connections between clients 102 and the canvas service 104 are persistent bidirectional communications channels (e.g., using WebSockets, gRPC bidirectional streaming, or any other suitable protocol) where clients 102 may make requests and the canvas service 104 may provide asynchronous responses and updates. In some embodiments, connections between clients 102 and the canvas service 104 are initiated by clients 102 with synchronous responses from the canvas service 104. In such cases, updates may be requested by clients 102 through a polling mechanism. As shown in FIG. 1, the canvas service 104 is represented in the system 100 as a single service, but it should be understood that the canvas service 104 may include multiple services. For example, individual functions of the canvas service 104 may be separated into independent services or actors (e.g., using a microservice architecture).

[0045] In some embodiments, the canvas service 104 authenticates clients 102 through a user service 112. A client 102 may provide user credentials or an authentication token to the canvas service 104, which may then be forwarded to the user service 112. In some embodiments, the user service 112 stores and retrieves user credentials from a user credential and permissions store114. The user service 112 may also use the user credential and permissions store 114 to validate the user credentials or authentication token provided by clients 102. In some embodiments, the user service 112 is implemented as one or more services (e.g., microservice) inside a cluster (e.g., Kubernetes cluster) that accepts requests (e.g., Hypertext Transfer Protocol (HTTP) requests) from the canvas service 104.

[0046] In some embodiments, the user service 112 forwards the user credentials or authentication token provided by clients 102 to an external identity provider 134, which may validate user credentials or authentication tokens and respond to user service 112 accordingly. In some embodiments, the external identity provider provides single sign-on (SSO) functionality. Some non-limiting examples of providers of SSO functionality include Google, Okta, and GitHub.

[0047] The user credential and permissions store 114 may include persistent storage for storing user credentials, permissions, configuration (e.g., feature enablement flags, compute resource limits, group associations, user / administrative roles, canvas read / write access permissions, etc.), etc. The user credential and permissions store 114 may be implemented with a relational database (e.g., a SQL database like Postgres, MariaDB, MySQL, Oracle, AWS RDB, AWS Aurora, AWS Athena, GCP Cloud SQL, Azure SQL, etc.). In some embodiments, the user credential and permissions store 114 is implemented with a custom permissions database (e.g., Google Zanzibar, OryKeto, etc.). In some embodiments, the user credential and permissions store 114 is implemented with a document database (e.g., AWS DynamoDB, MongoDB, etc.), a flat file, or any other persistent storage medium.

[0048] In one embodiment, the canvas service 104 authorizes requests from clients 102 through user service 112. The user service 112 stores and retrieves user permissions associated with the user of the client, for example, to determine whether the use is authorized to access, modify, and / or execute a workflow. In some embodiments, the user service 112 returns a bundle of all or a subset of permissions associated with the user to the canvas service 104 so that the canvas service 104 can cache the permissions and use them for fine-grained orchestration of different canvas service 104 functions. In some embodiments, the user service 112 looks up a specific permission and responds to the canvas service 104 with an indication (e.g., Boolean values) of whether the user has permission.

[0049] The canvas service 104 may store and retrieve code from the code store 116. The code store 116 may include one or more computer programs written in a human- or computer- readable programming language, also referred to herein as “code.” A computer program may be partitioned (e.g., by a user) into one or more blocks of code (referred to herein as code blocks 130), which may include one or more lines of the computer program (e.g., a function, method, library, or any set of instructions of the computer program). For example, the canvas service 104 may provide a graphical user interface (e.g., a canvas) for the user, and the user may interact with the graphical user interface to create a code block interface element representing a code block 130. Once the code block interface element is created, the user may input one or more lines of code into the code block interface element. The collection of the code in the code block interface elements may collectively represent a computer program. In some embodiments, the code in a code block may represent a computer program.

[0050] The plurality of code blocks 130 may have inter-dependencies with respect to a runtime state 128, including but not limited to program inputs (e.g., data sets, machine learning (ML) models, configuration flags, etc.) and outputs (e.g., data tables, graphs, plots, images, etc.), method inputs and outputs, variable states, stack and heap memory states, etc. In some embodiments, the code in the code store 116 is custom code (e.g., proprietary source code) provided by a user (e.g., a client 102) of the system 100.

[0051] A runtime state 128 may also be dependent upon a runtime configuration 124, which includes but is not limited to operating system type (e.g., Linux kernel version, processor architecture, instruction set extensions, etc.), resource requirements (e.g., virtual CPUs, random access memory (RAM), network bandwidth, etc.), additional hardware requirements (e.g., graphics processing units (GPUs), ephemeral block storage, persistent block storage, security isolation, etc.), software compiler, interpreter, SDK version and configuration (e.g., Java virtual machine (JVM) parameters, Java SDK version, Python interpreter version, Ruby interpreter version, Node.js version, etc.), third party software package dependencies, etc.

[0052] In addition to the plurality of code blocks 130, the code store 116 may store a transaction history 126, which may include a history of software development transactions related to the code blocks (e.g., a “version history”) and a history of runtime transactions involving the code blocks (e.g., instances of execution of versions of the code blocks in runtimeenvironments) (e.g., a “runtime history”).

[0053] A version history of a code block 130 may include information about various versions of the code block 130, such as who changed the code, when the code was changed, what code was changed, how the code was changed, etc. In some embodiments, the version history may include additional information about the most recent software development transactions for a code block 130 (e.g., code changes pending commit, code changes queued to be processed, any errors processing a code change, indications that code changes are complete and stable, etc.).

[0054] A runtime history of a code block 130 may include information about various instances in which the code block 130 was executed (e.g., a “runtime data”), such as who executed the code, when was the code executed, where was the code executed, how was the code executed, etc. In some embodiments, the runtime history may include data associating the code block 130 to runtime environments, such as pointers to the runtime configurations 124 used to initialize runtime environments for runtime transactions as well as pointers to runtime states 128 associated with (e.g., generated by, used to initialize, etc.) the runtime environments during the runtime transactions. In some examples, a previous version (e.g., of a code block 130, runtime configuration 124, runtime state 128, workflow, etc.) may be referred to as a “historical version.”

[0055] In some embodiments, the code store 116 is implemented with a relational database (e.g., a SQL database like Postgres, MariaDB, MySQL, Oracle, AWS RDB, AWS Aurora, AWS Athena, GCP Cloud SQL, Azure SQL, etc.). In some embodiments, the code store 116 is implemented with a document database (e.g., AWS DynamoDB, MongoDB, etc.). In some embodiments, the code store 116 is implemented as a flat file, or any other persistent storage medium. In some embodiments, the code store 116 is a remote source code repository (e.g., GitHub, GitLab, Bitbucket, etc.).

[0056] Connecting to a code store 116 may require credentials. The canvas service 104 may retrieve credentials from the credential service 120 to provide to the code store 116. The credential service 120 may store credentials (e.g., encrypted credentials) per user and per service in a persistent credential store 122. In some embodiments, the credential store 122 is a remote credential store (e.g., HashiCorp Vault, AWS KMS, Azure Key Vault, GCP Secret Manager, etc.). In some embodiments, the credential store 122 is implemented with a relational database (e.g., a SQL database like Postgres, MariaDB, MySQL, Oracle, AWS RDB, AWS Aurora, AWSAthena, GCP Cloud SQL, Azure SQL, etc.), a document database (e.g., AWS DynamoDB, MongoDB, etc.), a flat file, or any other persistent storage medium.

[0057] For the canvas service 104 to independently process a plurality of code blocks 130, a runtime configuration 124 and a runtime state 128 may be utilized. A runtime configuration 124 may specify how one or more code blocks 130 are to be run. For example, the runtime configuration 124 for a code block 130 may specify an amount of memory, a type of Al accelerator, a type of processor, etc. The runtime configuration 124 for each code block 130 may be stored in a configuration store 118. In some embodiments, the runtime configuration 124 is stored as a collection of files in a configuration store 118 implemented in cloud storage (e.g., in AWS S3, Azure Blob Storage, Google Storage, etc.). In some embodiments, the runtime configuration 124 for each code block 130 is stored as a collection of files in a configuration store 118 implemented on a storage area network (SAN), network attached storage (NAS), or local disk. In some embodiments, the runtime configuration 124 for each code block 130 is stored in a configuration store 118 implemented in a relational database (e.g., a SQL database like Postgres, MariaDB, MySQL, Oracle, AWS RDB, AWS Aurora, AWS Athena, GCP Cloud SQL, Azure SQL, etc.) or a document database (e.g., AWS DynamoDB, MongoDB, etc.).

[0058] The runtime state 128 associated with a code block 130 (e.g., a version of a code block 130) may be stored in a runtime store 110. A runtime state 128 may be a record of the conditions of a runtime environment at any point throughout a workflow (e.g., the execution of one or more code blocks 130). For example, the runtime state 128 may include a set of running processes, persistent data (e.g., input data to a process, output data from a process), temporary data (e.g., caches), etc. In some embodiments, the runtime state 128 is stored as a collection of files in a runtime store 110, which may be implemented in cloud storage (e.g., in AWS S3, Azure Blob Storage, Google Storage, etc.). In some embodiments, the runtime state 128 for each code block 130 is stored as a collection of files in a transaction store implemented on a storage area network (SAN), network attached storage (NAS), or local disk. In some embodiments, runtime state(s) 128 for each code block 130 is stored in a runtime store 110 implemented in a relational database (e.g., a SQL database like Postgres, MariaDB, MySQL, Oracle, AWS RDB, AWS Aurora, AWS Athena, GCP Cloud SQL, Azure SQL, etc.) or a document database (e.g., AWS DynamoDB, MongoDB, etc ).

[0059] Though the code store 116, configuration store 118, and runtime store 110 are logically represented as separate entities in system 100, it should be understood that any combination of the code store 116, configuration store 118, and / or runtime store 110 could reside in the same physical or logical persistence store or be further subdivided into separate physical or logical persistence stores. It should also be understood that the functionality described herein is not dependent on the underlying storage architecture.

[0060] The canvas service 104 may dispatch tasks for processing one or more code blocks 130 by sending a task request to the executor service 106. The task request may include the code blocks 130 (e.g., retrieved from the code store 116), the runtime configuration 124 (e.g., retrieved from the configuration store 118) to create a runtime environment, and pointers (e.g., retrieved from the code store 116) to the runtime state 128 in the runtime store 110 to initialize the runtime environment.

[0061] The executor service 106 may manage compute nodes 108. In some embodiments, compute nodes 108 are ephemeral, serverless compute nodes (e.g., AWS Lambda, Azure Functions, GCP Cloud Functions, Kubernetes Knative functions, etc.). In some embodiments, compute nodes 108 may be dedicated compute nodes (e.g., AWS EC2, Azure Virtual Machines, GCP GCE, bare metal servers, etc.).

[0062] The executor service 106 may map task requests to compute nodes 108. The executor service 106 may also manage the creating and tearing down of compute nodes 108 (e.g., to meet demand), limit the number of compute nodes 108 that can be created, monitor the state of tasks running on compute nodes 108, and terminate compute nodes 108 that are stalled or otherwise unhealthy.

[0063] In order to execute a code block 130, the runtime environment may first be provided. A runtime environment may include a set of resources and services that a computer program may utilize when running, such as operating systems, libraries, dependencies, and configuration. For example, a runtime environment may include an operating system (e.g., Windows or Linux), a virtual machine (e.g., VMware or Virtual Box), a container (e.g., Docker or Kubernetes), a cloud platform (e.g., AWS or Azure), etc., running on a compute node 108.

[0064] A runtime environment may be provided from a pool of existing runtime environments (e.g., idle or running on a set of compute nodes 108) having a runtimeconfiguration 124 and set to a runtime state 128. Additionally or alternatively, a runtime environment may be provided by first creating a new runtime environment (e.g., by an available compute node 108) using the runtime configuration 124 of the task request and initialized using the runtime state 128 (e.g., as pointed to by the runtime state 128 pointers of the task request). In some embodiments, the executor service 106 may first determine whether there is a suitable runtime environment available (e.g., matching the runtime configuration 124 in the task request) and, if not, assign a compute node 108 to create a new runtime environment (e.g., a virtual machine) with the runtime configuration 124 in the task request.

[0065] Once a runtime environment is available on a compute node 108, the executor service 106 may initialize the runtime by sending a configuration message to the compute node 108. The configuration message may include the runtime state 128 retrieved from the runtime store 110 (e.g., as pointed to by the runtime state 128 pointers provided in the task request). The runtime state 128 may be the start of a workflow (e.g., for running the entire workflow) or an intermediate state of the workflow (e.g., for resuming a previously incomplete execution of the workflow or otherwise executing a subset of the workflow). Once the runtime environment is initialized, the executor service 106 initiates processing of the plurality of code blocks 130. For example, the executor service 106 may send a run message to the compute node 108, which may include the plurality of code blocks 130 to be processed.

[0066] In some embodiments, task requests contain custom data retrieval tasks as part of the runtime creation (e.g., retrieving custom software package dependencies including ML models) and / or runtime initialization (e.g., retrieving input datasets from external databases or data services like Snowflake, Databricks, AWS RDB, AWS Aurora, AWS Athena, GCP Cloud SQL, Azure SQL, AWS DynamoDB, MongoDB, AWS S3, Azure Blob Storage, Google Storage, etc.). In these cases, the canvas service 104 may retrieve any credentials necessary to access data from external databases, data warehouses, or other services (e.g., Databricks, Snowflake, etc.) from the credential service 120 to include with the custom data retrieval task.

[0067] In some embodiments, the executor service 106 assigns task requests to execute in parallel if sufficient compute nodes 108 are available. If there are insufficient compute nodes 108 to execute a task request, the executor service 106 may notify the canvas service 104 of a failure to execute the task request.

[0068] The canvas service 104 may enforce dependencies between task requests. For example, if a plurality of code blocks A, B, and C are to be run in serial, the canvas service 104 may confirm B does not run before A and may confirm C does not run before B. As another example, if two users request to change and process the same code block 130, the canvas service 104 may resolve which user has priority (e.g., the first user to request, the user with higher permissions, etc.).

[0069] The canvas service 104 may also or instead synchronize inter-dependent task request completion. For example, if a code block C depends on both code blocks A and B, but A and B are independent of each other, the canvas service 104 may confirm that C is not executed until both A and B have completed and a coherent runtime state for C is created.

[0070] The canvas service 104 may also or instead retry failed task requests. The canvas service 104 may also or instead enforce compute limits on task execution based on user permissions (e.g., user license, user role, user group, subscription level, etc.) retrieved from the user service 112.

[0071] The compute nodes 108 may execute their assigned code blocks 130 in their configured and initialized runtime environment. As the plurality of code blocks 130 execute, the corresponding compute node 108 may track the runtime state 128, which may include program outputs (e.g., data tables, graphs, plots, images, etc.), method inputs and outputs, variable states, stack and heap memory state, runtime errors and exceptions, stack traces, core dumps, log messages, program metrics, host metrics (e.g., CPU usage, memory usage, network usage, etc.), runtime, etc. The compute node 108 may write the runtime state 128 to the runtime store 110, which may index runtime states 128 by task. In some embodiments, where the runtime store 110 is implemented as cloud storage, the runtime state 128 for a given task is stored as a collection of files. In some embodiments, where the runtime store 110 is implemented as a SAN, NAS, or local disk, the runtime state 128 for a given task is stored as a collection of files. In some embodiments, where the runtime store 110 is implemented as a transactional database, relational database, or document database, the runtime state 128 for a given task is stored in a database. The runtime state 128 may be used as a checkpoint to repeat the execution a code block 130 at a particular point in time.

[0072] In some embodiments, the runtime store 110 is a remote storage service. In someembodiments, connecting to the runtime store 110 requires credentials. If credentials are used to connect to the runtime store 110, the canvas service 104 may retrieve any credentials necessary to write data to the remote storage service from the credential service 120 and may include the credentials in the task request to the executor service 106. The executor service 106 may then provide the credentials to the compute node 108 executing the task so that the compute node 108 can write the runtime state 128 to the runtime store 110 (e.g., remote storage service). In some embodiments, compute nodes 108 write task execution progress to the runtime store 110. In some embodiments, compute nodes 108 publish periodic notifications with execution progress to the executor service 106. In some embodiments, the executor service 106 periodically polls compute nodes 108 for execution progress. In some embodiments, the executor service 106 periodically publishes task execution status reports to the canvas service 104, and the task execution status reports may be used to update transaction histories 126 associated with the code blocks 130 in the code store 116. In some embodiments, the canvas service 104 periodically polls the executor service 106 for task execution status, and the task execution status may be used to update transaction histories 126 associated with the code blocks in the code store 116.

[0073] In some embodiments, the canvas service 104 sends real-time status updates to clients 102. For example, the canvas service 104 may send runtime state 128 updates to clients 102 upon task completion. In some embodiments, clients 102 retrieve runtime state 128 for completed tasks from the canvas service 104. In some embodiments, the canvas service 104 redirects clients 102 (e.g., via HTTP redirect) to download a runtime state 128 directly from the runtime store 110.

[0074] In some embodiments, the canvas service 104 keeps track of all client 102 connections associated with a given user group and sends real-time status updates to all clients 102 associated with a given user group. In some embodiments, the canvas service 104 keeps track of all client 102 connections associated with a given plurality of code blocks 130 and sends real-time status updates to all clients 102 associated with a given plurality of code blocks 130. Clients 102 may be grouped in a variety of ways (e.g., by user group or by code blocks 130), and so it should be understood that the canvas service 104 is capable of supporting any and all potential groupings of clients 102 for the purposes of sending status updates (e.g., real-time or requested status updates).

[0075] FIG. 2 is a flow chart of a process 200 for executing a plurality of code blocks 130. The process 200 may be performed by the canvas service 104 of FIG. 1. The canvas service 104 may process requests from a plurality of clients 102, each of which may request operations on the plurality of code blocks 130. As the process 200 relates to the canvas service 104, the following discussion of FIG. 2 may be made with references to FIG. 1.

[0076] At step 202 of the process 200, a request may be received by the canvas service 104 from a client 102.

[0077] At step 232, the request may be authenticated and / or authorized. The canvas service 104 may authenticate the client 102 through the user service 112. The client 102 may provide user credentials or an authentication token to the canvas service 104, which the canvas service 104 may forward to the user service 112 for authentication. In some embodiments, the user service 112 stores and retrieves user credentials from a user credential and permissions store 114, and the user service 112 may use the credential and permissions store 114 to validate the user credentials or authentication token provided by the client 102. In some embodiments, the user service 112 forwards the user credentials or authentication token provided by the client 102 to an external identity provider 134 to validate the user credentials or authentication token provided by the client 102.

[0078] The canvas service 104 may also authorize requests from the client 102 through user service 112. The user service 112 may store and retrieve (e.g., from the user credential and permissions store 114) user permissions associated with the user associated with the request.

[0079] If either authentication or authorization fails, the process 200 may proceed to step 234 where the canvas service 104 rejects the request. Otherwise, the process 200 may proceed to steps 204, 206, or 208. If the request relates to creating or modifying a code block 130, the process 200 may proceed to step 204. If the request relates to the processing of a plurality of code blocks 130, the process 200 may proceed to step 206. If the request relates to updating the runtime configuration 124 for a plurality of code blocks 130, the process 200 may proceed to step 208.

[0080] At step 204, the client 102 may request to create or modify a code block 130. The code block 130 may be part of a larger computer program in the code store 116. Creating a new code block 130 may include receiving one or more lines of code for a computer program in thecode store 116. Modifying a code block 130 may include changing one or more lines of code in the code block 130, splitting an existing code block 130 into one or more code blocks 130, combining two or more code blocks 130 into a smaller set of code blocks 130, or deleting one or more code blocks 130.

[0081] At step 206, the client 102 may request to process one or more code blocks 130 of a computer program. Processing may include running (e.g., executing) a code block 130 that has never been run before (and therefore has no prior runtime state 128). Processing may also or instead include running a code block 130 that has been run previously and has not been modified since the previous run. In this case, the code block 130 may have an existing runtime state 128 that can be reused. Processing may also or instead include running a code block 130 that has been run previously but has been modified since the previous run. In this case, the code block 130 may have an existing runtime state 128 that may be re-initialized or modified for use.

[0082] At step 208, the client 102 may request to update the runtime configuration 124 for one or more code blocks 130. Updating the runtime configuration 124 may include creating a runtime configuration 124 for a new code block 130, which has never before been processed. Updating the runtime configuration 124 may also include deleting a runtime configuration 124 for an existing code block 130. Updating the runtime configuration 124 may also include modifying the runtime configuration 124 for an existing code block 130, when the existing code block 130 would run in a new runtime environment (e.g., while being processed at step 206). In some embodiments, the client 102 request may include a combination of runtime configuration 124 updates, code block 130 modifications, and / or code block 130 processing.

[0083] The process 200 may proceed from steps 204, 206, and 208 to step 210.

[0084] At step 210, it is determining whether the request can be processed immediately. The canvas service 104 enforces dependencies between task requests. For example, the canvas service 104 may enforce a particular order for code blocks 130 to run. As another example, if multiple users request changes to a code block, the canvas service 104 may resolve conflicts between the users (e.g., by prioritizing one user over another and executing their requests in serial). The canvas service 104 may also synchronize inter-dependent task request completion. For example, if a code block C depends on both code blocks A and B, but A and B are independent of each other, the canvas service 104 may verify that C is not executed until both Aand B have completed and a coherent runtime state for C is created. The canvas service 104 may also manage the retrying of failed task requests.

[0085] If the request is dependent upon one or more existing requests in the transaction history 126, the process 200 may continue to step 212 where the request is queued for future processing. At step 212, the canvas service 104 may create a new entry in the transaction history 126 to track the request. Once the existing requests on the queue have been completed, the current request may be re-evaluated at step 210.

[0086] If the request is not dependent upon one or more existing requests in the transaction history 126, the process 200 may proceed to step 214.

[0087] Although steps 214, 216, and 218 are presented sequentially, steps 214, 216, and 218 may be performed in any order, sequentially or in parallel.

[0088] At step 214, if the request includes creating or modifying a code block 130, the process 200 may continue to step 222 before proceeding to step 216; otherwise, the process 200 may continue directly to step 216.

[0089] At step 222, code blocks 130 are added, removed, and / or modified in the code store 116 according to the request. In some embodiments, the code store 116 is a remote source code repository (e.g., GitHub, GitLab, Bitbucket, etc.). In some embodiments, connecting to the code store 116 (e.g., remote source code repository) requires credentials. The canvas service 104 may retrieve credentials from the credential service 120 to provide to the code store 116. In some embodiments, when code blocks 130 are added to the code store 116 according to the request, runtime configurations 124 corresponding to each new code block 130 may also be created in the configuration store 118. In some embodiments, when code blocks 130 are removed from the code store 116 according to the request, runtime configurations 124 corresponding to each code block 130 may also be deleted from the configuration store 118.

[0090] Code block 130 changes (e.g., addition, removal, and / or modification to some or all of a code block 130) may be reflected by an entry in the transaction history 126. In some embodiments, sequential code block 130 changes may be aggregated into a single transaction history 126 entry. In some embodiments, sequential code block 130 changes are each given separate transaction history 126 entries.

[0091] At step 216, if the request includes a runtime configuration 124 update, the process200 may continue to step 224 before proceeding to step 218; otherwise, the process 200 may continue directly to step 218.

[0092] At step 224, one or more runtime configurations 124 for code blocks 130 specified by the request are modified in the configuration store 118 according to the request. In some embodiments, runtime configuration 124 changes may be sent by the canvas service 104 to the executor service 106 for creating new runtime environments, prior to a request for processing of a plurality of code blocks 130.

[0093] At step 218, if the request includes code block 130 processing, the process 200 may continue to step 226 before proceeding to step 220; otherwise, the process 200 may continue directly to step 220.

[0094] At step 226, the canvas service 104 may create a task request to send to the executor service 106. The task request may include the plurality of code blocks 130 retrieved from the code store 116, the runtime configuration 124 used to create the runtime environment (e.g., retrieved from the configuration store 118), and the runtime state 128 in the runtime store 110, all of which may be used to initialize the runtime environment. In some embodiments, the task request may also include custom data retrieval tasks as part of the runtime creation (e.g., retrieving custom software package dependencies including ML models) and / or runtime initialization (e.g., retrieving input datasets from external databases or data service like Snowflake, Databricks, AWS RDB, AWS Aurora, AWS Athena, GCP Cloud SQL, Azure SQL, AWS DynamoDB, MongoDB, AWS S3, Azure Blob Storage, Google Storage, etc.). If the task request includes custom data retrieval tasks, the canvas service 104 may retrieve credentials from the credential service 120 to access data from external databases or services and include the credentials with the custom data retrieval task. Once the task request is dispatched to the executor service 106, the process 200 may continue to step 220.

[0095] In some embodiments, the executor service 106 assigns task requests to execute in parallel as long as there are sufficient compute nodes 108 available. If there are insufficient compute nodes 108 to execute a task request, the executor service 106 may notify the canvas service 104 of a failure to execute the task request. In some embodiments, the executor service 106 periodically publishes task execution status reports to the canvas service 104, which may be used to update transaction histories 126 in the code store 116. In some embodiments, the canvasservice 104 periodically polls the executor service 106 for a task execution status, which may be used to update transaction histories 126 in the code store 116.

[0096] In step 220, the canvas service 104 sends a response to the client 102 for the client request received in step 202. In some embodiments, the canvas service 104 sends periodic, asynchronous real-time status updates to the client 102. In some embodiments, the canvas service 104 sends runtime state 128 updates to the client 102 upon task completion. In some embodiments, the client 102 retrieves runtime state 128 for completed tasks from the canvas service 104. In some embodiments, the canvas service 104 redirects clients 102 (e.g., via HTTP redirect) to download a runtime state 128 directly from the runtime store 110. In some embodiments, the canvas service 104 keeps track of all client 102 connections associated with a given user group and sends real-time status updates to all clients 102 associated with the user group. In some embodiments, the canvas service 104 keeps track of all client 102 connections associated with a given plurality of code blocks 130 and sends real-time status updates to all clients 102 associated with the plurality of code blocks 130. It should be understood that clients may be grouped in a variety of ways and that the canvas service 104 is capable of supporting any and all potential groupings of clients 102 for the purposes of sending status updates (e.g., realtime status updates).

[0097] FIG. 3 is a flow chart of a process 300 for exporting and replaying a plurality of code blocks. The process 300 may be performed by the canvas service 104 of FIG. 1. The canvas service 104 may export a plurality of code blocks 130 with corresponding runtime states 128 and / or runtime configurations 124 to reproduce a task execution from the transaction history 126. As the process 300 relates to the canvas service 104, the following discussion of FIG. 3 may be made with references to FIG. 1.

[0098] At step 302 of the process 300, an export request may be received by the canvas service 104 (e.g., from one of a plurality of clients 102). The export request may identify, for example, the code blocks 130 to be exported (e.g., the versions of the code blocks 130 to be exported). In such a case, the runtime states and / or runtime configurations associated with the requested code blocks may be identified based on the records of the transaction history 126 associated with the indicated code blocks. In some examples, the export request identifies a runtime transaction for which a runtime environment (or an interface to the runtime 1environment) is to be exported. In such examples, the runtime states and / or runtime configurations associated with the identified runtime transaction may be identified by the record of the runtime transaction in the transaction history 126. In some examples, the export request indicates that the code blocks and runtime states are to be exported as program code (e.g., source code) suitable for compilation and execution. In some examples, the export request indicates that a runtime environment or an interface to a runtime environment (e.g., a runtime environment wrapper, a Docker image, a Docker container, etc.) is to be exported, such that a suitable runtime environment (e.g., a runtime environment conforming to the runtime configuration(s), initialized with the runtime state(s), and loaded with the code block(s)) can be accessed via the interface.

[0099] At step 316, the request may be authenticated and / or authorized. The canvas service 104 may authenticate the client 102 through the user service 112. The client 102 may provide user credentials or an authentication token to the canvas service 104, which the canvas service 104 may forward to the user service 112 for authentication. In some embodiments, the user service 112 stores and retrieves user credentials from a user credential and permissions store 114, and the user service 112 may use the credential and permissions store 114 to validate the user credentials or authentication token provided by the client 102. In some embodiments, the user service 112 forwards the user credentials or authentication token provided by the client 102 to an external identity provider 134 to validate the user credentials or authentication token provided by the client 102.

[0100] The canvas service 104 may also authorize requests from the client 102 through user service 112 (e.g., to determine whether the canvas service 104 may facilitate the export request for the client 102). The user service 112 may store and retrieve (e.g., from the user credential and permissions store 114) user permissions associated with the user associated with the request.

[0101] If either authentication or authorization fails, the process 300 may proceed to step 318 where the canvas service 104 rejects the request. Otherwise, the process 300 may proceed to step 304.

[0102] At step 304, the canvas service 104 may collect (e.g., access, retrieve, download, etc.) the requested set of code blocks 130 (e.g., from the code store 116), the associated set of runtime configurations 124 (e.g., from the configuration store 118 as pointed to by the relevant record(s) of the transaction history 126), and the associated set of runtime states 128 (e.g., from theruntime store 110 as pointed to by the relevant record(s) of the transaction history 126). In some embodiments, the export includes a runtime environment generated from a runtime configuration 124 and runtime state 128, both of which may be associated with one or more code blocks 130 (e.g., as pointed to by the relevant record of the code block (e.g., the latest version of the code block) in the transaction history 126). In some embodiments, the export request specifies a particular version of the code block in the transaction history 126, for which specific historical versions of the code blocks 130 are retrieved from the code store 116, specific historical versions of the runtime configuration 124 are retrieved from the configuration store 118, and specific historical versions of the runtime state 128 are retrieved from the runtime store 110. In some embodiments, the export request specifies a particular runtime state of the workflow associated with the requested set of code blocks 130. For example, the export request may specify an intermediate runtime state associated with a previous incomplete execution of the workflow (e.g., that did not execute each code block 130). This way, the requester can resume the execution of the workflow from a particular point of the workflow.

[0103] In some examples, servicing the export request includes exporting the plurality of code blocks 130 as part of a runtime environment for offline execution. For example, the exported code blocks may be used for offline debugging, demonstration purposes, etc. In some examples, servicing the export request includes exporting the plurality of code blocks 130 with runtime state 128 converted to source code. For example, the exported source code may be used as part of a unit test, integration test, smoke test, canary test, etc.

[0104] At step 306, the canvas service 104 may determine whether the export request includes a request to export to source code. If the export request includes a request to export to source code, the process 300 may continue to step 308 before proceeding to step 310; otherwise, the process 300 may continue directly to step 310.

[0105] At step 308, the canvas service 104 may combine the plurality of code blocks 130 and associated runtime state 128 (e.g., as pointed to by the transaction history 126) into source code files in a programming language (e.g., Java, Python, C, Scala, Ruby, Go, R, etc.) specified by the export request. In some embodiments, each code block 130 is implemented as a method, the runtime state 128 (including the inputs to the method implementing the code block 130) is implemented as static variables, and a test framework corresponding to the programminglanguage (e.g., JUnit, pytest, CUnit, RSpec, etc.) is used to associate the runtime state 128 with the code blocks 130 to create an independently runnable (e.g., executable) source code program suitable for reproducible execution of code blocks 130.

[0106] At step 310, the canvas service 104 may determine whether the export request includes a request to export a runtime environment or an interface to a runtime environment (e.g., a runtime environment wrapper, a Docker image, a Docker container, etc.). If the export request includes a request to export a runtime environment or an interface to a runtime environment, the process 300 may continue to step 312 to configure the runtime environment and / or generate an interface to the runtime environment before proceeding to step 314; otherwise, the process 300 may continue directly to step 314.

[0107] At step 312, the canvas service 104 may request a runtime environment and / or an interface thereto. The runtime environment may conform to the runtime configuration 124 associated with the code block(s) 130. The runtime environment may be initialized using the runtime state(s) 128 associated with the code block(s) 130 (e.g., by the executor service 106). The runtime environment may include the code block(s) 130 (e.g., may be loaded with the code block(s) 130 by the executor service 106). If requested, a runtime wrapper (e.g., a script capable of modifying parameters of the runtime environment) may be generated (e.g., by the executor service 106).

[0108] The executor service 106 may determine whether a suitable runtime environment is available for export. If a suitable runtime environment is not available, the executor service 106 may assign a compute node 108 to generate a new runtime environment (e.g., build a Docker container) with the runtime configuration 124. Once the runtime environment is available, the executor service 106 may initialize the runtime environment using the runtime state 128, load the code block(s) 130 into the runtime environment, and return the initialized runtime environment (e.g., the virtual machine or Docker container) or an interface thereto.

[0109] In some embodiments, the runtime environment (or interface thereto) is saved to the runtime store 110, and the executor service 106 may respond to the request (e.g., via the canvas service 104) with a pointer to the runtime environment (or interface thereto) in the runtime store 110. In some embodiments, the canvas service 104 may retrieve (e.g., access, obtain, download, etc.) the runtime environment (or interface thereto) from the runtime store 110 to return to theclient 102 in response to the request. In some embodiments, the canvas service 104 stores only the pointer to the runtime environment (or interface thereto) in the runtime store 110 to return to the client in response to the request. The pointer to the runtime environment (or interface thereto) in the runtime store 110 may also or instead be stored in the transaction history 126. In some embodiments, a runtime wrapper may be or may be part of a runtime environment.

[0110] Though steps 306 and 310 are presented sequentially, steps 306 and 310 may be performed in any order, sequentially or in parallel.

[0111] At step 314, the canvas service 104 may return the exported runtime environment (or interface thereto) to the client 102 in response to the request. The exported runtime environment (or interface thereto) may be used for subsequent execution of the code blocks 130 (e.g., by the requester). In embodiments, the export includes an archive file with all the raw code blocks 130 and associated runtime state 128 (e.g., as pointed to by the transaction history 126). In some embodiments, the export includes an application container (e.g., a Docker container, an OCI container, etc.) containing the code blocks 130 built into a runtime environment initialized with the associated runtime state 128 (e.g., as pointed to by the transaction history 126). In some embodiments, the export includes an archive file with the generated source code files including the code blocks 130 and the associated runtime state 128 (e.g., as pointed to by the transaction history 126). It should be understood that there may be many possible export formats, and any suitable archive format can be generated at step 314.

[0112] Some examples of a canvas service 104 have been described. In some embodiments, the canvas service 104 stores and retrieves code blocks 130 (e.g., from a code store 116) of a workflow; stores and retrieves credential information (e.g., from a credential store 122) used to access data processed by code blocks 130 of a workflow; stores scheduled times to run code blocks 130; stores data regarding invocations (e.g., in runtime store 110) of a workflow and of the steps of a workflow; retrieves results (e.g., runtime state 128) corresponding to runs of a code block 130 in the workflow; communicates with an executor service 106 to send code blocks 130 for execution on computational resources (e.g., compute nodes 108), runtime configuration 124 data indicating the runtime configuration 124 in which the code blocks 130 run (e.g., the computational resources used to run the code blocks 130), and runtime state 128 data indicating the initial state of the code block 130 (e.g., inputs to the code block) and the location where thefinal state of the code block 130 (e.g., outputs from the code block) are to be stored; and communicates with the executor service 106 to obtain the results of a run of a code block 130, to orchestrate the next task to run, etc. In some embodiments, a code block 130 may implement a data preparation operation, a modeling operation (e.g., an ML model, one or more layers of an ML model, one or more portions of an ML model, etc.), or any other Al operation.

[0113] A workflow may include a plurality of code blocks 130 arranged (e.g., connected) as nodes on a canvas (e.g., a visual representation of code) to form a graph (e.g., a dataflow graph). In some embodiments, the workflow graph can be directed and / or acyclic, indicating an order of execution.

[0114] Some examples of an executor service 106 (or “execution service”) have been described. In some embodiments, the executor service 106 submits code blocks 130 to run as tasks on computing resources (e.g., compute nodes 108). In some embodiments, the executor service 106 also provides information that can be used to run the tasks correctly on the computing resources (e.g., runtime configuration 124 data, runtime state 128 data, etc.). In some embodiments, the executor service 106 determines when a task corresponding to a code block 130 has finished executing and communicates the results to other components of the system 100 (e.g., to the canvas service 104). In some embodiments, the executor service 106 is implemented as a microservice.

[0115] Some examples of runtime configuration 124 data have been described. In some embodiments, runtime configuration 124 data may correspond to one or more code blocks 130 (e.g., a plurality of code blocks 130 arranged in a workflow graph). Such runtime configuration 124 data may indicate dependencies (e.g., control dependencies, data dependencies, etc.) between or among code blocks 130 in a workflow. In some embodiments, runtime configuration 124 data indicates attributes of the computational resources on which each (or all) of the code blocks 130 will run. For example, the runtime configuration 124 data may indicate that a code block 130 will run on one or more central processing units (CPUs), one or more graphics processing units (GPUs), a processing device with at least a specified amount of internal RAM, a compute node of a particular cloud computing environment, etc. In some embodiments, runtime configuration 124 data identifies the language of the source code contained in each of the code blocks 130. In some embodiments, runtime configuration 124 data identifies a type of distributedcomputing operation associated with (e.g., performed by) a code block 130. Some non-limiting examples of distributed computing operations may include data processing, data shuffling, data aggregation, validating the condition of a conditional loop, etc.

[0116] Some examples of a runtime store 110 have been described. In some embodiments, the runtime store 110 has an object storage architecture, which can be implemented in short-term storage (e.g., in RAM) or in long-term storage (e.g., solid-state drives, flash memory, hard drives, etc.). In some embodiments, the runtime store 110 provides a hierarchical directory structure in which the data associated with each project is stored. For example, the directory structure may include the following directories:• ‘canvases / ’,• ‘canvases / {canvas_id} / ’,• ‘canvases / {canvas_id} / blocks / ’, and• ‘canvases / {canvas_id} / blocks / {block_id} / ’.

[0117] The ‘canvases’ directory may include a sub-directory ‘ {canvas id} ’ corresponding to each unique canvas (also referred to herein as a “project”). A canvas may include two or more code blocks 130 arranged to form one or more workflows. For each code block 130 in a canvas, the corresponding canvas directory may include a sub-directory ‘ blocks / {block id}’ corresponding to the code block 130. The runtime state 128 of the workflow upon arriving at a code block 130 and / or upon exiting the code block 130 may be stored in the directory of the code block 130, ‘blocks / {block_id}’.

[0118] A runtime state 128 may include (1) the data processed by a unique instance of a code block 130 (e.g., any inputs to that instance of the code block 130, including register reads, memory reads, and the runtime state 128 when that instance of the code block 130 starts executing), and (2) the data generated by that instance of the code block 130 (e.g., any outputs produced by that instance of the code block 130, including register writes, memory writes, and the runtime state when that instance of the code block 130 stops executing). In some embodiments, the runtime state 128 corresponding to an instance of a code block 130 also includes metadata indicating computational performance characteristics of an instance of the code block 130 (e.g., resource utilization, runtime, etc.), memory addresses to which the code block 130 instance stored data, indications of whether the code block 130 instance caused anyerror conditions, and / or any other suitable data associated with an instance of the code block 130.

[0119] The runtime state 128 for an instance of a code block 130 can include metadata (e.g., a JSON metafile) identifying memory addresses (e.g., project variables) to which the instance of the code block 130 writes information. Likewise, the runtime state 128 for an instance of a code block 130 can include the values of those memory addresses when the instance of the code block 130 finishes running. In some embodiments, the canvas service 104 tracks and enforces the control flow and data flow dependencies between instances of code blocks 130 in a project. For example, if an instance of a first code block 130 stores a value to a memory address Al and an instance of a dependent code block 130 subsequently loads the value at that memory address Al, the canvas service 104 can delay the execution of the instance of the dependent code block 130 until the instance of the first code block 130 has finished executing so that the correct value of the memory address Al is available before the instance of the dependent code block 130 attempts to load it. Likewise, when the dependent code block 130 begins executing, the canvas service 104 can initialize the runtime state 128 of the dependent code block 130 such that the value stored at the memory address Al within the runtime state 128 of the dependent code block 130 when the dependent code block 130 begins executing is the value stored to the memory address Al by the instance of the first code block 130. Alternatively, the canvas service 104 and executor service 106 can support “lazy loading,” such that the canvas service 104 provides the value written to the memory address Al by the instance of the first code block 130 to the instance of the dependent code block 130 when the instance of the dependent code block 130 first attempts to load the value of the memory address Al, rather than when the instance of the dependent code block 130 begins executing.

[0120] When a workflow of a project includes a loop (e.g., a conditional loop), such that one or more code blocks 130 of the workflow are executed multiple times, the system 100 can treat each loop iteration as a distinct instance of the one or more code blocks 130, such that runtime state 128 at the beginning and / or end of each iteration of each code block 130 is stored in the runtime store 110. In another approach, the system 100 can treat all iterations of the loop as a single instance of the one or more code blocks 130 such that the runtime state 128 at the beginning of the first loop iteration and at the end of the final loop iteration are stored in the runtime store 110. The former approach can use considerably more storage than the latterapproach but facilitates fine-grained replay of individual loop iterations of interest, which enhances the system’s tracing and debugging capabilities and can enhance the reproducibility of events of interest (e.g., runtime bugs). The latter approach can use considerably less storage than the former approach at the expense of providing a more coarse-grained replay of the loop, which can make debugging more difficult, make tracing more time-consuming, and reduce the reproducibility of events of interest.

[0121] Examples of Software Development Techniques

[0122] Scalable and reproducible. In some embodiments, the system 100 supports the development (e.g., collaborative, scalable, and / or reproducible development) of workflows using the techniques described below.

[0123] Roles. In some embodiments, user(s) create workflows by creating code blocks 130 and configuring the code blocks 130 on a canvas. In some embodiments, the canvas service 140 orchestrates the workflows, and the executor service 106 dispatches and tracks individual tasks. In some embodiments, the compute nodes 108 runs (e.g., execute) the code, reads data from the runtime store 110, and writes data to the runtime store 110.

[0124] Process. In some embodiments, a process for the development of workflows includes one or more of the following steps.

[0125] 1) A user may create an account on the system 100 (or with a service provided by an operator of the system 100). A reference may be created for the user in the user service 112. For each request submitted by the user, the system 100 may determine (e.g., via the user credentials and permissions store 114) whether the user has permission to perform the requested task or access the requested resource. Access control may be applied at the canvas level. In some embodiments, more fine-grained permissions are provided to control access to runtime configuration and usage.

[0126] 2) The user may create a canvas, which may be stored by the canvas service 104 (e.g., in the code store 116). The canvas may have a unique identifier that may be used as a common reference across all the stores.

[0127] 3) The user may create a set of code blocks 130 through the user interface (UI), APIs,SDKs, etc. For example, the user may interact with the UI to create a code block interface element in the UI representing a code block 130, and then the user may input one or more linesof code into the code block interface element.

[0128] 4) The user may create dependencies between code blocks 130. Referring to FIG. 4, these dependencies may be visually represented as connections 402. The user may create the connections 402 through the user interface, APIs, SDKs, etc. For example, the user may draw a connection 402 in the UI between two code block interface elements 404 to create a connection between the two code blocks 130 represented by the two code block interface elements 404. Connected code blocks 130 may have access to the data created from upstream code blocks 130 (e.g., code blocks 130 executed to their left in the graph 400). Connections 402 may inform the canvas service 104 which code blocks 130 to execute as part of a workflow and in what order to execute the code blocks 130.

[0129] 5) The user may configure the compute node 108 that each code block 130 runs on and indicate whether each code block 130 is used to split input data, shuffle input data, aggregate input data, or act as a logical gate or conditional loop within the workflow. Input data may refer to data provided as parameters when invoking the workflow or the runtime state 128 saved in the runtime store 110.

[0130] 6) The user may submit a workflow (e.g., a reference to a workflow) to the canvas service 104 for the canvas service 104 to run one or more code blocks 130 of the workflow. The user may provide input data for the workflow, which may become data available to the first code blocks 130 in the graph 400.

[0131] 7) The canvas service 104 may create one or more database entries (e.g., subdirectories in the directory structure of the runtime store 110) to track the progress of the workflow’s execution.

[0132] 8) The canvas service 104 may obtain (e.g., receive, download, access, etc.) the code blocks 130 (e.g., from the code store 116) for the workflow.

[0133] 9) The canvas services 104 may obtain data indicating the dependencies between the code blocks 130 and may build the graph (e.g., a control-flow and / or data-flow dependence graph).

[0134] 10) The canvas service 104 may request the executor service 106 to run the code blocks 130 as one or more tasks. For example, each task may be associated with a request to run a code block 130. Each task provided to the executor service 106 may include:10a) Data identifying the code blocks 130 to run;10b) Data identifying the programming language in which code blocks 130 are programmed;10c) The runtime configuration on which each code block 130 is executed; lOd) Data identifying the connections 402 between code blocks 130 (so the executor service 106 can correctly load the data); lOe) Input data (if provided); lOf) The block id and canvas id so that the correct storage paths can be constructed;10g) The semantics of the code block 130 within the context of the workflow (e.g., a conditional step, a data aggregation operation, etc.); and lOh) Any authentication information suitable for running the workflow within a cloud environment.

[0135] 11) The executor service 106 may use the programming language and runtime configuration information to trigger the code block 130 to run in the correct compute node 108 (e.g., a compute node 108 configured to interpret and execute the code block 130). Compute resources (e.g., the compute node 108) may be created or allocated at the time of execution if necessary.

[0136] 12) The compute node 108 may receive information regarding:12a) The code block 130 to run;12b) The canvas id and block id of the instance of the code block 130;12c) Temporary credentials for reading and writing to the runtime store 110;12d) Other blocks that may be connected to the code block 130; and12e) The semantics of the code block 130 in the context of the workflow.

[0137] 13) The compute node 108 may run the code block 130, analyze the instance of the code block 130, and store the results and metadata in the runtime store 110 (e.g., as a runtime state 128). In some embodiments, the system 100 may then shut down the compute node 108. In other examples, the compute node 108 is reused to execute similar code blocks 130.

[0138] 14) The executor service 106 may monitor the progress of each compute node 108.When a compute node 108 completes (e.g., an execution of a code block 130), the executor service 106 may communicate the completion with the canvas service 104, including informationsuch as:14a) The block id associated with code block 130; and 14b) Whether the task ran successfully.

[0139] 15) The canvas service 104 may perform one of the following operations:15a) Retry the task if it failed, when the canvas service 104 is configured to retry failed tasks;15b) Exit the workflow and mark the task run as being unsuccessful; or15c) Save the status of the task (e.g., as being successful), save the state (e.g., a complete or intermediate state) of the runtime (e.g., in a runtime store 110), and call the executor service 106 to run the next task(s).

[0140] 16) Steps 10-16 may be repeated until the process is complete. Repeating steps 10-16 may include looping over the workflow conditionally.

[0141] 17) Mark the task run as having been successful.

[0142] Throughout the process the user can request information on the status, intermediate results, resource utilization, data created, time taken, etc.

[0143] In some embodiments, the system 100 improves upon conventional software development systems in one or more of the following ways:

[0144] 1) Users can define compute resources at the granularity of the code block 130 rather than relying on the coarse-grained autoscaling logic used by many providers of cloud computing services.

[0145] 2) Automatic reconstruction of variable spaces (e.g., memory address spaces) from connected steps (e.g., code blocks 130) in the graph (e.g., graph 400).

[0146] 3) Saving intermediate states (e.g., runtime state 128) allows code blocks 130 to be re-run from a particular stage (e.g., point in time) and inspected for debugging purposes.

[0147] 4) The format / protocol for handling the metadata produced means that distributed computing can happen across compute nodes that are not directly connected in the network but can access the runtime store 110. Thus, the system 100 enables compute nodes 108 in workflows to work across cloud platforms (e.g., Spark on Azure, GPU in AWS).

[0148] 5) Multi-language support and interoperability within the same workflow.

[0149] 6) Support for running functional code and non-functional code. For example, normalPython code can be written without adhering to a specific framework. This capability is useful as the initial stage of development can be scaled without re-writing code.

[0150] 7) Support for full traceability and reproducibility and for replaying a workflow from the beginning of any instance of a code block 130.

[0151] 8) Support for sharing data across tasks (e.g., instances of code blocks 130) without relying on in-memory storage of the shared data.

[0152] FIG. 5 is a flow diagram of a process 500 for executing code blocks, in accordance with one or more embodiments. For explanatory purposes, the blocks of the process 500 are described herein as occurring sequentially or linearly. However, multiple blocks of the process 500 may occur in parallel. The blocks of the process 500 need not be performed in the order shown, and one or more blocks of the process 500 need not be performed or can be replaced by other operations.

[0153] At block 510, a first computing device may receive a first request to execute a workflow. The first request may be received from a second computing device. The workflow may include a set of code blocks. Each code block of the set of code blocks may be associated with a respective set of runtime configurations.

[0154] In some embodiments, the first computing device may provide to the second computing device a user interface. The user interface may be presented by the second computing device to a user (e.g., via a display of the second computing device). The first request may be received by the first computing device from the second computing device via the user interface.

[0155] At block 520, the first computing device may initialize a first runtime environment based on the set of runtime configurations associated with a first code block of the set of code blocks.

[0156] In some embodiments, the first request may include a modification to the set of runtime configurations associated with the first code block. The modification to the set of run time configurations may result in a different runtime environment.

[0157] At block 530, the first computing device may cause the first runtime environment to execute the first code block and / or to generate a first runtime state based on the execution of the first code block. In some embodiments, execution of the first code block may include performing a data analysis process (e.g., data collection, data cleaning, data transformation, dataaugmentation, data integration, feature engineering, etc.), and the first runtime state may include the results of the data analysis process.

[0158] In some embodiments, the first request may include a modification to the first code block. The modification to the first code block may result in a different runtime state.

[0159] At block 540, the first computing device may initialize a second runtime environment based on the first runtime state and / or based on the set of runtime configurations associated with a second code block of the set of code blocks.

[0160] In some embodiments, the second code block may execute after the first code block executes when the second code block has one or more data dependences on the first code block. For example, the input to the second code block may be an output of an execution of the first code block.

[0161] In some embodiments, the second runtime environment is identical to the first runtime environment when the set of runtime configurations associated with the second code block is identical to the set of runtime configurations associated with the first code block.

[0162] In some embodiments, subsequent requests to initialize runtime environments to execute the first and second code blocks may be queued to run after the first and second runtime environments when the second request is queued after the first request.

[0163] At block 550, the first computing device may cause the second runtime environment to execute the second code block and / or to generate a second runtime state based on the execution of the second code block. In some embodiments, the execution of the second code block may include performing a modeling process (e.g., model training, hyperparameter tuning, model evaluation, model validation, generating content or inferences using a trained model, etc.), and the second runtime states may include the results of the modeling process.

[0164] At block 560, the first computing device may store the first runtime state and / or the second runtime state. The first computing device may also or instead store the data associating the first runtime state and the second runtime state with the first code block and the second code block, respectively, such as pointers, links, etc.

[0165] In some embodiments, the first computing device may provide to the second computing device an indication (e.g., alert, notification, etc.) after the first code block and the second code block have executed. The indication may be an indication of success when the codeblocks have executed successfully. The indication may be an indication of failure when the code blocks have not executed successfully.

[0166] It should be understood that, although two code blocks, sets of runtime configurations, runtime states, and runtime environments were described, there may be more or fewer than two of any or all items.

[0167] FIG. 6 is a flow diagram of a process 600 for collaboratively executing one or more code blocks, in accordance with one or more embodiments. For explanatory purposes, the blocks of the process 600 are described herein as occurring sequentially or linearly. However, multiple blocks of the process 600 may occur in parallel. The blocks of the process 600 need not be performed in the order shown, and one or more blocks of the process 600 need not be performed or can be replaced by other operations.

[0168] At block 610, a first computing device may receive a first request to execute a workflow. The first request may be received from a second computing device. The workflow may include a set of code blocks. Each code block of the set of code blocks may be associated with a respective set of runtime configurations.

[0169] In some embodiments, the first computing device may provide to the second computing device a user interface. The user interface may be presented by the second computing device to a user (e.g., via a display of the second computing device). The first request may be received by the first computing device from the second computing device via the user interface.

[0170] At block 620, the first computing device may initialize a first runtime environment based on the set of runtime configurations associated with a code block of the set of code blocks.

[0171] In some embodiments, the first request may include a modification to the set of runtime configurations associated with the code block. The modification to the set of run time configurations may result in a different runtime environment.

[0172] At block 630, the first computing device may cause the first runtime environment to execute the code block and / or to generate a first runtime state based on the execution of the code block. In some embodiments, execution of the code block may include performing a data analysis process (e.g., data collection, data cleaning, data transformation, data augmentation, data integration, feature engineering, etc.), and the first runtime state may include the results of the data analysis process.

[0173] In some embodiments, the first request may include a modification to the code block. The modification to the code block may result in a different runtime state.

[0174] It should be understood that executing the workflow may include executing multiple code blocks.

[0175] At block 640, the first computing device may store the first generated runtime state and data associating the first generated runtime state with the code block, such as pointers, links, etc.

[0176] At block 650, a first computing device may receive a second request to execute the workflow. The second request may be received from the second computing device (similar to block 610) or a third computing device.

[0177] At block 660, the first computing device may initialize a second runtime environment based on the first runtime state and / or based on the set of runtime configurations associated with a second code block of the set of code blocks.

[0178] In some embodiments, the second runtime environment may be queued to run after the first runtime environments when the second request is queued after the first request.

[0179] It should be understood that executing the workflow may include executing multiple code blocks.

[0180] At block 670, the first computing device may cause the second runtime environment to execute the second code block and / or to generate a second runtime state based on the execution of the second code block. In some embodiments, execution of the second code block may include performing a modeling process (e.g., model training, hyperparameter tuning, model evaluation, model validation, generating content or inferences using a trained model, etc.), and the second runtime states may include the results of the modeling process.

[0181] At block 680, the first computing device may store the second generated runtime state and data associating the first generated runtime state with the code block, such as pointers, links, etc.

[0182] It should be understood that, although two code blocks, sets of runtime configurations, runtime states, and runtime environments were described, there may be more or fewer than two of any or all items.

[0183] FIG. 7 is a flow diagram of a process 700 for exporting a workflow, in accordance31with one or more embodiments. For explanatory purposes, the blocks of the process 700 are described herein as occurring sequentially or linearly. However, multiple blocks of the process 700 may occur in parallel. The blocks of the process 700 need not be performed in the order shown, and one or more blocks of the process 700 need not be performed or can be replaced by other operations.

[0184] At block 710, a first computing device may receive a first request to execute a workflow. The first request may be received from a second computing device. The workflow may include a set of code blocks. Each code block of the set of code blocks may be associated with a respective set of runtime configurations.

[0185] Blocks 720 and 730 may be performed for each code block of the set of code blocks.

[0186] At block 720, the computing device may initialize a respective runtime environment based on a respective set of runtime configurations associated with a respective code block of the set of code blocks.

[0187] In some embodiments, the first request may include a modification to the respective set of runtime configurations associated with the respective code block. The modification to the set of run time configurations may result in a different runtime environment.

[0188] At block 730, the first computing device may cause the respective runtime environment to execute the respective code block and / or to generate a respective runtime state based on the execution of the respective code block. In some embodiments, execution of the respective code block may include performing a data analysis process and / or modelling process, and the respective runtime state may include the results of the process(s).

[0189] In some embodiments, the first request may include a modification to the respective code block. The modification to the respective code block may result in a different runtime state.

[0190] At block 740, the first computing device may receive a second request to export the workflow. The second request may be received from the second computing device (similar to block 710) or a third computing device.

[0191] At block 750, the first computing device may generate a set of runtime environments based on the set of runtime configurations of each code block of the set of code blocks and / or based on the generated set of runtime states (from block 730).

[0192] In some embodiments, the second request may include a modification to a runtimeconfiguration of a set of runtime configurations, which may cause the generated set of runtime environments to be different from the generated set of runtime environments based on the unmodified runtime configuration.

[0193] At block 760, the first computing device may provide the generated set of runtime environments and / or the set of code blocks to the second computing device for subsequent execution of the set of code blocks (e.g., with the generated set of runtime environments). The generated set of runtime environments and / or the set of code blocks may be provided to the second computing device as an application container.

[0194] FIG. 8 is a flow diagram of a process 800 for executing at least one code block, in accordance with one or more embodiments. For explanatory purposes, the blocks of the process 800 are described herein as occurring sequentially or linearly. However, multiple blocks of the process 800 may occur in parallel. The blocks of the process 800 need not be performed in the order shown, and one or more blocks of the process 800 need not be performed or can be replaced by other operations.

[0195] At block 810, a first computing device may receive a request to execute a second code block of a workflow. The request may be received from a second computing device. The workflow may include a set of code blocks. The second code block may be associated with a runtime configuration and a first runtime state. The first runtime state may included data associated with (e.g., produced by) a first code block of the workflow.

[0196] In some embodiments, the first computing device may provide to the second computing device a user interface. The user interface may be presented by the second computing device to a user (e.g., via a display of the second computing device). The first request may be received by the first computing device from the second computing device via the user interface.

[0197] At block 820, the first computing device may initialize a runtime environment based on the runtime configuration and first runtime state associated with the second code block. In some embodiments, the first code block may include instructions of a data analysis process (e.g., instructions for data collection, data cleaning, data transformation, data augmentation, data integration, feature engineering, etc.), and the first runtime state may include the results of the data analysis process.

[0198] In some embodiments, the first request may include a modification to the runtimeconfiguration associated with the first code block. The modification to the runtime configuration may result in a different runtime environment.

[0199] At block 830, the first computing device may cause the runtime environment to execute the second code block and / or to generate a second runtime state based on the execution of the second code block. In some embodiments, execution of the second code block may include performing a modeling process (e.g., model training, hyperparameter tuning, model evaluation, model validation, generating content or inferences using a trained model, etc.), and the second runtime state may include the results of the modeling process.

[0200] In some embodiments, the request may include a modification to the second code block. The modification to the second code block may result in a different second runtime state.

[0201] At block 840, the first computing device may store the second runtime state. The first computing device may also or instead store data associating the second runtime state with the second code block, such as pointers, links, etc.

[0202] In some embodiments, the first computing device may provide to the second computing device an indication (e.g., alert, notification, etc.) after the second code block has executed. The indication may be an indication of success when the second code block has executed successfully. The indication may be an indication of failure when the second code block has not executed successfully.

[0203] It should be understood that, although two code blocks, sets of runtime configurations, runtime states, and runtime environments were described, there may be more or fewer than two of any or all items.

[0204] Examples of Computing Devices

[0205] FIG. 9 illustrates an exemplary implementation of a computing device 900 that may be used in a system (e.g., system 100) implementing the techniques described herein. It should be understood that FIG. 9 is intended neither to be a depiction of necessary components for a computing device to operate in accordance with the principles described herein nor a comprehensive depiction.

[0206] The computing device 900 may include one or more processors 902, network adapters 904, and computer-readable storage mediums 906. The computing device 900 may be, for example, a desktop or laptop personal computer, a personal digital assistant (PDA), a smartmobile phone, a server, a wireless access point or other networking element, or any other suitable computing device.

[0207] The network adapter 904 may be any suitable hardware and / or software to enable the computing device 900 to communicate wired and / or wirelessly with any other suitable computing device over any suitable computing network. The computing network may include wireless access points, switches, routers, gateways, and / or other networking equipment as well as any suitable wired and / or wireless communication medium or media for exchanging data between two or more computers, including the Internet.

[0208] The computer-readable storage medium 906 may be adapted to store data to be processed and / or instructions to be executed by one or more processors 902. Processor 902 enables the processing of data and execution of instructions. The data and instructions may be stored on the computer-readable storage medium 906. The data and instructions stored on computer-readable storage medium 906 may comprise computer-executable instructions implementing techniques that operate according to the principles described herein. In the example of FIG. 9, the computer-readable storage medium 906 may store computer-executable instructions implementing various facilities and storing various information as described above. The computer-readable storage medium 906 may store a client facility 912 configured to perform the operations of client 102, a canvas service facility 914 configured to perform the operations of canvas service 104, an executor service facility 916 configured to perform the operations of executor service 106, a user service facility 922 configured to perform the operations of user service 112, and / or a credential service facility 930 configured to perform the operations of credential service 120.

[0209] While not illustrated in FIG. 9, a computing device may additionally have one or more components and peripherals, including input and output devices. Output devices can be used, among other things, to present a user interface and input devices can be used, among other things, to interact with the user interface. Examples of output devices that can be used to provide a user interface include printers or display screens for visual presentation of output and speakers or other sound-generating devices for audible presentation of output. Examples of input devices that can be used for interacting with a user interface include keyboards and pointing devices(e.g., mice, touch pads, and digitizing tablets). As another example, a computing device 900 may receive input information through speech recognition or in other audible format.

[0210] Terminology

[0211] Terms such as “coupled” and “connected,” along with their derivatives, may be used in this disclosure, the claims, or both. “Coupled” may be used to indicate that two or more elements, which may or may not be in direct physical or electrical contact with each other, cooperate or interact with each other. “Connected” may be used to indicate the establishment of communication (e.g., a communicative relationship) between two or more elements that are coupled with each other. Further, in some embodiments set forth herein, an element, component, or module may be configured to perform a function if it is capable of performing or otherwise structurally arranged to perform that function.

[0212] A “network element” or “node” may include one or more pieces of service network equipment, including hardware and software that communicatively interconnects other equipment on a network (e.g., other network elements, end stations, etc.) and is adapted to host one or more applications or services with respect to one or more subscribers. As such, some network elements may be disposed in a wireless / wireline telecommunications network or a cable provider network, whereas other network elements may be disposed in a public packet- switched network infrastructure (e.g., a local area network, wide area network, the internet, etc.), private packet-switched network infrastructures (e.g., intranets and enterprise networks), as well as service provider network infrastructures (e.g., cloud platforms), any of which may span or involve a variety of access networks and core networks in a hierarchical arrangement. Accordingly, some network elements may comprise multiple services network elements that provide support for multiple network-based functions.

[0213] Software” and “code” may be understood to refer to the literary representation of the computer program. The literary representation of a computer program may include high-level languages (e.g., C, C++, Java, Kotlin, Python, R, Go, Scala, etc.) and / or low-level languages (e.g., assembly code, binary, or byte code representations). “Software” and “code” may also be understood to refer to any literary work that is syntactically compatible with a software compiler or interpreter for the execution of a computer program represented by the literary work.

[0214] Techniques operating according to the principles described herein may be implemented in any suitable manner. Included in the discussion above are flow charts showing the steps and acts of various processes that facilitate collaborative software development. The processing and decision blocks of the flow charts above represent steps and acts that may be included in algorithms that carry out these various processes. Algorithms derived from these processes may be implemented as software integrated with and directing the operation of one or more single- or multi-purpose processors. Processors 902 may be implemented as integrated circuits, such as a Digital Signal Processing (DSP) circuit, Field Programmable Gate Array (FPGA), an Application- Specific Integrated Circuit (ASIC), or any other suitable manner. It should be understood that the flow charts included herein do not depict the syntax or operation of any particular circuit or programming language. Rather, the flow charts illustrate the information one of ordinary skill in the art may use to fabricate circuits or to implement computer software algorithms to perform the techniques described herein. It should also be understood that, unless otherwise indicated herein, the particular sequence of steps and / or acts described in each flow chart is merely illustrative and not limiting.

[0215] Accordingly, in some embodiments, the techniques described herein may be embodied in computer-executable instructions implemented as software, including application software, system software, firmware, middleware, embedded code, or any other type of software. Such computer-executable instructions may be written using any suitable programming languages and / or programming or scripting tools, and also may be compiled as executable machine language code or intermediate code that is executed on a computer (e.g., a physical or logical computing device).

[0216] When techniques described herein are embodied as computer-executable instructions, the computer-executable instructions may be implemented in any suitable manner, including as a number of functional facilities, each providing one or more operations to complete execution of algorithms operating according to these techniques. A “functional facility,” however instantiated, may be a structural component of a computer system that, when integrated with and executed by one or more computers, may cause the one or more computers to perform a specific operational role. A functional facility may be a portion of or an entire software element. For example, a functional facility may be implemented as a function of a process, a discrete process, or as anyother suitable unit of processing. If techniques described herein are implemented as multiple functional facilities, each functional facility may be implemented in its own way and need not be implemented the same way. Additionally, the functional facilities may be executed in parallel and / or serially, as appropriate, and may pass information between one another using a shared memory on the computer(s) on which they are executing (e.g., by using a message-passing protocol, or in any other suitable way).

[0217] Functional facilities may include routines, programs, objects, components, data structures, etc., that perform particular tasks or implement particular abstract data types. The functionality of the functional facilities may be combined or distributed as desired in the systems in which they operate. In some implementations, one or more functional facilities carrying out techniques herein may together form a complete software package (e.g., computer program). The functional facilities may, in some embodiments, be adapted to interact with others (e.g., unrelated functional facilities and / or processes) to implement a software program application.

[0218] Some exemplary functional facilities are described herein for carrying out one or more tasks. It should be understood that the functional facilities and division of tasks described are merely illustrative of the type of functional facilities that may implement embodiments described herein and that embodiments are not limited to being implemented in any specific number, division, or type of functional facilities. In some embodiments, all functionalities described herein may be implemented in a single functional facility. It should also be appreciated that, in some embodiments, some of the functional facilities described herein may be implemented together with or separately from others (e.g., as a single unit or separate units), or some of these functional facilities may not be implemented.

[0219] Computer-executable instructions implementing the techniques described herein (e.g., when implemented as one or more functional facilities or in any other manner) may, in some embodiments, be encoded on one or more computer-readable mediums to provide functionality to the mediums. Computer-readable mediums may include magnetic media such as a hard disk drive, optical media such as a Compact Disk (CD) or a Digital Versatile Disk (DVD), a persistent or non-persistent solid-state memory (e.g., flash memory, RAM, etc.), or any other suitable storage media. A computer-readable medium may be implemented in any suitablemanner, including as a computer-readable storage medium 906 of FIG. 9 (e.g., as a portion of a computing device 900) or as a stand-alone, separate storage medium.

[0220] As used herein, “computer-readable medium” (also called “computer-readable storage medium”) may refer to one or more tangible storage mediums. Tangible storage mediums may be non-transitory and may have at least one physical, structural component. In a “computer-readable medium,” as used herein, at least one physical, structural component has at least one physical property that may be altered in some way during a process of creating the medium with embedded information, a process of recording information thereon, or any other process of encoding information onto the medium. For example, a magnetization state of a portion of a physical structure of a computer-readable medium may be altered during a recording process for storing data onto the medium.

[0221] Further, some techniques described above comprise acts of storing information (e.g., data and / or instructions) in certain ways for use by these techniques. In some implementations of these techniques — such as implementations where the techniques are implemented as computerexecutable instructions — the information may be encoded on a computer-readable storage medium. Where specific structures are described herein as advantageous formats in which to store this information, these structures may be used to impart a physical organization of the information when encoded on the storage medium. These advantageous structures may then provide functionality to the storage medium by affecting the operations of one or more processors interacting with the information (e.g., by increasing the efficiency of computer operations performed by the processor(s)).

[0222] In some embodiments in which the techniques described herein may be embodied as computer- executable instructions, the instructions may be executed on one or more suitable computing device(s) operating in any suitable computer system (e.g., the system 100), or one or more computing devices (e.g., one or more processors 902 of one or more computing devices 900) may be programmed to execute the computer-executable instructions. A computing device 900 or processor 902 may be programmed to execute instructions when the instructions are stored in a manner accessible to the computing device / processor, such as in a local memory (e.g., an on-chip cache or instruction register, a computer-readable storage medium accessible via a bus, a computer-readable storage medium accessible via one or more networks and accessible bythe device / processor, etc.). Functional facilities that comprise these computer-executable instructions may be integrated with and direct the operation of a single multi-purpose programmable digital computer apparatus, a coordinated system of two or more multi-purpose computer apparatuses sharing processing power, a single computer apparatus or coordinated system of computer apparatuses (e.g., co-located or geographically distributed), one or more FPGAs, or any other suitable system.

[0223] Embodiments have been described where the techniques are implemented in circuitry and / or computer-executable instructions. It should be understood that some embodiments may be in the form of a method, of which at least one example has been provided. The acts performed as part of the method may be ordered in any suitable way. Accordingly, embodiments may be constructed in which acts are performed in an order different than illustrated, which may include performing some acts simultaneously, even though shown as sequential acts in illustrative embodiments.

[0224] Various aspects of the embodiments described above may be used alone, in combination, or in a variety of arrangements not specifically discussed in the embodiments described in the foregoing and are therefore not limited in their application to the details and arrangement of components set forth in the foregoing description or illustrated in the drawings. For example, aspects described in one embodiment may be combined in any manner with aspects described in other embodiments.

[0225] Use of ordinal terms such as “first,” “second,” “third,” etc., in the claims to modify a claim element does not by itself connote any priority, precedence, or order of one claim element over another or the temporal order in which acts of a method are performed, but are used merely as labels to distinguish one claim element having a certain name from another element having a same name (but for use of the ordinal term) to distinguish the claim elements.

[0226] Also, the phraseology and terminology used herein are for the purpose of description and should not be regarded as limiting. The use of “including,” “comprising,” “having,” “containing,” “involving,” and variations thereof herein, is meant to encompass the items listed thereafter and equivalents thereof as well as additional items.

[0227] The phrase “and / or,” as used in the specification and in the claims, should be understood to mean “either or both” of the elements so conjoined, i.e., elements that areconjunctively present in some cases and disjunctively present in other cases. Multiple elements listed with “and / or” should be construed in the same fashion, i.e., “one or more” of the elements so conjoined. Other elements can optionally be present other than the elements specifically identified by the “and / or” clause, whether related or unrelated to those elements specifically identified. Thus, as a non-limiting example, a reference to “A and / or B”, when used in conjunction with open-ended language such as “comprising” can refer, in one embodiment, to A only (optionally including elements other than B); in another embodiment, to B only (optionally including elements other than A); in yet another embodiment, to both A and B (optionally including other elements); etc.

[0228] The word “exemplary” is used herein to mean serving as an example, instance, or illustration. Any embodiment, implementation, process, feature, etc., described herein as exemplary should therefore be understood to be an illustrative example and should not be understood to be a preferred or advantageous example unless otherwise indicated.

[0229] Having thus described several aspects of at least one embodiment, it is to be appreciated that various alterations, modifications, and improvements will readily occur to those skilled in the art. Such alterations, modifications, and improvements are intended to be part of this disclosure and are intended to be within the spirit and scope of the principles described herein. Accordingly, the foregoing description and drawings are by way of example only.

[0230] The present disclosure describes some examples of methods and systems for processing code blocks in a collaborative online software development environment. In some embodiments, a computer program is divided into a plurality of code blocks. The plurality of code blocks may be stored in a code block persistence store. In some embodiments, a runtime configuration is associated with the plurality of code blocks and stored in a runtime configuration persistence store. A runtime environment may be created using the runtime configuration, and the plurality of code blocks may be executed in the runtime environment. The execution output may include a plurality of runtime states. The plurality of runtime states may be stored in a runtime state persistence store and associated with the plurality of code blocks. In some embodiments, the computer program may be a machine learning model. In some embodiments, a runtime state includes inference outputs from a machine learning model.

[0231] In some embodiments, a subset of the plurality of code blocks is identified for execution. The subset of the plurality of runtime states associated with the subset of the plurality of code blocks may be identified and used to initialize a runtime environment. In some embodiments, an existing runtime environment based on the runtime configuration may be used. In some embodiments, a new runtime environment is created using the runtime configuration. The runtime environment may be initialized using the subset of the plurality of runtime states, and the subset of the plurality of code blocks may be executed in the runtime environment. The execution output may be another plurality of runtime states. The plurality of runtime states may be stored in a runtime state persistence store and may be associated with the subset of the plurality of code blocks.

[0232] In some embodiments, a subset of the plurality of code blocks is identified for execution. The subset of the plurality of runtime states associated with the subset of the plurality of code blocks may be identified and used to initialize a runtime environment. In some embodiments, an existing runtime environment based on the runtime configuration is used. In some embodiments, a new runtime environment is created using the runtime configuration. The runtime environment may be initialized using the subset of the plurality of runtime states, and the subset of the plurality of code blocks may be loaded into the runtime environment. In some embodiments, an archive file is made of the runtime environment.

[0233] In some embodiments, a subset of the plurality of code blocks is identified for execution. The subset of the plurality of runtime states associated with the subset of the plurality of code blocks may be identified and used to generate runtime state initializing source code. In some embodiments, an archive file is created containing the subset of the plurality of code blocks and the runtime state initializing source code. In some embodiments, an appropriate testing framework is selected and test wrapper methods for the subset of the plurality of code blocks are generated, and test initialization wrapper methods for the runtime state initializing source code are generated.

[0234] In some embodiments, input data to be provided to the plurality of code blocks is retrieved and used to initialize a runtime environment. In some embodiments, an existing runtime environment based on the runtime configuration is used. In some embodiments, a new runtime environment is created using the runtime configuration. The input data may be providedas input to the plurality of code blocks upon executing the plurality of code blocks. In some embodiments, the input data is training data for a machine learning model. In some embodiments, the input data is retrieved from a remote data source.

[0235] In some embodiments, multiple requests are simultaneously received attempting to either: modify the plurality of code blocks, modify the runtime configuration, or execute the plurality of code blocks. Upon receiving more than one request, the requests may be prioritized, and lower priority requests may be queued until after higher priority requests have been completed. In some embodiments, the prioritization is first-come-first-served. In some embodiments, the prioritization is based on user permissions.

[0236] A selection of some embodiments described herein may be represented by the following clauses:

[0237] Clause 1. A method of processing code blocks in a collaborative online software development system, the method comprising: dividing a computer program into a plurality of code blocks; storing the plurality of code blocks in a code block persistence store; storing a runtime configuration in a runtime configuration persistence store; associating the runtime configuration with the plurality of code blocks; creating a first runtime environment using the runtime configuration; executing the plurality of code blocks in the first runtime environment; storing a first plurality of runtime states in a runtime state persistence store; and associating the plurality of runtime states with the plurality of code blocks.

[0238] Clause 2. The method of clause 1, further comprising: selecting a subset of the plurality of code blocks; identifying the subset of the plurality of runtime states associated with the subset of the plurality of code blocks; creating a second runtime environment using the runtime configuration; initializing the second runtime environment using the subset of the plurality of runtime states; executing the subset of the plurality of code blocks in the second runtime environment; storing a second plurality of runtime states in a runtime persistence store; and associating the second plurality of runtime states with the subset of the plurality of code blocks.

[0239] Clause 3. The method of clause 2, wherein the second runtime environment reuses the first runtime environment.

[0240] Clause 4. The method of clause 1, further comprising: selecting a subset of the plurality of code blocks; identifying the subset of the plurality of runtime states associated with the subset of the plurality of code blocks; creating a second runtime environment using the runtime configuration; initializing the second runtime environment using the subset of the plurality of runtime states associated with the subset of the plurality of code blocks; loading the subset of the plurality of code blocks into the second runtime environment; and creating an archive file of the second runtime environment.

[0241] Clause 5. The method of clause 1, further comprising: selecting a subset of the plurality of code blocks; identifying the subset of the plurality of runtime states associated with the subset of the plurality of code blocks; and converting the subset of the plurality of runtime states associated with the subset of the plurality of code blocks into runtime state initializing source code.

[0242] Clause 6. The method of clause 5, further comprising: creating an archive file of the subset of the plurality of code blocks and the runtime state initializing source code.

[0243] Clause 7. The method of clause 5, further comprising: selecting an appropriate testing framework; generating test wrapper methods for the subset of the plurality of code blocks; and generating test initialization wrapper methods for the runtime state initializing source code.

[0244] Clause 8. The method of clause 1, wherein the computer program is a machine learning model.

[0245] Clause 9. The method of clause 8, wherein the first plurality of runtime states includes inference outputs from the machine learning model.

[0246] Clause 10. The method of clause 1, further comprising: retrieving input data to be provided to the plurality of code blocks upon execution; initializing the first runtime environment with the input data to be provided to the plurality of code blocks upon execution; and providing the input data to be provided to the plurality of code blocks upon execution to the plurality of code blocks when executing the plurality of code blocks in the first runtime environment.

[0247] Clause 11. The method of clause 10, wherein the input data to be provided to the plurality of code blocks upon execution is training data for a machine learning model.

[0248] Clause 12. The method of clause 10, wherein the input data to be provided to the plurality of code blocks upon execution is retrieved from a remote data source.

[0249] Clause 13. The method of clause 1, further comprising: receiving a first request to perform an action selected from the group consisting essentially of: modifying the plurality of code blocks, modifying the runtime configuration, and executing the plurality of code blocks; receiving a second request to perform an action selected from the group consisting essentially of modifying the plurality of code blocks, modifying the runtime configuration, and executing the plurality of code blocks; prioritizing the first request and the second request; executing the higher priority request; and queuing the lower priority request until after the higher priority request completes.

[0250] Clause 14. The method of clause 13, wherein the prioritization is first come first served.

[0251] Clause 15. The method of clause 13, wherein the prioritization is based on user permissions.

[0252] Clause 16. A system of one or more computerized devices operable as a collaborative online software development system, the system comprising: a memory storing computer program instructions; one or more processors for executing the computer program instructions; input / output (I / O) circuitry connecting the one or more computerized devices to external devices; and interconnect circuitry interconnecting the memory, the one or more processor(s), and the input / output (I / O) circuitry, wherein the computer program instructions are operative when executed by the one or more processor(s) to cause the one or more computerized devices to perform a method of processing code blocks, the method comprising: dividing a computer program into a plurality of code blocks; storing the code blocks in a code block persistence store; storing a runtime configuration in a runtime configuration persistence store; associating the runtime configuration with the plurality of code blocks; creating a first runtime environment using the runtime configuration; executing the plurality of code blocks in the first runtime environment; storing a first plurality of runtime states in a runtime state persistence store; and associating the plurality of runtime states with the plurality of code blocks.

[0253] Clause 17. A computer program product comprising a non-transitory computer readable medium having computer program instructions stored thereon, the computer programinstructions being executable by processing circuitry of one or more computerized devices to cause the one or more computerized devices to perform a method of processing code blocks, the method comprising: dividing a computer program into a plurality of code blocks; storing the code blocks in a code block persistence store; storing a runtime configuration in a runtime configuration persistence store; associating the runtime configuration with the plurality of code blocks; creating a first runtime environment using the runtime configuration; executing the plurality of code blocks in the first runtime environment; storing a first plurality of runtime states in a runtime state persistence store; and associating the plurality of runtime states with the plurality of code blocks.

Claims

CLAIMSWhat is claimed is:

1. A method comprising: receiving, by a first computing device and from a second computing device, a first request to execute a workflow, wherein the workflow includes a set of code blocks, and each code block of the set of code blocks is associated with a respective set of runtime configurations; initializing, by the first computing device, a first runtime environment based on the set of runtime configurations associated with a first code block of the set of code blocks; causing, by the first computing device, the first runtime environment to execute the first code block and to generate a first runtime state based on the execution of the first code block; initializing, by the first computing device, a second runtime environment based on the first runtime state and based on the set of runtime configurations associated with a second code block of the set of code blocks; causing, by the first computing device, the second runtime environment to execute the second code block and to generate a second runtime state based on the execution of the second code block; and storing, by the first computing device, the first runtime state, the second runtime state, and data associating the first runtime state and the second runtime state with the first code block and the second code block, respectively.

2. The method of claim 1, wherein: the first code block causes the first runtime environment to perform a data analysis process; the first runtime state includes a set of results of the data analysis process; the second code block causes the second runtime environment to perform a modeling process; and the second runtime states includes a set of results of the modeling process.

3. The method of claim 1, wherein the first computing device causes the second code block to execute after the first code block executes in response to the second code block having one or more data dependences on the first code block.

4. The method of claim 1 , wherein the second runtime environment is identical to the first runtime environment, in response to the set of runtime configurations associated with the second code block being identical to the set of runtime configurations associated with the first code block.

5. The method of claim 1, further comprising providing, by the first computing device to the second computing device, a user interface for presenting to the user via the second computing device, wherein the first request is received by the first computing device from the second computing device via the user interface.

6. The method of claim 1, further comprising, providing, by the first computing device and to the second computing device, an indication that the first code block and the second code block have successfully executed, in response to storing the first runtime state and the second runtime state.

7. The method of claim 1, further comprising: receiving, by the first computing device and from a third computing device, a second request to execute the workflow; initializing, by the first computing device, a third runtime environment based on the set of runtime configurations associated with the first code block of the set of code blocks; causing, by the first computing device, the third runtime environment to execute the first code block and to generate a third runtime state based on the execution of the first code block; initializing, by the first computing device, a fourth runtime environment based on the third runtime state and based on the set of runtime configurations associated with the second code block of the set of code blocks;causing, by the first computing device, the fourth runtime environment to execute the second code block and to generate a fourth runtime state based on the execution of the second code block; and storing the third runtime state, the fourth runtime state, and data associating the third runtime state and the fourth runtime state with the first code block and the second code block, respectively.

8. The method of claim 7, wherein the first runtime environment and the third runtime environment are different and the second runtime environment and the fourth runtime environment are different, in response to the first request being different from the second request.

9. The method of claim 7, wherein: the second request includes a modification to the set of runtime configurations associated with the first code block; and the first runtime environment is different from the third runtime environment based on the modification to the set of runtime configurations associated with the first code block.

10. The method of claim 7, wherein: the second request includes a modification to the first code block; and the third runtime state is different from the first runtime state based on the modification to the first code block.

11. The method of claim 7, wherein the third and fourth runtime environments are initialized after the first and second runtime environments, in response to the second request being queued after the first request.

12. A computing device comprising: a processor; and a memory storing instructions that, when executed by the processor, cause the computingdevice to perform operations comprising: receiving, by the computing device, a first request to execute a workflow, wherein the workflow includes a set of code blocks, and each code block of the set of code blocks is associated with a respective set of runtime configurations; initializing, by the computing device, a first runtime environment associated with a code block of the set of code blocks and based on the set of runtime configurations associated with the code block; causing, by the computing device, the first runtime environment to execute the code block and to generate a first runtime state based on the execution of the code block; storing, by the computing device, the first generated runtime state and data associating the first generated runtime state with the code block; receiving, by the computing device, a second request to execute the code block; initializing, by the computing device, a second runtime environment based on the first generated runtime state associated with the code block and based on the set of runtime configurations associated with the code block; causing, by the computing device, the second runtime environment to execute the code block and to generate a second runtime state based on the execution of the code block; and storing the second generated runtime state and data associating the second generated runtime state with the code block.

13. The computing device of claim 12, further comprising providing, by the computing device to another computing device, a user interface for presenting to the user via the other computing device, wherein the first request is received by the first computing device from the second computing device via the user interface.

14. The computing device of claim 12, wherein: the computing device includes a plurality of historical versions of the workflow; and the workflow is a historical version of the plurality of historical versions specified by thefirst request.

15. The computing device of claim 12, further comprising providing, by the computing device and to another computing device, an indication that the code block has successfully executed, in response to storing the second generated runtime state.

16. The computing device of claim 12, wherein: the second request includes a modification to the set of runtime configurations associated with the code block; and the first generated runtime environment is different from the second generated runtime environment based on the modification to the set of runtime configurations associated with the first code block.

17. The computing device of claim 12, wherein: the second request includes a modification to the code block; and the second generated runtime state is different from the first generated runtime state based on the modification to the code block.

18. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor of a computing device, causes the computing device to perform operations comprising: receiving, by the computing device, a first request to execute a workflow, wherein the workflow includes a set of code blocks, and each code block of the set of code blocks is associated with a respective set of runtime configurations; for each respective code block of the set of code blocks: initializing, by the computing device, a respective runtime environment based on the respective set of runtime configurations associated with the respective code block of the set of code blocks; and causing, by the computing device, the respective runtime environment to execute the respective code block and to generate a respective runtime state based onthe execution of the respective code block; receiving, by the computing device, a second request to export the workflow; generating, by the computing device, a set of runtime environments based on the set of runtime configurations of each code block of the set of code blocks and based on the generated set of runtime states; and providing, by the computing device, the generated set of runtime environments and the set of code blocks to another computing device for subsequent execution of the set of code blocks with the generated set of runtime environments.

19. The non-transitory computer-readable storage medium of claim 18, wherein the generated set of runtime environments and the set of code blocks are provided to the other computing device as an application container.

20. The non-transitory computer-readable storage medium of claim 18, the operations further comprising: receiving, by the computing device, a third request to export the workflow; generating, by the computing device, a second set of runtime environments based on the set of runtime configurations of each code block of the set of code blocks and based on the generated set of runtime states; and providing, by the computing device, the second generated set of runtime environments and the set of code blocks to another computing device for subsequent execution of the set of code blocks with the second generated set of runtime environments on the other computing device.

21. The non-transitory computer- readable storage medium of claim 20, wherein: the generated set of runtime environments is a first generated set of runtime environments; the third request includes a modification to a runtime configuration of a set of runtime configurations associated with a code block of the set of code blocks; and the second generated set of runtime environments is different from the first generated setof runtime environments based on the modification to the runtime configuration.

22. A method comprising: receiving, by a first computing device and from a second computing device, a first request to execute a second code block of a workflow, wherein the second code block is associated with a runtime configuration and a first runtime state, and wherein the first runtime state includes data produced by a first code block of the workflow; initializing, by the first computing device, a runtime environment based on the runtime configuration and the first runtime state associated with the second code block; causing, by the first computing device, the runtime environment to execute the second code block and to generate a second runtime state based on the execution of the second code block; and storing, by the first computing device, the second runtime state and data associating the second runtime state with the second code block.

23. The method of claim 22, wherein: the first code block includes instructions of a data analysis process; the first runtime state includes a set of results of the data analysis process; the second code block includes instructions of a modeling process; and the second runtime state includes a set of results of applying the modeling process to at least a portion of the set of results of the data analysis process.

Images