Code generation tools for cloud-native high-performance computing

Abstract

Methods, storage media, and systems for translating software expressions from user application programming interface (API) calls into software development kit (SDK) API calls are disclosed. Some examples may include: receiving a tagged expression indicating that the translation of the software expression from the user API call to the SDK's API call is to be performed, the SDK being associated with a cloud-native high-performance computing environment; processing an abstract syntax tree (API) associated with the software expression, the processing including replacing symbols in the API with appropriate variables; replacing return statements in the API with serialization instructions to write the result to local storage; and serializing the processed API, and providing the serialized API and one or more resource files to the cloud-native high-performance computing environment for execution.

Description

Background Technology

[0001] Large-scale data parallel optimization problems represent one of the most computationally challenging problems in scientific computing due to the high dimensionality of the unknown models and the massive amounts of data involved. Traditionally, these workloads are deployed to local high-performance computing (HPC) clusters, leveraging large-scale shared file systems, fast inter-node connections, and highly resilient resources. The default approach to running these applications in the cloud is to replicate the internal HPC cluster using on-demand Infrastructure as a Service (IaaS) resources, such as HPC virtual machines and parallel file systems. These on-premises, cloud-based cluster solutions are costly, unresilient, and add administrative overhead. While some implementation services make cluster management and deployment easier, users still bear the responsibility of selecting the right set of resources and managing the cluster throughout its lifecycle. This places a significant burden on users, who are often domain experts without extensive HPC knowledge.

[0002] Beyond management complexity, on-demand clustering also suffers from drawbacks related to resilience and cost. While cluster management tools support elasticity and manage resilience, most applications running on a cluster use Messaging Interfaces (MPI) and do not inherently support elasticity. Unless an application specifically implements resilience handling, it cannot effectively leverage the resilience offered by the cloud platform. Similar limitations exist in terms of cost. Data-parallel optimization algorithms typically synchronize at specific times during execution (e.g., broadcasting updates to all worker nodes), resulting in temporary idle computing resources. In principle, cloud services provide autoscaling capabilities, but applications running on a cluster typically do not support this capability themselves. Therefore, the service's autoscaling capabilities are primarily used to scale resources across jobs, rather than within a single job.

[0003] Alternative approaches to running HPC workloads in the energy space of dedicated (virtual) clusters include serverless Function as a Service (FaaS) and semi-serverless approaches, which combine FaaS and Platform as a Service (PaaS). The objective function of data-parallel optimization problems like seismic imaging is difficult to evaluate, thus providing an excellent opportunity to leverage batch processing tools. Since iterative optimization algorithms also involve concatenated components (i.e., collecting updates, updating weights / model parameters), batch processing can be combined with workflow management tools capable of representing and executing directed acyclic graphs (DAGs). This can be achieved through serverless services, which can effectively replace dedicated master nodes. Using a combination of batch processing and serverless computing offers users several advantages, such as inherent resilience (because batch jobs process tasks independently), the possibility of adding or removing nodes at runtime, leveraging resource collection, and near-unlimited scalability (because synchronization between worker and master nodes is not required). These advantages result in significant cost savings for users, as serverless methods maximize resource utilization, thereby reducing some costs by up to 90%. In addition, serverless workflow management removes the classic master node as a single point of failure and enables automatic resource allocation for workflow execution.

[0004] While running large-scale data-parallel applications in a serverless and semi-serverless manner offers many advantages, it requires fundamentally redesigning and reimplementing the application using multiple sets of potentially complex software development kits (SDKs). Furthermore, the implementation is heavily platform-dependent and cannot be ported to other clouds or on-premises clusters. The need to manually implement serverless and semi-serverless approaches places a significant burden on users and renders this approach unsuitable for research and development purposes.

[0005] Embodiments have been described with respect to these and other general considerations. Furthermore, although relatively specific problems have been discussed, it should be understood that the embodiments should not be limited to solving the specific problems identified in the background section. Summary of the Invention

[0006] This disclosure relates to a framework for providing distributed scaling that allows users to offload computation to the cloud in serverless and semi-serverless manner, based on batch processing and serverless computing. More specifically, such a framework can be implemented by extending packages for distributed computing in high-level languages ​​like Python or Julia to cloud services such as, but not limited to, AWS / Azure Batch and Azure (Durable) Functions, AWS Lambda Functions, or Google Cloud Functions. An additional layer of software abstraction is implemented on top of the existing RESTful SDK, allowing users to offload computation to the cloud using remote procedure calls with APIs similar to those of current (cluster-based) distributed packages. This enables users to run their code in a cloud-native manner using simple, high-level code statements with minimal modifications to existing code.

[0007] Based on the examples in this disclosure, users can efficiently run data-parallel applications using batch processing and serverless computing, enabling them to scale across multiple regions to large-scale problem sizes and reduce costs by up to 90%. Therefore, users can implement a single version of a program that can run on a local PC, a private cloud, or an internal cluster, and execute it in a cloud-native manner using various PaaS and FaaS products. This bridges the gap between the development and deployment phases, as the same program can run in different computing environments. In some implementations, users can interact with the cloud in a completely serverless manner, where users can offload any type of computing to the cloud without having to specify or manage resources, and, equally importantly, without having to make significant modifications to existing code.

[0008] According to various aspects of this disclosure, a method for converting a software expression from a user application programming interface (API) call into a software development kit (SDK) API call is described. The method may include receiving a tagged expression indicating that the conversion of the software expression from the user API call to the SDK API call is to be performed, the SDK being associated with a cloud-native high-performance computing environment; processing an abstract syntax tree (API) associated with the software expression, the processing including replacing symbols in the API with appropriate variables, replacing return statements in the API with serialization instructions to write the result to local storage, and serializing the processed API; and providing the serialized API and one or more resource files to the cloud-native high-performance computing environment for execution.

[0009] According to various aspects of this disclosure, a computer-readable storage medium comprising instructions, when executed by a processor, causes the processor to: translate a software expression from a user application programming interface (API) call into an API call of a software development kit (SDK). When executed by the processor, the instructions can cause the processor to: receive a marked expression indicating that the translation of the software expression from the user API call to the SDK API call is to be performed, the SDK being associated with a cloud-native high-performance computing environment; process the abstract syntax tree associated with the software expression by substituting symbols in the abstract syntax tree with appropriate variables and replacing return statements in the abstract syntax tree with serialization instructions to write the results to local storage; and serialize the processed abstract syntax tree; and provide the serialized abstract syntax tree and one or more resource files to the cloud-native high-performance computing environment for execution.

[0010] According to various aspects of this disclosure, a system is described. The system may include one or more hardware processors configured by machine-readable instructions to: receive a tagged expression indicating that a transformation of a software expression from a user API call to an SDK API call is to be performed, the SDK being associated with a cloud-native high-performance computing environment; process an abstract syntax tree associated with the software expression, the processing including replacing symbols in the abstract syntax tree with appropriate variables, replacing return statements in the abstract syntax tree with serialization instructions to write the result to local storage, and serializing the processed abstract syntax tree; and providing the serialized abstract syntax tree and one or more resource files to the cloud-native high-performance computing environment for execution.

[0011] This summary is provided to introduce a series of concepts in a simplified form, which are further described in the detailed description below. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. Attached Figure Description

[0012] The following figures illustrate a non-limiting and non-exhaustive example.

[0013] Figure 1 Details of the operating environment for a code generation tool for a cloud-native high-performance computing environment, based on examples of this disclosure, are described.

[0014] Figure 2 Details of a code generation tool based on examples provided in this disclosure are described.

[0015] Figure 3 Details of the first macro extension based on the example in this disclosure are described.

[0016] Figure 4Details of the second macro extension based on the example of this disclosure are described.

[0017] Figure 5 Details of the third macro extension based on the examples in this disclosure are described.

[0018] Figure 6 Details of a method for generating code for use in cloud-native high-performance computing environments, based on examples of this disclosure, are described.

[0019] Figure 7 Details of a method for generating a serialized abstract syntax table for use in a cloud-native high-performance computing environment, based on examples of this disclosure, are described.

[0020] Figure 8 Details of a method for generating an environment for use in a cloud-native high-performance computing environment, based on examples of this disclosure, are described.

[0021] Figure 9 A block diagram illustrating the physical components of a computing system that can implement various aspects of this disclosure is shown.

[0022] Figures 10A-10B Details of one or more computing systems according to examples of this disclosure are described.

[0023] Figure 11 An architecture for processing data received at a computing system, according to an example of this disclosure, is described. Detailed Implementation

[0024] In the following detailed description, reference is made to the accompanying drawings, which form a part of the description, and specific embodiments or examples are illustrated therein. These aspects may be combined, other aspects may be utilized, and structural changes may be made without departing from this disclosure. Embodiments may be implemented as methods, systems, or devices. Therefore, embodiments may take the form of hardware implementations, entirely software implementations, or implementations combining software and hardware aspects. Consequently, the following detailed description is not to be construed as limiting, and the scope of this disclosure is defined by the appended claims and their equivalents.

[0025] Traditional approaches to distributed computing for scientific applications are based on bilateral communication, primarily through the Message Passing Interface (MPI) standard. Many existing legacy programs and coding applications, such as those in the energy sector, are built on this paradigm and implemented using low-level languages ​​such as Fortran or C. While the MPI standard provides very fine-grained control over the communication pattern, it comes with high complexity, which often stifles innovation and makes code maintenance cumbersome. With the rise of higher-level programming languages ​​like Python or Julia, one-sided communication has become the de facto standard for parallelizing tasks and applications. Unlike bilateral communication, users only need to manage the main process and utilize remote worker programs via Remote Function / Procedure Calls (RPC).

[0026] In the cloud, distributed applications based on one-way communication can run on interconnected clusters of virtual machines (VMs). However, this programming model no longer applies if users want to leverage cloud-native services for (distributed) computing, such as batch processing or serverless functions (FaaS), because PaaS and FaaS products are almost entirely exposed via REST APIs. Users can interact with these services through SDKs provided by the cloud platform, and these SDKs are therefore platform-specific, even if the underlying services are typically not. Therefore, if users want to run distributed applications using PaaS and FaaS products, they need to manually prepare their applications and implement glue code for specific cloud SDKs. Glue code is simply used to "adapt" different parts of code that were originally incompatible. Glue code doesn't contribute any functionality to meet program requirements; instead, glue code typically appears in code that allows existing libraries or programs to interoperate, such as in language bindings or external function interfaces. For example, to run code as a batch job on the Microsoft Azure cloud, a user needs to create a pool of batch workers (each with the correct dependencies installed), prepare their application, create a batch job with the correct input and output bindings, and specify how the remote workers execute the code. A simple hello-world-style example from Azure Batch requires nearly 400 lines of Python code. Similar steps apply to running serverless code via Azure or AWS Lambda functions. Because the required glue code is highly application- and platform-dependent, it often needs to be rewritten multiple times.

[0027] Existing distributed computing solutions in the cloud can be mainly divided into three categories: (1) cluster-based cloud computing; (2) (multiple) functions and platform as a service; and (3) academic solutions.

[0028] In the examples, cluster-based cloud computing includes classic cluster managers (such as Azure CycleCloud, AWS ParallelCluster) or cluster managers (such as Kubernetes, Docker Swarm) used for container orchestration. In either case, the user is responsible for managing the cluster's lifecycle, which includes creating a set of instances, establishing virtual networks and connections between nodes / containers, and mounting parallel file systems. The user is also responsible (in their application) for distributing and scheduling parallel tasks within their program (e.g., as parallel loops). This incurs significant management overhead for the user. Another drawback of traditional clustering approaches to HPC in the cloud is the master worker scheme upon which cluster-based computation is based. This exposes the master worker to a single point of failure and makes long-running applications susceptible to resilience issues.

[0029] In the example, (multiple) functions and platform-as-a-services include serverless solutions (e.g., AWS Lambda, Azure Functions, Google Cloud Functions), managed compute and orchestration services (e.g., AWS StepFunctions, Azure Durable Functions, AWS / Azure Batch, Azure Logic Apps), and cloud-native storage solutions (e.g., AWS S3, Azure Blob Storage, Dynamo DB, Cosmos DB). These services offer numerous advantages for running HPC workloads in the cloud, such as automatic and semi-automatic resource allocation, high scalability, and reduced costs due to improved resource utilization. However, integrating these services into existing HPC applications requires significant software changes because cloud services are exposed via platform / vendor-specific APIs or require specific application representations (e.g., state machines in JSON format). However, software in HPC is based on bilateral (MPI) or one-sided communication statements (i.e., task / future / broadcast / decrease), which are fundamentally different from the REST APIs of cloud services. Currently, there are no commercial or academic solutions to bridge this gap.

[0030] Existing academic solutions tend to be geared towards serverless computing as a service (FaaS) for general-purpose computing. Several frameworks enable users to execute workloads over serverless environments, such as AWS Lambda, including PyWren, gg, mu, and Kappa. PyWren, gg, and mu are three academic serverless frameworks that support running applications through a large number of serverless functions. However, these projects focus solely on FaaS and do not allow execution over other services (e.g., batch processing via AWS / Azure Batch). These frameworks require users to manually partition their applications into components suitable for the serverless function framework, i.e., components that can be executed within given time and memory constraints. Furthermore, none of these existing frameworks provide concurrency APIs that are very similar to existing APIs for cluster-based distributed programming.

[0031] The Kappa framework is a serverless orchestration framework that does provide a concurrent API based on task-based distributed programming. However, Kappa only executes workloads through serverless functions and does not provide users with access to other PaaS products. This severely limits the possibility of applying Kappa to real-world HPC applications due to the hardware limitations of serverless computing (limited memory / execution time). Kappa addresses the limited execution time through automatic checkpointing, but this is not a viable solution for long-running HPC applications such as seismic imaging, which typically requires frequent moving of gigabytes or terabytes of data into and out of storage. Technically, Kappa achieves this through checkpointing and uses the current state of the capture procedure to continue.

[0032] Therefore, unlike existing cluster-based cloud computing technologies, the examples provided in this paper allow users to leverage FaaS / PaaS products that shift responsibilities from the user to the cloud platform, such as distributing and scheduling parallel tasks within the program (e.g., as parallel loops). Furthermore, unlike Kappa, the examples provided in this paper offer greater flexibility to interface with a wider range of services (e.g., for batch computing) and allow the creation of IO bindings or resource files required for specific cloud services.

[0033] This disclosure illustrates examples of distributed computing leveraging cloud-native PaaS and FaaS products based on additional abstraction layers built on top of existing RESTful cloud SDKs. APIs similar to existing APIs used for distributed computing based on one-sided communication and remote function calls can be utilized on the user side. On the backend, remote function calls (RFCs) are translated into API calls for various cloud services. For users, the additional level of abstraction makes cloud-native services more readily adaptable to distributed computing and eliminates the need for any platform-specific glue code based on multiple cloud SDKs. Therefore, these abstractions improve user productivity and open PaaS and FaaS products to researchers and developers who wish to focus on applications rather than cloud architecture and how to deploy code. Furthermore, by gaining easy access to services that would otherwise require significant glue code, users can leverage services that enable them to significantly reduce costs, improve resilience, and increase scalability compared to cluster-based approaches.

[0034] This disclosure demonstrates a software design based on a mullet architecture, where a one-sided communication statement and RFC API are provided on the user side, while the backend is implemented via a RESTful cloud SDK. Therefore, users can write code similar to that of traditional distributed applications, but remote function calls are executed through services such as batch processing computations or cloud functions. Since these services do not provide a direct network connection between the user / host and the worker / function, it is not possible to utilize traditional RPC APIs. Because cloud services almost always expose themselves via HTTP-based endpoint REST APIs, an abstraction layer is implemented that translates remote function calls and high-level user-side code statements into REST API calls using the existing cloud SDK.

[0035] According to the examples in this disclosure, the conversion from remote function calls to cloud API calls (including creating correct input-output bindings) will be fully automated. Therefore, traditional function definitions and function calls need to be converted into code executable via the cloud SDK. For example, if a user defines a function with a return statement and then executes that function remotely, the expression can be analyzed, and the return statement can be replaced with a serialization step that writes the results to a local (remote) disk. This implements automatic file upload to a cloud object storage device resource binding, making the data available to the user through another function call, without requiring direct user interaction with the storage device client. Automatic code generation includes several steps, such as capturing input-output parameters, creating IO bindings, and breaking down the expression into individual function calls or batch tasks. The generated code and IO bindings are then executed via the cloud service using the appropriate SDK.

[0036] In the example, the proposed framework can be implemented using the Julia language, a programming language designed from the ground up for numerical science and distributed computing. Similar to Python, Julia allows for high-level programming, enabling rapid prototyping and development. However, unlike Python, Julia naturally supports metaprogramming, where expressions are represented through Julia data structures and can be analyzed and manipulated within the language itself. Furthermore, unlike Python, Julia offers optional typing and is based on Just-In-Time (JIT) compilation using, for example, LLVM, making the language nearly as fast as C code. Julia also interoperates well with other programming languages, allowing direct interfaces with Python, Fortran, or C without requiring any glue code.

[0037] Julia also provides a built-in package for distributed computing, which is part of the language itself. Therefore, one-sided remote function calls to cloud-based services can be implemented by extending Julia's distributed functions, providing a set of high-level instructions for executing functions, such as, but not limited to, AWS / Azure Batch jobs, Azure / Lambda / Google Cloud Functions, and Azure Durable Functions calls. These functions enable users to define serverless workflows and perform batch computations via simple Julia macros. The Julia language provides such macros to map computations to interconnected pools of Julia workers, and in the examples described in this article, these are extended to other cloud-based services, such as those provided in AWS, GCP, or Azure environments. User-side remote function calls are translated into SDK calls to the corresponding cloud services by remotely executing the code through parsing Julia code (i.e., expressions) and creating the necessary I / O bindings. While the examples provided in this article are generally for the Julia programming language, other programming languages ​​can also be considered.

[0038] Figure 1Details of an operating environment 100 for implementing a code generation tool for a cloud-native high-performance computing environment 124, according to an example of this disclosure, are depicted. In the example, the operating environment 100 includes a computing device 104 at which a user can generate or otherwise create application instructions 110 defining the workload to be performed by the cloud-native high-performance computing environment 124. For example, while the computing device 104 is depicted as a desktop computer, it can be any of a portable or non-portable computing device. Examples of the computing device 104 include, but are not limited to, virtual machines, laptops, desktops, or servers. The cloud-native high-performance computing environment 124 describes an environment for large-scale workloads requiring a large number of cores, typically hundreds or thousands. Scenarios that can leverage the cloud-native high-performance computing environment 124 include, but are not limited to, image rendering, fluid dynamics, financial risk modeling, oil exploration, drug design, and engineering stress analysis. In the example, the work to be processed can be broken down into discrete tasks that can run simultaneously across multiple cores. Generally, each task is finite, accepting input, performing some processing, and then producing output. For some applications, tasks are independent and can run in parallel. In other cases, tasks are tightly coupled, meaning they must interact or exchange intermediate results.

[0039] Development environment 106 can be executed by computing device 104; development environment typically includes a combination of a text editor and a runtime implementation. The text editor allows user 102 to write code or application instructions 110 for a specific application or workload. In the example, development environment 106 includes a user application programming interface (API) and a code generator 116. The user API and code generator 116 translate remote function calls into cloud API calls specific to the cloud-native high-performance computing environment 124. For example, application instructions 110 may include one or more expressions 112a-112c. One or more expressions 112a-112c may be associated with previously generated programs or workloads deployed to an on-premises HPC cluster.

[0040] For example, to configure application instructions 110 for use with the cloud-native high-performance computing environment 124, one or more tags (e.g., @macro_) can be used to identify one or more expressions 112a-112c that are to be transformed for use with a software development kit (SDK) to access the REST and RESTful endpoints 125 associated with the cloud-native high-performance computing environment 124. In some examples, the user 102 manually adds tags to those expressions 112a-112c that need to be transformed. In other examples, tags can be automatically added to the expressions 112a-112c that need to be transformed. For example, tags can be automatically associated with one or more expressions 112a-112c that match the expressions or keywords used for transformation. Therefore, based on the tags identifying the expressions used for transformation, the traditional function definitions and function calls (e.g., 114A-114C) associated with that expression are transformed into code that can be executed via the cloud SDK to access the REST and RESTful endpoints 125 associated with the cloud-native high-performance computing environment 124. For example, if a user-defined expression includes a function with a return statement, and such a function will be executed at a cloud-native high-performance computing environment 124, the expression is analyzed, and the return statement is replaced with a serialization step that writes the result to a file in a local or remote storage device region. A resource binding is then automatically created, which uploads the file to a storage device region (e.g., a cloud object storage device) available to the user using another function call, without requiring the user to interact directly with the storage device client.

[0041] The automatic transformation involves several steps, such as capturing input-output parameters, creating the IO bindings required for the cloud service, and breaking down the expression into individual function calls or batch tasks. As an example, an abstract syntax tree (AST) is collected for each expression to be executed at the cloud-native high-performance computing environment 124. An AST is a tree representation of the abstract syntactic structure of source code written in a programming language. Each node in the tree represents a construct that appears in the source code. Each collected AST is analyzed to capture runtime parameters, and variable symbols are replaced with actual variables. Each AST is then serialized and provided to the cloud-native high-performance computing environment 124 as a resource file via network 140. For expressions with return statements, the return statements are replaced with output filenames (e.g., randomly named output files). The output filenames are then added as output resources and provided to the cloud-native high-performance computing environment 124 via network 140.

[0042] An example abstract syntax tree (while b ≠ 0, a := ab, return a) is provided as abstract syntax tree 160. Abstract syntax tree 160 may include statement sequence 162 and return the value 166 (e.g., a) indicated by return statement 164. A while branch 168 indicates that the while loop will continue execution based on comparison statement 170 (e.g., ≠) between comparison variable 174 (e.g., b) and constant 176 (e.g., 0). The assignment at 172 will assign the output of the operation (e.g., "-") between variable 182 (e.g., a) and variable 184 (e.g., b) to variable 178 (e.g., a). In the example, the symbols representing variables (e.g., a and b) will be replaced with the actual variables. The return statement (e.g., 164) will be replaced with the serialization of the argument (e.g., a) and the operation of writing the serialized file to a local disk storage device or cloud object storage device using a randomly generated filename. Therefore, the processed abstract syntax tree can then be serialized and provided as one or more resource files 118. One or more resource files 118 may include transformed function calls and / or output filenames.

[0043] like Figure 1As shown, resources including (multiple) input files and (multiple) applications for processing (multiple) input files can be uploaded to the cloud-native high-performance computing environment 124. Input files can be any data processed by application instructions, such as financial modeling data, video files to be transcoded, etc. Application files can include scripts or applications, such as one or more resource files 118 for processing data. Resources 120 can be uploaded to storage device area 138. A pool 134 of compute nodes 136A-136B can be created and associated with a cloud service account 126. The cloud service account 126 is associated with the cloud-native high-performance computing environment 124 and determines which compute resources are available and in what type of configuration these resources exist. Compute nodes 136 can be virtual machines that process part of the workload of the application. For example, compute nodes 136 can process or otherwise execute one or more tasks associated with job 130. Job 130 can be defined as running a workload on pool 134, where pool 134 is a collection of nodes on which the application will run. Job 130 is a collection of tasks 132A-132C. Job 130 manages how its tasks 132A-132C perform computations on compute nodes 136A-136C within pool 134. Job 130 specifies the pool 134 in which the job will run. A new pool 134 can be created for each job 130, or a single pool 134 can be used for many jobs 130. A task, such as task 132a, is a computational unit associated with a job, such as job 130. Each task 132A-132C runs on a corresponding node 136A-136C. Tasks 132 are assigned to nodes 136 for execution or are queued until node 136 becomes idle. Tasks 132 run one or more programs or scripts on compute nodes 136 to perform the work to be done. Tasks 132A-132C can be defined by resource 120 and associated with job 130.

[0044] Figure 2 A framework for distributed computing based on cloud PaaS and FaaS components, according to examples of this disclosure, is described. More specifically, code generator 206 bridges a platform-independent user API 204 and a cloud SDK to provide access to cloud service 210 to existing workloads and applications originally designed for HPC clusters. User API 204 may be platform-independent and provides remote function calls and advanced one-way communication statements. Code generator 206 may be platform-independent and translates remote function calls and advanced code statements at the user side (e.g., user API 204) into REST API calls via an existing platform-dependent cloud SDK 208. User API 204, code generator 206, and cloud SDK 208 can be used in applications such as computing device 104 (…). Figure 1Cloud services 210 can be implemented in user computing devices and / or virtual machines. Cloud services 210 can be implemented via REST and RESTful endpoints 125. Figure 1 (To visit)

[0045] Figure 3 This document describes a first example of a macro implemented at User API 304 according to the examples in this disclosure. The macro `@bcast` can be added to existing expressions to make them available as resources in cloud-native high-performance computing environments. It can be provided with the User API and code generator 116 (…). Figure 1 Code generator 306, with similar or identical functionality, can transform an expression from user API 304 to create an object storage container, upload the expression to a cloud object storage device, and create batch resource files from the object using cloud API calls 308 exposed via the cloud SDK. More specifically, code generator 306 can expand the expression into macros to serialize the expression, generate the required resource bindings, and return a future or reference to an object storage device such as a blob to the user. Therefore, implementing the @bcast expr macro via user API 304 allows the implementation of the existing expression expr in a cloud-native high-performance computing environment.

[0046] Figure 4 A second example of a macro implemented at User API 404, according to the examples in this disclosure, is depicted. The macro `@batchdef` can be added to an existing expression to prepare it for execution on the local machine and / or on subsequent batch workers and / or serverless functions. It can be provided with the User API and code generator 116 ( Figure 1 Code generator 406, with similar or identical functionality, appends an expression (its abstract syntax tree) to a global state variable. When `@batchexec` is called, code generator 406 can serialize the expression and then upload the serialized expression to a cloud storage device, such as Azure blob storage. Therefore, neither user API 404 nor code generator 406 calls cloud API 408 before calling `@batchexec`.

[0047] Figure 5 A third example of a macro implemented at User API 504, based on the examples in this disclosure, is depicted. The macro `@batchexec` can be added to an existing expression to execute it in the cloud as a single-task or multi-task batch job. In the example, it can be provided with the User API and code generator 116 ( Figure 1Code generator 506, with similar or identical functionality, transforms expressions from user API 504 to upload a serialized abstract syntax tree to a storage device, create batch input / output resources or other resources required by corresponding cloud services, and create jobs and tasks using cloud API calls 508 exposed via the cloud SDK and submit them to a cloud-native high-performance computing environment. More specifically, code generator 506 expands the expression `expr` by collecting and analyzing all symbols in the abstract syntax tree. In the example, those expressions, including parallel mappings (pmaps), can be split into separate tasks, creating a single expression for each task to be executed in parallel. Code generator 506 replaces symbolic variables in the abstract syntax tree with actual variables, replaces return statements in the abstract syntax tree with serialization, serializes the abstract syntax tree, and uploads the serialized syntax tree to a storage device, such as an Azure blob storage device. Furthermore, code generator 506 creates input / output bindings, including but not limited to input resource files and output resource files as previously described. In the example, `@batchexec expr` creates jobs, such as batch jobs and tasks, and may return a reference to future output stored in object storage. When executed in a cloud-native high-performance computing environment, worker programs or serverless functions can deserialize the abstract syntax tree and compile it into an executable that is then executed on local hardware. Cloud API call 508 provides an interface to upload the serialized abstract syntax tree to storage, create input / output resources (depending on the cloud service being called), and then create and submit jobs / tasks for execution. Therefore, implementing the `@batchexec expr` macro via user API 504 allows existing `expr` expressions to be implemented in cloud-native high-performance computing environments.

[0048] User API 304 ( Figure 3 ), 404 Figure 4 ) and 504 Figure 5 ) is the same user API that provides multiple macro expansion functions. Similarly, code generator 306 ( Figure 3 ), 406 Figure 4 ) and 506 Figure 5 This is the same code generator that translates expressions from user APIs into cloud API calls and / or cloud SDKs. Cloud API call 308 ( Figure 3 ), 408 Figure 4 ) and 508 Figure 5 It uses the same cloud API / SDK and relies on a cloud-native high-performance computing environment.

[0049] Figure 6 Details of a method 600 for receiving, generating, and then executing code in a cloud-native high-performance computing environment, according to an example of this disclosure, are described. Figure 6 The diagram illustrates the general sequence of steps in method 600. Typically, method 600 begins at 602 and ends at 610. Method 600 may include more or fewer steps, or may differ from the steps described above. Figure 6 The steps shown are arranged in order. Method 600 can be executed as a set of computer-executable instructions that are executed by a computer system and encoded or stored on a computer-readable medium. In the example, aspects of method 600 are executed by one or more processing devices such as a computer or server. Furthermore, method 600 can be executed by gates or circuits associated with a processor, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), system-on-a-chip (SoC), neural processing unit, or other hardware device. References will be incorporated herein by reference. Figures 1-5 Method 600 is explained by describing the system, components, modules, software, data structures, user interface, etc.

[0050] The method begins at 602, where the process can proceed to 604. At 604, marker codes can be received and / or analyzed, including, but not limited to, application instruction 110. Figure 1 The tag application expressions, including tag expressions 114A-114C ( Figure 1 For example, development environment 106 ( Figure 1 ) can be generated by computing device 104 ( Figure 1 ) executes, and includes code or application instructions for a specific application or workload 110 ( Figure 1 In the example, development environment 106 includes a user application programming interface (API) and a code generator 116. Expressions included in the code can be analyzed, and those expressions containing matching macro tags can be identified. At 606, each expression with macro tags can be expanded. In the example, macro expansion can be performed by code generators 306 / 406 / 506 and / or the user API and code generator 116. The user API and code generator 116 translate remote function calls into one or more cloud API calls specific to the cloud-native high-performance computing environment 124.

[0051] Furthermore, macro extension 606 may include configuring one or more parameters to manage the requirements for performing computational operations. For example, this may include creating batch jobs, creating containers in a cloud object storage device, and creating dedicated input and output bindings that may be required for individual cloud services (e.g., batch services). Method 600 may proceed to 608, where the batch job is executed in a cloud-native high-performance computing environment. Method 600 may return data or provide access to data through one or more output resources in the form of remote references. Method 600 may end at 610.

[0052] Figure 7 A method 700 for converting a remote function call into an API call available via a cloud SDK, according to an example of this disclosure, is described. Figure 7 The general sequence of steps in method 700 is shown. Typically, method 700 begins at 702 and ends at 712. Method 700 may include more or fewer steps, or may differ from the steps shown. Figure 7 The steps shown are arranged in order. Method 700 can be executed as a set of computer-executable instructions that are executed by a computer system and encoded or stored on a computer-readable medium. In the example, aspects of method 700 are executed by one or more processing devices such as a computer or server. Furthermore, method 700 can be executed by gates or circuits associated with a processor, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), system-on-a-chip (SoC), neural processing unit, or other hardware device. References will be incorporated herein by reference. Figure 1-6 Method 700 is explained by describing the system, components, modules, software, data structures, user interface, etc.

[0053] The method begins at 702, where the process can proceed to 704. At 704, an abstract syntax tree (API) is collected for each expression to be executed in the cloud-native high-performance computing environment. Expressions can represent function definitions / calls, variable assignments, module loading statements, etc. Each collected API is analyzed to capture runtime parameters (in the case of function calls), and at 706, the symbols for variables are replaced with actual variables. Each API is then serialized at 708 and provided as a resource file to the cloud-native high-performance computing environment at 719. For expressions with return statements (i.e., function calls), at 706, the return statement is replaced with a serialized return argument and a WRITE statement of the serialized expression to the local storage device of the remote cloud worker, using a randomly generated file / object name. Depending on the cloud service used to perform the computation, output bindings may need to be created for the generated objects, so that the locally stored objects are moved to a durable cloud storage service after the computation in 710 is complete, from where the user can access the object at a later point in time. Method 700 ends at 712.

[0054] Figure 8 The present disclosure describes in detail a method 800 for creating / configuring one or more parameters to manage the requirements for performing computational operations, based on examples of this disclosure. Figure 8 The diagram illustrates the general sequence of steps in method 800. Typically, method 800 begins at 802 and ends at 810. Method 800 may include more or fewer steps, or may differ from the steps outlined herein. Figure 8The steps shown are arranged in order. Method 800 can be executed as a set of computer-executable instructions that are executed by a computer system and encoded or stored on a computer-readable medium. In the example, aspects of method 800 are executed by one or more processing devices such as a computer or server.

[0055] Furthermore, method 700 can be executed by gates or circuits associated with a processor, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), system-on-a-chip (SoC), neural processing unit, or other hardware device. References will be made below. Figure 1-7 Method 800 is explained by describing the system, components, modules, software, data structures, user interface, etc.

[0056] The method begins at 802, where the process can proceed to 804. At 804, a cloud computing service is launched, which may include (but is not limited to) cloud batch jobs or serverless function execution. In the case of cloud batch jobs, the launched job is a set of tasks, where the cloud service manages how this set of parallel tasks is executed in a pool of compute nodes. The method can proceed to 806, where a container for the application is created. Using containers provides a way to run tasks without having to manage the environment and dependencies for running the application. Containers deploy applications as lightweight, portable, self-contained units that can run in several different environments. Container-based tasks can also leverage the characteristics of non-container tasks, including application packages and the management of resource files and output files. The method can proceed to 808, where a dedicated resource storage container is created. Resource files place data on virtual machines in a cloud-native high-performance computing environment, but the data type and usage are flexible. In the example, several options can be used to generate resource files, which may depend on where the data is to be stored. Such options may include a storage container URL, a storage container name, and a network endpoint. A storage container URL generates resource files from any storage container. The storage device container name is generated from the name of the container in the linked storage device account to create a resource file. The network endpoint is generated from any valid HTTP URL to create a resource file.

[0057] The process of creating resource files varies depending on the location of the original data storage. Dedicated resource containers may include, but are not limited to, input and output resource files. Method 800 may end at 810.

[0058] Figures 9-11 The associated description provides a discussion of various operating environments in which aspects of this disclosure may be implemented. However, relative to... Figures 1 to 2 The devices and systems shown and discussed, 9-11, are for illustrative purposes and are not limited to a wide range of computing system configurations that can be used to implement the aspects of this disclosure described herein.

[0059] Figure 9 This is a block diagram illustrating the physical components (e.g., hardware) of a computing system 900, in which aspects of this disclosure may be implemented. The computing system components described below are applicable to the computing and / or processing devices described above. In a basic configuration, the computing system 900 may include at least one processing unit 902 and a system memory 904. Depending on the configuration and type of the computing system, the system memory 904 may include, but is not limited to, volatile memory (e.g., random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM)), flash memory, or any combination of such memories.

[0060] System memory 904 may include an operating system 905 and one or more program modules 906 for running software application 920, such as one or more components supported by the system described herein. As an example, system memory 904 may include a user API 921, a code generator 922, and a cloud SDK 923. User API 921 can be used with user API 204 (…). Figure 2 ), 304 Figure 3 ), 404 Figure 4 ) and 504 Figure 5 The code generator 922 is the same as or similar to the code generator 206. Figure 2 ), 306 Figure 3 ), 406 Figure 4 ) and 506 Figure 5 The same as or similar to Cloud SDK 923. Cloud SDK 923 can be used with Cloud SDK 208. Figure 2 ), 308 Figure 3 ), 408 Figure 4 ) and 508 Figure 5 (Similar or identical. In the example, User API & Code Generator 116) Figure 1 This may include a user API 921 and a code generator 922. For example, an operating system 905 may be adapted to control the operation of a computing system 900.

[0061] Furthermore, embodiments of this disclosure can be implemented with graphics libraries, other operating systems, or any other application, and are not limited to any particular application or system. Figure 9 In the diagram, the components within the dashed line 908 illustrate this basic configuration. The computing system 900 may have additional features or functions. For example, the computing system 900 may also include additional data storage devices (removable and / or non-removable), such as disks, optical discs, or magnetic tapes. This additional storage... Figure 9 The image shows a removable storage device 909 and a non-removable storage device 910.

[0062] As described above, numerous program modules and data files can be stored in system memory 904. When executed on processing unit 902, program module 906 (e.g., software application 920) can perform processes including, but not limited to, the aspects described herein. Other program modules that can be used according to aspects of this disclosure may include email and contact applications, word processing applications, spreadsheet applications, database applications, PowerPoint presentation applications, drawing or computer-aided programs, etc.

[0063] Furthermore, embodiments of this disclosure can be implemented on discrete electronic components, packaged or integrated electronic chips containing logic gates, circuits utilizing microprocessors, or single chips containing electronic components or microprocessors. For example, embodiments of this disclosure can be implemented via a system-on-a-chip (SoC), wherein... Figure 9 Each or more components shown can be integrated onto a single integrated circuit. Such a SoC device may include one or more processing units, graphics units, communication units, system virtualization units, and various application functions, all integrated (or “burned in”) as a single integrated circuit onto a chip substrate. When operating via the SoC, the functions described herein can be operated via application-specific logic integrated with other components of the computing system 900 on a single integrated circuit (chip), relative to the client’s ability to switch protocols. Embodiments of this disclosure can also be implemented using other techniques capable of performing logical operations, such as AND, OR, and NOT, including but not limited to mechanical, optical, fluid, and quantum technologies. Furthermore, embodiments of this disclosure can be implemented within a general-purpose computer or in any other circuit or system.

[0064] The computing system 900 may also have one or more input devices 912, such as a keyboard, mouse, pen, voice or speech input device, touch or swipe input device, etc. It may also include output devices (multiple) 914, such as a display, speaker, printer, etc. The above devices are examples and other devices may be used. The computing system 900 may include one or more communication connections 916 that allow communication with other computing systems 950. Examples of suitable communication connections 916 include, but are not limited to, radio frequency (RF) transmitters, receivers and / or transceiver circuitry; universal serial bus (USB), parallel and / or serial ports.

[0065] As used herein, the term computer-readable medium can include computer storage media. Computer storage media can include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, or program modules. System memory 904, removable storage device 909, and non-removable storage device 910 are examples of computer storage media (e.g., memory storage devices). Computer storage media can include RAM, ROM, electrically erasable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile disk (DVD) or other optical storage devices, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other article of manufacture that can be used to store information and is accessible by computing system 900. Any such computer storage medium may be part of computing system 900. Computer storage media does not include carrier waves or other propagated or modulated data signals.

[0066] Communication media can be implemented by computer-readable instructions, data structures, program modules, or other data (such as carrier waves or other transmission mechanisms) in modulated data signals, and include any information transmission medium. The term "modulated data signal" can describe a signal having one or more characteristics set or altered in a manner that encodes information in the signal. By way of example and not limitation, communication media can include wired media, such as wired networks or direct wired connections, and wireless media, such as acoustic, radio frequency (RF), infrared, and other wireless media.

[0067] Figures 10A-10B A computing system 1000 is shown, such as a mobile phone, smartphone, wearable computer (e.g., smartwatch), tablet computer, desktop computer, laptop computer, etc., by which embodiments of the present disclosure can be implemented. Reference Figure 10A The diagram illustrates one aspect of a computing system 1000 used to implement these aspects. In a basic configuration, the computing system 1000 is a desktop computer with input and output elements. The computing system 1000 typically includes a display 1005, which may also be used as an input device (e.g., a touchscreen display). The computing system 1000 may also include a keyboard 1035. The keyboard 1035 may be a physical keyboard or a "soft" keyboard generated on the touchscreen display.

[0068] In various embodiments, output elements include a display 1005 for displaying a graphical user interface (GUI), a visual indicator 1020 (e.g., a light-emitting diode), and / or an audio transducer 1025 (e.g., a speaker). In another aspect, the computing system 1000 includes input and / or output ports for sending signals to or receiving signals from external devices, such as audio input ports (e.g., microphone jacks), audio output ports (e.g., headphone jacks), and video output ports (e.g., HDMI ports).

[0069] Figure 10B This is a block diagram illustrating the architecture of one aspect of a mobile computing system. That is, computing system 1000 can incorporate system (e.g., architecture) 1002 to achieve certain aspects. In one embodiment, system 1002 is implemented as a "computing system" capable of running one or more applications (e.g., browser, email, calendar, contact manager, messaging client, game, and media client / player). In some aspects, system 1002 is integrated as a computing system, such as a desktop computer.

[0070] One or more applications 1066 may be loaded into memory 1062 and run on or associated with operating system 1064. Examples of applications include telephone dialers, email programs, personal information management (PIM) programs, word processing programs, spreadsheet programs, internet browser programs, messaging programs, map programs, etc. System 1002 also includes a non-volatile storage area 1068 within memory 1062. The non-volatile storage area 1068 can be used to store persistent information that should not be lost when system 1002 is powered off. Applications 1066 may use and store information, such as emails or other messages used by email applications, in the non-volatile storage area 1068. A synchronization application (not shown) also resides on system 1002 and is programmed to interact with a corresponding synchronization application residing on the host to keep the information stored in the non-volatile storage area 1068 synchronized with the corresponding information stored on the host. It should be understood that other applications can be loaded into memory 1062 and run on the computing system 1000 described herein (e.g., search engine, extractor module, relevance ranking module, answer scoring module, etc.).

[0071] System 1002 has a power supply 1070, which can be implemented as one or more batteries. The power supply 1070 may also include an external power source, such as an AC adapter or power docking bracket for supplementing or recharging batteries.

[0072] System 1002 may also include a radio interface layer 1072, which performs the functions of transmitting and receiving radio frequency communications. Radio interface layer 1072 facilitates wireless connectivity between system 1002 and the "external world" through a communications operator or service provider. Transmissions to and from radio interface layer 1072 are conducted under the control of operating system 1064. In other words, communications received by radio interface layer 1072 can be propagated to application 1066 via operating system 1064, and vice versa.

[0073] System 1002 may also include a video interface 1076, which enables the onboard camera 1030 to operate for recording still images, video streams, etc. The computing system 1000 implementing system 1002 may have additional features or functions. For example, computing system 1000 may also include additional data storage devices (removable and / or non-removable), such as disks, optical discs, or magnetic tapes. Such additional storage devices... Figure 10B The non-volatile storage region 1068 is shown in the middle.

[0074] As described above, data / information generated or captured by computing system 1000 and stored via system 1002 can be stored locally on computing system 1000, or the data can be stored on any number of storage media accessible by the device via radio interface layer 1072, or accessed via a wired connection between computing system 1000 and a separate computing system associated with computing system 1000 (e.g., a server computer in a distributed computing network, such as the Internet). It should be understood that such data / information can be accessed via computing system 1000 via radio interface layer 1072 or via a distributed computing network. Similarly, such data / information can be easily transferred between computing systems used for storage and use based on well-known data / information transmission and storage devices (including email and collaborative data / information sharing systems).

[0075] Figure 11 This illustrates one aspect of the architecture of a system, as described above, for processing data received at a computing system from a remote source (such as a personal computer 1104, a tablet computing device 1106, or a mobile computing device 1108). The content at server device 1102 can be stored in different communication channels or other storage types. For example, various documents can be stored using directory services 1127, web portals 1128, email services 1129, instant messaging storage 1130, or social networking services 1131.

[0076] As described above, one or more of the previously described program modules 906 or software applications 920 can be used by server device 1102 and / or personal computer 1104, tablet computing device 1106, or mobile computing device 1108. For example, server device 1102 may include user API 1121, code generator 1122, and cloud SDK 1123. User API 1121 can be used with user API 204 ( Figure 2 ), 304 Figure 3 ), 404 Figure 4 ) and 504 Figure 5 The code generator 1122 is the same as or similar to the code generator 206. Figure 2 ), 306 Figure 3 ), 406 Figure 4 ) and 506 Figure 5 The cloud SDK can be the same as or similar to the cloud SDK 208. Figure 2 ), 308 Figure 3 ), 408 Figure 4 ) and 508 Figure 5 (Similar or identical. In the example, User API & Code Generator 116) Figure 1 It may include user API 1121 and code generator 1122.

[0077] Server device 1102 can provide data to and from client computing devices (such as personal computer 1104, tablet computing device 1106, and / or mobile computing device 1108 (e.g., smartphone)) via network 1115. For example, the aforementioned computer system can be implemented in personal computer 1104, tablet computing device 1106, and / or mobile computing device 1108 (e.g., smartphone). In addition to receiving content that can be preprocessed at the graphics initiation system or post-processed at the receiving computing system, any of these embodiments of the computing device can also obtain content from repository 1116.

[0078] Furthermore, the aspects and functions described herein can operate on distributed systems (e.g., cloud-based computing systems), where application functions, memory, data storage devices and retrieval, and various processing functions can remotely operate on each other via distributed computing networks (such as the Internet or intranets). Various types of user interfaces and information can be displayed via onboard computing device displays or via remote display units associated with one or more computing devices. For example, various types of user interfaces and information can be displayed on and interacted with on a wall, or projected onto a wall. Interaction with numerous computing systems that can implement embodiments of the present invention includes keystroke input, touchscreen input, voice or other audio input, gesture input, wherein the associated computing device is equipped with detection (e.g., camera) functions for capturing and interpreting user gestures used to control the functions of the computing device, etc.

[0079] For example, various aspects of this disclosure are described above with reference to block diagrams and / or operational schematics of methods, systems, and computer program products according to various aspects of this disclosure. Functions / actions indicated in the blocks may not occur in the order shown in any flowchart. For example, two blocks shown successively may, in fact, be performed substantially simultaneously, or sometimes may be performed in reverse order, depending on the functions / actions involved.

[0080] This disclosure relates to systems and methods for translating software expressions from user application programming interface (API) calls into software development kit (SDK) API calls, at least according to the examples provided in the following sections:

[0081] (A) In one aspect, some examples include a method for converting a software expression from a user application programming interface (API) call into an API call of a software development kit (SDK). The method may include receiving a tagged expression indicating that the conversion of the software expression from the user API call to the SDK API call is to be performed, the SDK being associated with a cloud-native high-performance computing environment; processing an abstract syntax tree associated with the software expression, the processing including replacing symbols in the abstract syntax tree with appropriate variables, replacing return statements in the abstract syntax tree with serialization instructions to write the result to local storage, and serializing the processed abstract syntax tree; and providing the serialized abstract syntax tree and one or more resource files to the cloud-native high-performance computing environment for execution.

[0082] (A2) In some examples of A1, the method includes creating a batch job to execute at least a portion of a serialized abstract syntax tree in a cloud-native high-performance computing environment; creating a storage container associated with the batch job; and creating a dedicated resource container associated with one or more resource files.

[0083] (A3) In some examples of A1-A2, batch jobs are multi-tasking batch jobs.

[0084] (A4) In some examples of A1-A3, the method includes serializing the output associated with the return statement into a filename; and adding the filename as an output resource to one or more resource files.

[0085] (A5) In some examples of A1-A4, the method includes expanding the software expression by collecting and analyzing symbols in the abstract syntax tree; splitting the software expression into multiple separate tasks such that a single expression is created for each task; and executing multiple separate tasks in parallel.

[0086] (A6) In some examples of A1-A5, the method includes analyzing multiple software expressions; and automatically labeling at least one of the multiple software expressions that lead to the labeled expression.

[0087] (A7) In some examples of A1-A6, the method includes providing a serialized abstract syntax tree resource file to a cloud-native high-performance computing environment.

[0088] In another aspect, some examples include a system comprising one or more processors and memory coupled to the one or more processors, the memory storing one or more instructions that, when executed by the one or more processors, cause the one or more processors to perform any of the methods described herein (e.g., A1-A7 described above).

[0089] In another aspect, some examples include a computer-readable storage medium storing one or more programs for execution by one or more processors of the device, the one or more programs including instructions for performing any of the methods described herein (e.g., A1-A7 described above).

[0090] The description and illustration of one or more aspects provided in this application are not intended to limit or restrict the scope of the claimed disclosure in any way. The aspects, examples, and details provided in this application are considered sufficient to convey the best mode of possession and to enable others to make and use the claimed disclosure. The claimed disclosure should not be construed as limited to any aspect, example, or detail provided in this application. Various features (structural and methodological) are intended to be selectively included or omitted, whether shown and described in combination or separately, to produce embodiments with a particular set of features. After the description and illustration of this application have been provided, those skilled in the art can conceive of variations, modifications, and alternatives within the spirit and scope of the broader aspects of the general inventive concept practiced in this application without departing from the wider scope of the claimed disclosure.

Claims

1. A method for converting software expressions from User Application Programming Interface (API) calls to Software Development Kit (SDK) API calls, the method comprising: Receive a tokenized expression indicating that a software expression transformation from a user API call to an SDK API call is to be performed, the SDK being associated with a cloud-native high-performance computing environment; Processing an abstract syntax tree associated with the software expression, the processing includes replacing symbols in the abstract syntax tree with corresponding variables, replacing return statements in the abstract syntax tree with serialization instructions to write the results to a local storage device, and serializing the processed abstract syntax tree; as well as The serialized abstract syntax tree and one or more resource files are provided to the cloud-native high-performance computing environment for execution.

2. The method according to claim 1, further comprising: Create a batch job to execute at least a portion of the serialized abstract syntax tree in the cloud-native high-performance computing environment; Create a storage container associated with the batch job; as well as Create a dedicated resource container associated with the one or more resource files.

3. The method according to claim 2, wherein the batch processing job is a multi-task batch processing job.

4. The method according to claim 1, further comprising: Serialize the output associated with the return statement into a filename; as well as Add the filename as an output resource to the one or more resource files.

5. The method according to claim 1, further comprising: The software expression is extended by collecting and analyzing the symbols in the abstract syntax tree; The software representation is divided into multiple individual tasks, such that a single representation is created for each task. as well as The multiple individual tasks are executed in parallel.

6. The method according to claim 5, further comprising: Analyze multiple software expressions; as well as At least one of the plurality of software expressions is automatically tagged, thereby generating the tagged expression.

7. The method of claim 1, further comprising providing the serialized abstract syntax tree as a resource file to the cloud-native high-performance computing environment.

8. A computer-readable storage medium comprising instructions that, when executed by a processor, cause the processor to: Receive a tokenized expression indicating that a software expression transformation from a user API call to an SDK API call is to be performed, the SDK being associated with a cloud-native high-performance computing environment; The abstract syntax tree associated with the software expression is processed by replacing the symbols in the abstract syntax tree with corresponding variables, replacing the return statements in the abstract syntax tree with serialization instructions to write the results to the local storage device, and the processed abstract syntax tree is serialized. as well as The serialized abstract syntax tree and one or more resource files are provided to the cloud-native high-performance computing environment for execution.

9. The computer-readable storage medium of claim 8, wherein the instructions, when executed by a processor, cause the processor to: Create a batch job to execute at least a portion of the serialized abstract syntax tree in the cloud-native high-performance computing environment; Create a storage container associated with the batch job; and Create a dedicated resource container associated with the one or more resource files.

10. The computer-readable storage medium of claim 9, wherein the batch processing job is a multi-tasking batch processing job.

11. The computer-readable storage medium of claim 8, wherein the instructions, when executed by a processor, cause the processor to: Serialize the output associated with the return statement into a filename; and Add the filename as an output resource to the one or more resource files.

12. The computer-readable storage medium of claim 8, wherein the instructions, when executed by a processor, cause the processor to: The software expression is extended by collecting and analyzing the symbols in the abstract syntax tree; The software representation is divided into multiple individual tasks, such that a single representation is created for each task; and The multiple individual tasks are executed in parallel.

13. The computer-readable storage medium of claim 12, wherein the instructions, when executed by a processor, cause the processor to: Analyze multiple software expressions; and At least one of the plurality of software expressions is automatically tagged, thereby generating the tagged expression.

14. The computer-readable storage medium of claim 8, wherein the instructions, when executed by a processor, cause the processor to provide the serialized abstract syntax tree as a resource file to the cloud-native high-performance computing environment.

15. A system comprising: One or more hardware processors configured by machine-readable instructions, to: Receive a tokenized expression indicating that a software expression transformation from a user API call to an SDK API call is to be performed, the SDK being associated with a cloud-native high-performance computing environment; Processing an abstract syntax tree associated with the software expression, the processing includes replacing symbols in the abstract syntax tree with corresponding variables, replacing return statements in the abstract syntax tree with serialization instructions to write the results to a local storage device, and serializing the processed abstract syntax tree; as well as The serialized abstract syntax tree and one or more resource files are provided to the cloud-native high-performance computing environment for execution.

16. The system of claim 15, wherein the one or more hardware processors are further configured by machine-readable instructions to: Create a batch job to execute at least a portion of the serialized abstract syntax tree in the cloud-native high-performance computing environment; Create a storage container associated with the batch job; and Create a dedicated resource container associated with the one or more resource files.

17. The system of claim 16, wherein the batch processing job is a multi-task batch processing job.

18. The system of claim 15, wherein the one or more hardware processors are further configured by machine-readable instructions to: Serialize the output associated with the return statement into a filename; and Add the filename as an output resource to the one or more resource files.

19. The system of claim 15, wherein the one or more hardware processors are further configured by machine-readable instructions to: The software expression is extended by collecting and analyzing the symbols in the abstract syntax tree; The software representation is divided into multiple individual tasks, such that a single representation is created for each task; and The multiple individual tasks are executed in parallel.

20. The system of claim 19, wherein the one or more hardware processors are further configured by machine-readable instructions to: Analyze multiple software expressions; and At least one of the plurality of software expressions is automatically tagged, thereby generating the tagged expression.

Images