Heterogeneous Workflow Execution Engine with Intermediary Data Conversion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing workflows that combine functions written in different programming languages face challenges due to varying technical features, grammar, and execution contexts, limiting the ability to compile and execute them seamlessly in a single task, leading to performance degradation and increased costs for data transfer and language proficiency.
Innovation Solution
A method and device for executing heterogeneous language functions by obtaining workflows with functions in multiple programming languages, converting output data from one language to a format processable by another, and selecting an optimal combination of candidate functions for execution, allowing for seamless data transfer and minimized execution time without external storage or separate application programs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If functions from multiple programming languages are combined in a workflow, then the optimal performance and result can be derived by using the function depending on data characteristics and the analysis purpose, but the workflow cannot be compiled in one task because a compiling manner is different for each programming language
Solution Approach 1:
The patent introduces an intermediary compilation mechanism that translates heterogeneous programming language functions into a unified intermediate representation. This mediator handles the compilation of multiple programming languages (e.g., SQL, Python, R) into a common format that can be executed within a single workflow task, resolving the conflict between language diversity and compilation unity.
Solution Approach 2:
The patent creates a universal workflow execution engine that can handle multiple programming languages through a single compilation process. The system provides multi-functional capability by supporting various language-specific compilation modes while maintaining a unified execution framework, allowing the workflow to compile and execute functions from different languages without requiring separate compilation processes.
2Ease of operation
If data transfer between heterogeneous language functions is performed manually, then language proficiency is required which increases costs, but automated data transfer and format conversion are not implemented
Solution Approach 1:
The patent implements self-service automated data transfer mechanisms where the workflow execution engine automatically performs data format conversion and transfer between heterogeneous language functions. The system detects data type mismatches and performs appropriate conversions (e.g., between SQL result sets and Python data structures) without requiring manual intervention, thereby eliminating the need for language proficiency while reducing transfer time.
Solution Approach 2:
The patent performs preliminary data format standardization and conversion before executing heterogeneous language functions. The system pre-processes data to ensure compatibility between different language environments, converting data into appropriate formats in advance of function execution, which eliminates the need for manual data transfer and reduces execution time.
3Productivity
If output data of one function is set in a format capable of being processed in another programming language, then seamless execution in one task is enabled, but additional format conversion steps are required
Solution Approach 1:
The patent introduces an intermediary data representation layer that automatically handles format conversion between different programming languages. The system uses a universal intermediate data format that can represent data structures from multiple languages (SQL, Python, R, etc.) and automatically converts between them, enabling seamless execution without manual format conversion while managing the complexity through automated translation mechanisms.
Data Source
AI summary
A method and device for executing a workflow includes functions written in a heterogeneous programming language. The method for executing heterogeneous language functions includes obtaining a workflow that includes a call for a first function written in a first programming language and a call for a second function written in a second programming language, wherein input data of the second function includes output data of the first function, and setting, in response to completing execution of the first function, the output data of the first function in a format capable of being processed in the second programming language.