DAG Task Flow Language with Layered Definitions and Automatic Parallelism
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing task flow description languages in scientific computing lack precision in task granularity, require high computer knowledge for non-specialists, lack data reuse capabilities, and do not support automatic parallelism, making them difficult to use for both algorithm writers and users.
Innovation Solution
A general description language system for directed acyclic graph automatic task flow, comprising a Step definition layer, Workflow definition layer, and Template definition layer, with TypeDef definition layer for custom types, enabling high-precision task descriptions, data checking, and automatic parallelism through keywords like 'scatter-gather'.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If existing general description language is used for task flow definition, then versatility is improved, but task description granularity becomes rough and data checking capability is lost
Solution Approach 1:
The patent segments the task description into multiple hierarchical layers: workflow layer (task flow), step layer (individual task), and parameter layer (detailed parameters). Each layer handles specific aspects of task definition, allowing fine-grained parameter description while maintaining overall workflow versatility. The step layer specifically addresses granularity by enabling detailed parameter definitions with data types, default values, and validation rules.
Solution Approach 2:
The patent introduces a vertical dimension to task description by adding structured parameter definitions with multiple attributes (type, required, default_value, enum, etc.). This transforms the flat task description into a multi-dimensional structure where parameters can be defined with rich metadata, enabling both fine granularity and comprehensive validation while maintaining versatility.
2Adaptability or versatility
If existing general description language is used, then computer field coverage is improved, but knowledge threshold increases making it inconvenient for non-computer professionals
Solution Approach 1:
The patent introduces an intermediary layer (the structured parameter definition system) between the workflow definition and actual task execution. This intermediary provides standardized parameter templates with pre-defined data types, validation rules, and default values, which simplifies the arrangement process for non-computer professionals while maintaining compatibility with computer science domains through its structured nature.
Solution Approach 2:
The patent changes the parameter representation from simple text strings to structured objects with multiple attributes (type, required, default_value, enum, etc.). This parameter transformation enables automatic validation, provides better defaults, and reduces the cognitive load on users by presenting organized parameter groups rather than raw text, thereby lowering the knowledge threshold while maintaining broad applicability.
3Adaptability or versatility
If existing general description language is used for scientific computing, then parameter definition capability is improved, but information complexity increases and data reuse is lost
Solution Approach 1:
The patent applies preliminary action by providing pre-defined parameter templates with standard data types, validation rules, and default values. These templates are prepared in advance and can be reused across multiple workflows and steps, reducing information complexity through standardization while maintaining strong parameter definition capability. The enum type with predefined options is a specific example of this preliminary structuring.
Solution Approach 2:
The patent enables copying and reusing of parameter definitions across different workflows and steps through its structured parameter layer. Parameter templates can be defined once and referenced multiple times, reducing information complexity by eliminating redundant definitions while maintaining adaptability through the rich parameter attribute structure that can be consistently applied across different scientific computing contexts.
4Adaptability or versatility
If existing general description language is used, then task flow definition is improved, but automatic parallelism capability is lost
Solution Approach 1:
The patent introduces dynamics to the task flow definition by enabling parameters that control execution behavior, including parallelism. Through structured parameter definitions, workflows can specify dynamic execution patterns such as parallel task execution, conditional branching, and resource allocation. This allows the system to automatically adapt the execution flow based on parameter values while maintaining a clear task flow definition structure.
Data Source
AI summary
The present invention provides a general description language data system for directed acyclic graph automatic task flow, including: Step definition layer, Workflow definition layer and Template definition layer; The Step definition layer is the description of a single task, for the input and output declarations of each docker image or other executor, comprises name, type, file and parameters. The Workflow definition layer is a workflow composed of one or more Steps, the dependency topology of these Steps needs to be defined, and shared parameters can also be defined. The Template definition layer is based on a Workflow definition layer. The Template definition layer pre-sets the parameters, and supplies the descriptions, checkers or data source definitions of the parameters. The data center of the present invention is used with the task execution tool, and a programming language needs to be used to implement the corresponding tool.