An ANTLR and DSL-based distributed task orchestration method, system, device and medium
By using the distributed task orchestration method of ANTLR and DSL, the problems of high flexibility, readability and deployment cost in existing task orchestration technologies are solved, and a task orchestration solution with rapid iteration and low resource consumption is realized.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- INSPUR ZHUOSHU BIG DATA IND DEV CO LTD
- Filing Date
- 2026-01-29
- Publication Date
- 2026-06-19
AI Technical Summary
Existing task orchestration technologies suffer from several drawbacks: hard-coded orchestration lacks flexibility, XML/JSON-based configuration orchestration has poor readability and maintainability, and general workflow engines have high learning and deployment costs.
A distributed task orchestration method based on ANTLR and DSL is adopted. By obtaining DSL grammar rules, lexical and grammar rules are generated, an analyzer is generated using ANTLR tools, an abstract syntax tree is constructed, the semantic relationships of grammar nodes are read, a directed acyclic graph of tasks is constructed, and task node objects are executed.
It enables rapid iteration and flexible adjustment of business processes, improves script readability and maintainability, reduces system resource consumption and deployment complexity, and is suitable for small teams and complex distributed scenarios.
Smart Images
Figure CN122240113A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of distributed task orchestration technology, and in particular to a distributed task orchestration method, system, device and medium based on ANTLR and DSL. Background Technology
[0002] With the popularization of microservice architecture and distributed systems, complex business logic is usually broken down into multiple independent, reusable tasks or services. These tasks need to be orchestrated and executed in a specific logical order (such as serial, parallel, conditional branching, looping, etc.) to complete a complete business process.
[0003] The existing task orchestration technologies mainly include the following: 1. Hard-coded orchestration: Directly writing task invocation logic in programming languages (such as Java and Python). The disadvantage of this approach is its extremely poor flexibility; any change in the process requires modifying the code, recompiling, and deploying, making it difficult to adapt to rapidly changing business needs.
[0004] 2. XML / JSON-based configuration orchestration: Using common configuration files such as XML or JSON to define task flows. While this achieves decoupling, these common formats are verbose and lack expressiveness. They often fail to intuitively represent business processes, lack necessary syntax validation and capabilities (such as conditional statements and loops), and still have poor readability and maintainability.
[0005] 3. Use a general-purpose workflow engine: Existing workflow systems, such as Apache Airflow, are powerful but too heavyweight, with high learning and deployment costs. Summary of the Invention
[0006] This application provides a distributed task orchestration method, system, device, and medium based on ANTLR and DSL to solve the problems of poor flexibility of existing hard-coded orchestration, poor readability and maintainability of XML / JSON-based configuration orchestration, and high learning and deployment costs of general workflow engines.
[0007] Firstly, this application provides a distributed task orchestration method based on ANTLR and DSL, the method comprising: Obtain the DSL syntax rules (Domain-Specific Language) involved in all tasks in the current distributed task orchestration scenario. Based on the DSL grammar rules, lexical rules and grammar rules are generated; using the ANTLR tool, lexical analyzers corresponding to the lexical rules and grammar analyzers corresponding to the grammar rules are generated. Obtain the actual DSL script file, input the actual DSL script file into the lexical analyzer to obtain lexical unit tokens, input the lexical unit tokens into the syntax analyzer to obtain the abstract syntax tree (AST); Read the syntax nodes and semantic relationships between them from the abstract syntax tree (AST); convert the syntax nodes into predefined task node objects; and construct a directed acyclic graph (DAG) for the task based on the semantic relationships and the predefined task node objects. The task execution engine loads the task directed acyclic graph (DAG) and executes the preset task node objects based on the semantic relationships of the task DAG.
[0008] In one implementation of this application, the syntax rules include at least: task declaration, sequential flow, parallel flow, conditional branching, and loop structure; The task declaration includes all tasks involved in the DSL syntax rules and the input parameters for each task; and all tasks are divided into start tasks, intermediate tasks and end tasks. Sequential flow includes the sequential order of tasks; Parallel streams consist of several tasks running in parallel; Conditional branches include triggering preset feedback data when the task meets preset conditions; The loop structure includes executing a preset loop procedure when the task satisfies a preset loop keyword.
[0009] In one implementation of this application, lexical rules and grammatical rules are generated based on DSL grammar rules, specifically including: Based on the atomic symbols in task declarations, sequential streams, parallel streams, conditional branches, and loop structures involved in the DSL syntax rules, lexical rules for reading atomic symbols of regular expression patterns are generated; where atomic symbols include: task identifiers, parameter binders, control keywords, delimiters, and literals; Construct syntax rules for recursively defining sequential flow, parallel flow, conditional branch, and loop structures, with the starting task in the task declaration as the root node; among them, the sequential flow, parallel flow, conditional branch, and loop structures define intermediate and terminating tasks in the task declaration.
[0010] In one implementation of this application, the actual DSL script file is obtained, the actual DSL script file is input into a lexical analyzer to obtain lexical unit tokens, and the lexical unit tokens are input into a syntax analyzer to obtain an abstract syntax tree (AST), specifically including: Obtain the actual DSL script file through the preset data acquisition interface; Input the actual DSL script file into the lexical analyzer to obtain the initial symbols; Transform the initial symbols into lexical units (Tokens) of preset standard atomic symbols; The lexical unit Token is input into the parser, and the lexical unit Token is filled into the preset grammar rules in the parser to generate an abstract syntax tree (AST).
[0011] In one implementation of this application, syntax nodes and semantic relationships between syntax nodes are read from the Abstract Syntax Tree (AST); the syntax nodes are converted into preset task node objects; and a directed acyclic graph (DAG) for tasks is constructed based on the semantic relationships and the preset task node objects, specifically including: Traverse the AST to obtain all syntax nodes and their nested semantic relationships. The syntax nodes include task declaration nodes, sequential flow connection nodes, parallel flow branching nodes, parallel flow converging nodes, conditional branch judgment nodes, conditional branch merging nodes, loop entry nodes, and loop return nodes. The nested semantic relationships between syntax nodes are directional semantic relationships between nodes. Syntax nodes are mapped to a unified task node object model of DAG; edge connections of DAG are constructed based on the directional semantic relationships between syntax nodes.
[0012] Secondly, this application provides a distributed task orchestration system based on ANTLR and DSL, the system comprising: The acquisition module is used to acquire the DSL syntax rules involved in all tasks in the current distributed task orchestration scenario; The generation module is used to generate lexical rules and grammar rules based on DSL grammar rules; and to generate lexical analyzers corresponding to lexical rules and grammar analyzers corresponding to grammar rules using the ANTLR tool. The syntax tree module is used to obtain the actual DSL script file, input the actual DSL script file into the lexical analyzer to obtain lexical unit tokens, and input the lexical unit tokens into the syntax analyzer to obtain the abstract syntax tree (AST). The execution module is used to read syntax nodes and semantic relationships between syntax nodes from the abstract syntax tree (AST); convert syntax nodes into preset task node objects; construct a directed acyclic graph (DAG) of tasks based on semantic relationships and preset task node objects; load the task DAG of tasks through the task execution engine; and execute the preset task node objects based on the semantic relationships of the task DAG of tasks.
[0013] In one implementation of this application, the generation module includes a generation unit. Lexical rules for generating regular expression patterns by reading atomic symbols in task declarations, sequential streams, parallel streams, conditional branches, and loop structures based on DSL syntax rules; wherein atomic symbols include: task identifiers, parameter binders, control keywords, delimiters, and literals; Construct syntax rules for recursively defining sequential flow, parallel flow, conditional branch, and loop structures, with the starting task in the task declaration as the root node; among them, the sequential flow, parallel flow, conditional branch, and loop structures define intermediate and terminating tasks in the task declaration.
[0014] In one implementation of this application, the syntax tree module includes syntax tree units. Used to obtain the actual DSL script file through a preset data acquisition interface; Input the actual DSL script file into the lexical analyzer to obtain the initial symbols; Transform the initial symbols into lexical units (Tokens) of preset standard atomic symbols; The lexical unit Token is input into the parser, and the lexical unit Token is filled into the preset grammar rules in the parser to generate an abstract syntax tree (AST).
[0015] Thirdly, this application provides a distributed task orchestration device based on ANTLR and DSL, the device comprising: processor; And a memory containing executable code, which, when executed, causes the processor to execute a distributed task orchestration method based on ANTLR and DSL, as described above.
[0016] Fourthly, this application provides a non-volatile computer storage medium storing computer instructions, which, when executed, implement a distributed task orchestration method based on ANTLR and DSL as described above.
[0017] As can be seen from the above technical solutions, this application has the following advantages: Compared to the inefficient process of hard-coding, which requires modifying source code and recompiling and deploying, this application defines task flows through independent DSL script files. Any business process adjustment only requires modifying the DSL script, without touching the underlying execution engine or recompiling the system. This loosely coupled design allows business personnel to directly participate in process definition, enabling rapid iteration of "configuration as process," greatly shortening the requirement response cycle, and fundamentally solving the core pain points of rigidity and high change costs of traditional hard-coding solutions.
[0018] Compared to general configuration formats like XML / JSON, the domain-specific language used in this application is designed specifically for task orchestration scenarios. It possesses a syntax structure that natively supports business logic such as conditional branching, loops, and parallel control, offering clear semantics and concise expression. The ANTLR-generated lexical and syntax analyzers can perform syntax verification and semantic constraints during the script parsing phase, improving script readability and maintainability. Business processes are presented in a near-natural language DSL format, allowing both technical and non-technical personnel to intuitively understand task dependencies. This avoids the problems of lengthy, nested, and obscure logic found in general configuration files, achieving a qualitative leap from "configuration files" to "business documents."
[0019] Compared to general-purpose workflow engines like Apache Airflow, this application eliminates the need for complex service clusters, database dependencies, message queues, or web UIs. It only requires a lightweight DSL parser and task execution engine. The parser generated by ANTLR is a statically compiled artifact, incurring no additional framework overhead at runtime and consuming minimal system resources. Deployment only requires distributing the DSL scripts and executor binaries. Furthermore, by first constructing an AST to extract semantic relationships and then converting it into a task DAG, the decoupling of syntax parsing and execution scheduling is achieved. The DAG serves as the standard input for the distributed task execution engine, directly recognized by the engine and scheduled for execution according to semantic relationships. The overall architecture focuses on core orchestration needs, with no redundant functional modules. The deployment process is simple and resource-efficient, enabling rapid deployment at a low cost for both small teams and complex distributed scenarios, thus solving the problem of limited applicability of general-purpose workflow engines. Attached Figure Description
[0020] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0021] Figure 1 This is a flowchart of a distributed task orchestration method based on ANTLR and DSL provided in an embodiment of this application.
[0022] Figure 2 This is a schematic diagram of the internal structure of a distributed task orchestration system based on ANTLR and DSL provided in an embodiment of this application.
[0023] Figure 3 This is a schematic diagram of the internal structure of a distributed task orchestration device based on ANTLR and DSL provided in an embodiment of this application. Detailed Implementation
[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0025] Those skilled in the art should understand that the embodiments described below are merely preferred embodiments of this disclosure and do not imply that this disclosure can only be implemented through these preferred embodiments. These preferred embodiments are merely used to explain the technical principles of this disclosure and are not intended to limit the scope of protection of this disclosure. Based on the preferred embodiments provided by this disclosure, all other embodiments obtained by those skilled in the art without creative effort should still fall within the scope of protection of this disclosure.
[0026] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.
[0027] The technical solutions proposed in the embodiments of this application will be described in detail below with reference to the accompanying drawings.
[0028] The embodiment provides a distributed task orchestration method based on ANTLR and DSL, such as Figure 1 As shown in the embodiments of this application, the method mainly includes the following steps: Step 110: Obtain the DSL syntax rules involved in all tasks in the current distributed task orchestration scenario.
[0029] In some embodiments, the syntax rules include at least: task declaration, sequential flow, parallel flow, conditional branching, and loop structure; The task declaration includes all tasks covered by the DSL syntax rules and the input parameters for each task; and all tasks are divided into start tasks, intermediate tasks and end tasks.
[0030] As an example, a single task declaration can be specifically defined as: TASK task_name(param1, param2). task_name is the task identifier, and param1 and param2 are parameter binders.
[0031] Sequential flow includes the sequential order of tasks; as an example, the -> symbol is used to indicate the order in which tasks are executed.
[0032] Example: TaskA -> TaskB; means that TaskB will only be executed after TaskA has been successfully executed.
[0033] Parallel streams consist of several tasks running in parallel; as an example, the '&' symbol is used to indicate that multiple tasks can be executed in parallel.
[0034] Example: TaskA&TaskB ->TaskC; means that TaskA and TaskB are executed in parallel, and TaskC is executed only after both are successful.
[0035] Conditional branching includes triggering preset feedback data when a task meets preset conditions; as an example, the IF...ELSE... keyword is used to implement conditional routing based on execution results or context.
[0036] Example: IF condition; ->TaskSuccess; ELSE; ->TaskFail; ENDIF.
[0037] Loop structures involve executing a predefined loop procedure when a task satisfies a predefined loop keyword. As an example, the FOR or WHILE keywords are used to implement loop execution.
[0038] Example: FOR i IN range(1, 10) DO ->ProcessItem(i). Here, i represents the default loop keyword.
[0039] Step 120: Based on the DSL grammar rules, generate lexical rules and grammar rules; using the ANTLR tool, generate the lexical analyzer corresponding to the lexical rules and the grammar analyzer corresponding to the grammar rules.
[0040] Specifically, based on DSL grammar rules, lexical rules and grammar rules are generated, which can be: Based on the atomic symbols in task declarations, sequential streams, parallel streams, conditional branches, and loop structures involved in the DSL syntax rules, lexical rules (.g4 files) for reading atomic symbols of regular patterns are generated; where atomic symbols include: task identifiers, parameter binders, control keywords, delimiters, and literals; Construct syntax rules for recursively defining sequential flow, parallel flow, conditional branch, and loop structures, with the starting task in the task declaration as the root node; among them, the sequential flow, parallel flow, conditional branch, and loop structures define intermediate and terminating tasks in the task declaration.
[0041] It should be further explained that in step 120, the generation of lexical rules is based on the atomic symbols defined in the DSL syntax rules. Task identifiers are mapped to regular expressions [a-zA-Z_][a-zA-Z0-9_]*. Parameter binding operators := and control keywords such as if, else, while, do, and end are directly matched in literal form. Delimiters use exact character matching, and literal strings are identified through the pattern "([^"\\]|\\.)*". These regular expression patterns are encapsulated into an ANTLR .g4 lexical grammar file to ensure that the lexical analyzer can accurately distinguish between identifiers and keywords when parsing the DSL script, avoiding the misidentification of reserved words as task names, thereby ensuring the uniqueness and traceability of lexical unit tokens.
[0042] The grammar rules are defined recursively, with the starting task as the root node. Sequential flows are implemented using the rule `statement : task(',' task)+;`, enabling chained task calls. Parallel flows are defined by `parallelBlock : '{' task(',' task)* '}';`, supporting concurrent entry points for multiple tasks. Conditional branches use `ifBlock : 'if' expression 'then' statement ('else' statement)`, where `expression` is a Boolean sub-rule. Loop structures, `whileBlock : 'while' expression 'do' statement 'end'`, are nested within statement rules, allowing the loop body to contain any legal statement, including nested conditions or loops. ANTLR automatically generates a parser based on this grammar, capable of recursively parsing nested structures of arbitrary depth, ensuring that the construction of the task's directed acyclic graph is completely consistent with the DSL semantics.
[0043] Step 130: Obtain the actual DSL script file, input the actual DSL script file into the lexical analyzer to obtain lexical unit tokens, input the lexical unit tokens into the syntax analyzer to obtain the abstract syntax tree (AST).
[0044] This step can be specifically described as follows: Obtain the actual DSL script file through the preset data acquisition interface; Input the actual DSL script file into the lexical analyzer to obtain the initial symbols; Transform the initial symbols into lexical units (Tokens) of preset standard atomic symbols (such as TaskNode, SequenceFlow, ParallelGateway, ConditionNode, etc.). The lexical unit Token is input into the parser, and the lexical unit Token is filled into the preset grammar rules in the parser to generate an abstract syntax tree (AST).
[0045] It should be noted that in step 130, the actual DSL script file is read in UTF-8 encoding format through a preset data acquisition interface, and the file content is input as a character stream to the lexical analyzer generated by ANTLR. The lexical analyzer scans the input stream character by character, identifies the initial symbols according to the regular expression patterns defined in the .g4 file, and classifies them into lexical units Tokens of preset standard atomic symbols, including but not limited to: TaskNode (corresponding to task identifiers), SequenceFlow (corresponding to task chains composed of commas), ParallelGateway (corresponding to parallel blocks enclosed in { and}), ConditionNode (corresponding to combinations of "if", "then", and "else" keywords), and LoopNode (corresponding to "while", "do", and "end" structures). The lexical analysis process does not skip whitespace characters but ignores comment lines (starting with " / / ") to ensure that the Token sequence strictly corresponds to the DSL source code structure and avoids parsing failures due to encoding or format differences.
[0046] The generated token sequences are input sequentially into the ANTLR-generated parser. The parser loads the grammar rules constructed in step 120, recursively matching the token sequences with the starting rule `program : statement EOF` as the entry point. For sequential, parallel, conditional, and loop structures, the parser constructs syntax tree nodes layer by layer according to preset recursive rules. Each successfully matched grammar rule corresponds to an AST node, whose type is consistent with the token type. For example, a `SequenceFlow` token triggers a `StatementList` node, and a `ConditionNode` triggers an `IfStatement` node. If an invalid token or missing structure is encountered during parsing, a syntax error is immediately thrown and parsing is terminated without error recovery, ensuring that the AST is generated only when the DSL semantics are completely valid. Each node in the generated AST carries positional information (row number, column number) and semantic labels, providing an accurate structural mapping basis for subsequent conversion into a directed acyclic graph (DAG).
[0047] Step 140: Read the syntax nodes and semantic relationships between them from the Abstract Syntax Tree (AST); convert the syntax nodes into preset task node objects; and construct a Directed Acyclic Graph (DAG) for tasks based on the semantic relationships and the preset task node objects.
[0048] In some embodiments, syntax nodes and semantic relationships between syntax nodes are read from the abstract syntax tree (AST); syntax nodes are converted into preset task node objects; and a directed acyclic graph (DAG) for tasks is constructed based on the semantic relationships and the preset task node objects, specifically including: Traverse the AST to obtain all syntax nodes and their nested semantic relationships. The syntax nodes include task declaration nodes, sequential flow connection nodes, parallel flow branching nodes, parallel flow converging nodes, conditional branch judgment nodes, conditional branch merging nodes, loop entry nodes, and loop return nodes. The nested semantic relationships between syntax nodes are directional semantic relationships between nodes. Syntax nodes are mapped to a unified task node object model of DAG; edge connections of DAG are constructed based on the directional semantic relationships between syntax nodes.
[0049] In step 140, the Abstract Syntax Tree (AST) is traversed using depth-first search to identify and extract the type and nested semantic relationships of each node. Syntax nodes include: task declaration nodes (corresponding to task identifiers in the DSL), sequential flow connection nodes (triggered by commas), parallel flow branching nodes (started by {), parallel flow converging nodes (closed by}), conditional branch judgment nodes (triggered by if expressions), conditional branch merging nodes (identified by else or implicit path merging points), loop entry nodes (triggered by while), and loop return nodes (marked by end to indicate the jump target). During the traversal, each node records its parent, child, and sibling relationships in the AST, forming a directional semantic chain to ensure the logical integrity of the task execution path and avoid circular dependencies or broken paths.
[0050] It should be further explained that syntax nodes are mapped to a unified task node object model, which includes fields: node ID (unique identifier), node type (enumeration values: Task, SequenceFlow, ParallelFork, ParallelJoin, ConditionJudge, ConditionMerge, LoopEntry, LoopBack), input dependency list (pointing to the set of parent node IDs), and output dependency list (pointing to the set of child node IDs). Based on the directional semantic relationships established in the AST, directed edges are added to each pair of parent and child nodes in the DAG, with an edge weight of 0 (only indicating the control flow order, not carrying computational overhead). Parallel flow branching nodes generate multiple output edges, and convergence nodes merge multiple input edges; conditional branching nodes dynamically select the output path based on the expression evaluation result, and loop return nodes point to the entry node to form a closed loop but do not constitute a cycle, as the number of times the loop body is executed is controlled by runtime conditions. After the DAG is constructed, the node set and edge set constitute a static topology structure that can be directly loaded by the task execution engine, ensuring that the execution order is completely consistent with the DSL semantics.
[0051] Step 150: Load the task directed acyclic graph (DAG) through the task execution engine, and execute the preset task node objects according to the semantic relationships of the task directed acyclic graph (DAG).
[0052] The task execution engine receives the generated directed acyclic graph (DAG) of tasks and schedules them according to the dependencies within the DAG. The engine is responsible for: 1. Analyze the specific implementation of each task node (e.g., associating it with a Java class, an HTTP interface, or a gRPC service).
[0053] 2. Manage the execution context, responsible for passing parameters and status between tasks.
[0054] 3. Trigger task execution asynchronously according to the logic (sequential, parallel, conditional) defined by the DAG.
[0055] 4. Monitor task execution status (success, failure), handle retries, timeouts, and failover.
[0056] In addition, this application Figure 2 This application provides a distributed task orchestration system based on ANTLR and DSL. For example... Figure 2 As shown in the embodiments of this application, the system mainly includes: The acquisition module 210 is used to acquire the DSL syntax rules involved in all tasks in the current distributed task orchestration scenario; The generation module 220 is used to generate lexical rules and grammar rules based on DSL grammar rules; and to generate lexical analyzers corresponding to lexical rules and grammar analyzers corresponding to grammar rules using the ANTLR tool. The generation module 220 includes a generation unit for generating lexical rules for reading atomic symbols of regular patterns based on atomic symbols in task declarations, sequential streams, parallel streams, conditional branches, and loop structures involved in the DSL syntax rules; wherein, atomic symbols include: task identifiers, parameter binders, control keywords, delimiters, and literals; Construct syntax rules for recursively defining sequential flow, parallel flow, conditional branch, and loop structures, with the starting task in the task declaration as the root node; among them, the sequential flow, parallel flow, conditional branch, and loop structures define intermediate and terminating tasks in the task declaration.
[0057] The syntax tree module 230 is used to obtain the actual DSL script file, input the actual DSL script file into the lexical analyzer to obtain lexical unit tokens, and input the lexical unit tokens into the syntax analyzer to obtain the abstract syntax tree (AST).
[0058] Syntax tree module 230 includes syntax tree units. Used to obtain the actual DSL script file through a preset data acquisition interface; Input the actual DSL script file into the lexical analyzer to obtain the initial symbols; Transform the initial symbols into lexical units (Tokens) of preset standard atomic symbols; The lexical unit Token is input into the parser, and the lexical unit Token is filled into the preset grammar rules in the parser to generate an abstract syntax tree (AST).
[0059] The execution module 240 is used to read syntax nodes and semantic relationships between syntax nodes from the abstract syntax tree (AST); convert syntax nodes into preset task node objects; construct a task directed acyclic graph (DAG) based on semantic relationships and preset task node objects; load the task DAG through the task execution engine; and execute the preset task node objects based on the semantic relationships of the task DAG.
[0060] The above are method embodiments of this application. Based on the same inventive concept, embodiments of this application also provide a distributed task orchestration device based on ANTLR and DSL. Figure 3 As shown, the device includes: a processor; and a memory storing executable code thereon, which, when executed, causes the processor to perform a distributed task orchestration method based on ANTLR and DSL as described in the above embodiment.
[0061] Specifically, the server obtains the DSL syntax rules involved in all tasks in the current distributed task orchestration scenario; based on the DSL syntax rules, it generates lexical rules and syntax rules; using the ANTLR tool, it generates lexical analyzers corresponding to the lexical rules and syntax analyzers corresponding to the syntax rules; it obtains the actual DSL script file, inputs the actual DSL script file into the lexical analyzer to obtain lexical unit tokens, inputs the lexical unit tokens into the syntax analyzer to obtain an abstract syntax tree (AST); it reads the syntax nodes and semantic relationships between syntax nodes from the AST; it converts the syntax nodes into preset task node objects, and constructs a directed acyclic graph (DAG) of tasks based on the semantic relationships and the preset task node objects; it loads the task DAG of tasks through the task execution engine, and executes the preset task node objects according to the semantic relationships of the task DAG of tasks.
[0062] In addition, embodiments of this application also provide a non-volatile computer storage medium storing executable instructions, which, when executed, implement the distributed task orchestration method based on ANTLR and DSL as described above.
[0063] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A distributed task orchestration method based on ANTLR and DSL, characterized in that, The method includes: Obtain the DSL syntax rules involved in all tasks in the current distributed task orchestration scenario; Based on the DSL grammar rules, lexical rules and grammar rules are generated; using the ANTLR tool, lexical analyzers corresponding to the lexical rules and grammar analyzers corresponding to the grammar rules are generated. Obtain the actual DSL script file, input the actual DSL script file into the lexical analyzer to obtain lexical unit tokens, input the lexical unit tokens into the syntax analyzer to obtain the abstract syntax tree (AST); Read the syntax nodes and semantic relationships between them from the abstract syntax tree (AST); convert the syntax nodes into predefined task node objects; and construct a directed acyclic graph (DAG) for the task based on the semantic relationships and the predefined task node objects. The task execution engine loads the task directed acyclic graph (DAG) and executes the preset task node objects based on the semantic relationships of the task DAG.
2. The distributed task orchestration method based on ANTLR and DSL according to claim 1, characterized in that, The syntax rules should include at least: task declaration, sequential flow, parallel flow, conditional branching, and loop structures; The task declaration includes all tasks involved in the DSL syntax rules and the input parameters for each task; and all tasks are divided into start tasks, intermediate tasks and end tasks. Sequential flow includes the sequential order of tasks; Parallel streams consist of several tasks running in parallel; Conditional branches include triggering preset feedback data when the task meets preset conditions; The loop structure includes executing a preset loop procedure when the task satisfies a preset loop keyword.
3. The distributed task orchestration method based on ANTLR and DSL according to claim 1, characterized in that, Based on DSL grammar rules, lexical and grammatical rules are generated, specifically including: Based on the atomic symbols in task declarations, sequential streams, parallel streams, conditional branches, and loop structures involved in the DSL syntax rules, lexical rules for reading atomic symbols of regular expression patterns are generated; where atomic symbols include: task identifiers, parameter binders, control keywords, delimiters, and literals; Construct syntax rules for recursively defining sequential flow, parallel flow, conditional branch, and loop structures, with the starting task in the task declaration as the root node; among them, the sequential flow, parallel flow, conditional branch, and loop structures define intermediate and terminating tasks in the task declaration.
4. The distributed task orchestration method based on ANTLR and DSL according to claim 1, characterized in that, Obtain the actual DSL script file, input the actual DSL script file into the lexical analyzer to obtain lexical unit tokens, input the lexical unit tokens into the syntax analyzer to obtain the abstract syntax tree (AST), which specifically includes: Obtain the actual DSL script file through the preset data acquisition interface; Input the actual DSL script file into the lexical analyzer to obtain the initial symbols; Transform the initial symbols into lexical units (Tokens) of preset standard atomic symbols; The lexical unit Token is input into the parser, and the lexical unit Token is filled into the preset grammar rules in the parser to generate an abstract syntax tree (AST).
5. The distributed task orchestration method based on ANTLR and DSL according to claim 1, characterized in that, Read the syntax nodes and semantic relationships between them from the Abstract Syntax Tree (AST); The syntax nodes are converted into predefined task node objects. Based on semantic relationships and the predefined task node objects, a directed acyclic graph (DAG) for the task is constructed, specifically including: Traverse the AST to obtain all syntax nodes and their nested semantic relationships. The syntax nodes include task declaration nodes, sequential flow connection nodes, parallel flow branching nodes, parallel flow converging nodes, conditional branch judgment nodes, conditional branch merging nodes, loop entry nodes, and loop return nodes. The nested semantic relationships between syntax nodes are directional semantic relationships between nodes. Syntax nodes are mapped to a unified task node object model of DAG; edge connections of DAG are constructed based on the directional semantic relationships between syntax nodes.
6. A distributed task orchestration system based on ANTLR and DSL, characterized in that, The system includes: The acquisition module is used to acquire the DSL syntax rules involved in all tasks in the current distributed task orchestration scenario; The generation module is used to generate lexical rules and grammar rules based on DSL grammar rules; and to generate lexical analyzers corresponding to lexical rules and grammar analyzers corresponding to grammar rules using the ANTLR tool. The syntax tree module is used to obtain the actual DSL script file, input the actual DSL script file into the lexical analyzer to obtain lexical unit tokens, and input the lexical unit tokens into the syntax analyzer to obtain the abstract syntax tree (AST). The execution module is used to read syntax nodes and semantic relationships between syntax nodes from the abstract syntax tree (AST); convert syntax nodes into preset task node objects; construct a directed acyclic graph (DAG) of tasks based on semantic relationships and preset task node objects; load the task DAG of tasks through the task execution engine; and execute the preset task node objects based on the semantic relationships of the task DAG of tasks.
7. The distributed task orchestration system based on ANTLR and DSL according to claim 6, characterized in that, The generation module includes generation units. Lexical rules for generating regular expression patterns by reading atomic symbols in task declarations, sequential streams, parallel streams, conditional branches, and loop structures based on DSL syntax rules; wherein atomic symbols include: task identifiers, parameter binders, control keywords, delimiters, and literals; Construct syntax rules for recursively defining sequential flow, parallel flow, conditional branch, and loop structures, with the starting task in the task declaration as the root node; among them, the sequential flow, parallel flow, conditional branch, and loop structures define intermediate and terminating tasks in the task declaration.
8. The distributed task orchestration system based on ANTLR and DSL according to claim 6, characterized in that, The syntax tree module includes syntax tree units. Used to obtain the actual DSL script file through a preset data acquisition interface; Input the actual DSL script file into the lexical analyzer to obtain the initial symbols; Transform the initial symbols into lexical units (Tokens) of preset standard atomic symbols; The lexical unit Token is input into the parser, and the lexical unit Token is filled into the preset grammar rules in the parser to generate an abstract syntax tree (AST).
9. A distributed task orchestration device based on ANTLR and DSL, characterized in that, The device includes: processor; And a memory storing executable code, which, when executed, causes the processor to perform a distributed task orchestration method based on ANTLR and DSL as described in any one of claims 1-5.
10. A non-volatile computer storage medium, characterized in that, It stores computer instructions, which, when executed, implement a distributed task orchestration method based on ANTLR and DSL as described in any one of claims 1-5.