Task blood relationship map generation method and device

Abstract

The invention discloses a task blood relationship map generation method and device. The method comprises the steps of obtaining scheduling task dependency description information of multiple workflows from a database; analyzing the scheduling task dependency description information of the plurality of workflows, and inserting the analyzed scheduling task dependency description information into a database table to obtain a dependency relationship table; determining a target scheduling task which is not depended on in the dependency table; searching a historical dependency record of the target scheduling task which is not depended on, and determining a workflow which depends on the target scheduling task; inserting a workflow depending on the target scheduling task into the dependency table; according to the task blood relationship diagram generation method and device, the dependency relationship among the scheduling tasks of all the workflows is generated by analyzing the dependency relationship of the scheduling tasks of each workflow, the complete dependency relationship among the scheduling tasks is accurately and efficiently determined, and the task blood relationship diagram generation efficiency and accuracy are improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method and device for generating a task lineage graph. Background Art

[0002] This section is intended to provide a background or context to the embodiments of the invention recited in the claims. No admission is made that the description herein is prior art by inclusion in this section.

[0003] Currently, there are many complex dependencies between the scheduling tasks in the scheduling system. In the workflow configuration interface, you can only see the dependencies of the scheduling tasks in the current workflow and the external workflow information that the scheduling tasks depend on. You cannot see the dependencies between external workflows, and cannot generate a complete task lineage graph. Summary of the invention

[0004] The embodiment of the present invention provides a method for generating a task lineage graph, which is used to accurately and efficiently generate a complete task lineage graph. The method includes:

[0005] Obtaining scheduling task dependency description information of multiple workflows from a database; the scheduling task dependency description information is used to describe the dependency relationship between each scheduling task or between a scheduling task and a workflow;

[0006] Parse the scheduling task dependency description information of multiple workflows, insert the parsed scheduling task dependency description information into the database table, and obtain a dependency relationship table;

[0007] Determine the target scheduling tasks that are not dependent on the dependency table;

[0008] Find the historical dependency records of the target scheduling tasks that are not dependent on it, and determine the workflow that depends on the target scheduling tasks;

[0009] Insert the workflow that depends on the target scheduling task into the dependency table;

[0010] Generate a task lineage graph based on the dependency table.

[0011] The embodiment of the present invention further provides a task lineage graph generation device for accurately and efficiently generating a complete task lineage graph, the device comprising:

[0012] The target scheduling task determination module is used to obtain the scheduling task dependency description information of multiple workflows from the database; the scheduling task dependency description information is used to describe the dependency relationship between each scheduling task or between the scheduling task and the workflow; parse the scheduling task dependency description information of multiple workflows, insert the parsed scheduling task dependency description information into the database table, and obtain a dependency table; determine the target scheduling task that is not dependent in the dependency table;

[0013] The task lineage graph generation module is used to find the historical dependency records of the non-dependent target scheduling tasks, determine the workflows that depend on the target scheduling tasks; insert the workflows that depend on the target scheduling tasks into the dependency table; and generate the task lineage graph based on the dependency table.

[0014] In an embodiment of the present invention, scheduling task dependency description information of multiple workflows is obtained from a database; the scheduling task dependency description information is used to describe the dependency relationship between each scheduling task or between a scheduling task and a workflow; the scheduling task dependency description information of multiple workflows is parsed, and the parsed scheduling task dependency description information is inserted into a database table to obtain a dependency table; the target scheduling task that is not dependent on the dependency table is determined; the historical dependency records of the target scheduling task that is not dependent on are searched to determine the workflow that is dependent on the target scheduling task; the workflow that is dependent on the target scheduling task is inserted into the dependency table; and a task lineage graph is generated according to the dependency table. Compared with the prior art, the target scheduling task that is not dependent on the dependency table is determined by parsing the dependency relationship of the scheduling task of each workflow, and the dependency relationship between the complete scheduling tasks is accurately and efficiently determined, thereby improving the efficiency and accuracy of task lineage graph generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0016] Figure 1 A schematic diagram of the process of generating a task lineage graph provided by the present invention;

[0017] Figure 2 A schematic diagram of the process of generating a task lineage graph provided by the present invention;

[0018] Figure 3 A schematic diagram of the process of generating a task lineage graph provided by the present invention;

[0019] Figure 4 A schematic diagram of the process of generating a task lineage graph provided by the present invention;

[0020] Figure 5 A schematic diagram of the structure of the task blood relationship graph generating device provided by the present invention. DETAILED DESCRIPTION

[0021] To make the purpose, technical solution and advantages of the embodiments of the present invention more clear, the embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings. Here, the exemplary embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0022] Figure 1 The flowchart corresponding to the task lineage graph generation method provided in the embodiment of the present invention is as follows: Figure 1 As shown, the method includes:

[0023] Step 101: Obtain scheduling task dependency description information of multiple workflows from a database.

[0024] It should be noted that the scheduling task dependency description information is used to describe the dependency relationship between various scheduling tasks or between a scheduling task and a workflow.

[0025] In a possible implementation, the scheduling t_ds_process_definition uses json to describe dependencies.

[0026] Step 102: parsing the scheduling task dependency description information of multiple workflows, inserting the parsed scheduling task dependency description information into a database table to obtain a dependency relationship table.

[0027] In a possible implementation, the JSON statement of each workflow is parsed, and the dependencies between all tasks in the workflow and external dependencies are obtained and inserted into the dependency table.

[0028] For example, the dependency table is shown in Table 1:

[0029] Table 1

[0030]

[0031] Step 103: determine the target scheduling task that is not dependent in the dependency table.

[0032] Step 104 , searching for historical dependency records of non-dependent target scheduling tasks, and determining workflows that are dependent on the target scheduling tasks.

[0033] Step 105: insert the workflow that depends on the target scheduling task into the dependency table.

[0034] Step 106, generating a task lineage graph according to the dependency table.

[0035] In the embodiment of the present invention, the dependency table is read and a task lineage graph is generated according to the dependency relationships.

[0036] The above solution generates the dependency relationships between tasks of all workflows by parsing the dependency relationships of the scheduling tasks of each workflow, accurately and efficiently obtains the complete dependency relationships between scheduling tasks, and generates a complete task lineage graph, which facilitates the acquisition of the complete link of dependent tasks.

[0037] In the embodiment of the present invention, the target scheduling task that is not dependent in the dependency table is determined, and the steps are as follows: Figure 2 As shown, the details are as follows:

[0038] Step 201: determine the first scheduled task identifier set in the dependency table.

[0039] It should be noted that the first scheduling task identifier set includes the scheduling task identifiers of the scheduling tasks on which each scheduling task depends.

[0040] Step 202 : Compare the second scheduled task identification set of all scheduled tasks in multiple workflows with the first scheduled task identification set to obtain the scheduled task identification of the non-dependent target scheduled task.

[0041] Step 203: Determine the non-dependent target scheduling task according to the scheduling task identifier of the non-dependent target scheduling task.

[0042] The above scheme improves the efficiency of task lineage graph generation by comparing the second scheduling task identification set of all scheduling tasks in multiple workflows with the first scheduling task identification set to determine the non-dependent target scheduling tasks.

[0043] In the embodiment of the present invention, the workflow that depends on the target scheduling task is inserted into the dependency table, and the steps are as follows: Figure 3 As shown, the details are as follows:

[0044] Step 301: Generate a virtual scheduling task node according to a workflow that depends on a target scheduling task.

[0045] Step 302: insert the scheduling task identifier of the virtual scheduling task node into the dependency table.

[0046] In order to show the dependency relationship more clearly, the above solution treats the workflow as a virtual task node to reduce the complexity of the dependency relationship.

[0047] After searching for the historical dependency records of the target scheduling tasks that are not dependent, the steps of the embodiment of the present invention are as follows: Figure 4 As shown, the details are as follows:

[0048] Step 401, if there is no workflow that depends on the target scheduling task in the historical dependency record of the non-dependent target scheduling task;

[0049] Step 402: determine the workflow to which the target scheduling task belongs.

[0050] Step 403: insert the workflow to which the target scheduling task belongs into the dependency table.

[0051] The above solution generates the dependency relationships between tasks of all workflows by parsing the dependency relationships of the scheduling tasks of each workflow, accurately and efficiently obtains the complete dependency relationships between scheduling tasks, and generates a complete task lineage graph, which facilitates the acquisition of the complete link of dependent tasks.

[0052] The present invention also provides a server traffic bandwidth prediction device based on cloud monitoring, as described in the following embodiments. Figure 5 As shown, the device comprises:

[0053] The target scheduling task determination module 501 is used to obtain the scheduling task dependency description information of multiple workflows from the database; the scheduling task dependency description information is used to describe the dependency relationship between each scheduling task or between the scheduling task and the workflow; parse the scheduling task dependency description information of multiple workflows, insert the parsed scheduling task dependency description information into the database table, and obtain a dependency table; determine the target scheduling task that is not dependent in the dependency table;

[0054] The task lineage graph generation module 502 is used to find the historical dependency records of the non-dependent target scheduling tasks, determine the workflow that depends on the target scheduling tasks; insert the workflow that depends on the target scheduling tasks into the dependency table; and generate a task lineage graph based on the dependency table.

[0055] In the embodiment of the present invention, the target scheduling task determination module 501 is specifically used for:

[0056] Determine a first scheduling task identifier set in the dependency table; the first scheduling task identifier set includes scheduling task identifiers of scheduling tasks that each scheduling task depends on;

[0057] Compare the second scheduling task identification set of all scheduling tasks in the multiple workflows with the first scheduling task identification set to obtain the scheduling task identification of the non-dependent target scheduling task;

[0058] The non-dependent target scheduling task is determined according to the scheduling task identifier of the non-dependent target scheduling task.

[0059] In the embodiment of the present invention, the task lineage graph generation module 502 is specifically used for:

[0060] Generate a virtual scheduling task node according to the workflow that depends on the target scheduling task;

[0061] Insert the scheduling task ID of the virtual scheduling task node into the dependency table.

[0062] In the embodiment of the present invention, the task lineage graph generation module 502 is further used for:

[0063] After searching for the historical dependency records of the non-dependent target scheduling task, if there is no workflow that depends on the target scheduling task in the historical dependency records of the non-dependent target scheduling task;

[0064] Determine the workflow to which the target scheduling task belongs;

[0065] Insert the workflow to which the target scheduling task belongs into the dependency table.

[0066] Since the principle of solving the problem by the device is similar to that of the task bloodline graph generation method, the implementation of the device can refer to the implementation of the task bloodline graph generation method, and the repeated parts will not be repeated.

[0067] An embodiment of the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned task lineage graph generation method when executing the computer program.

[0068] An embodiment of the present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned task bloodline graph generation method is implemented.

[0069] An embodiment of the present invention also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the above-mentioned task lineage graph generation method is implemented.

[0070] In an embodiment of the present invention, scheduling task dependency description information of multiple workflows is obtained from a database; the scheduling task dependency description information is used to describe the dependency relationship between each scheduling task or between a scheduling task and a workflow; the scheduling task dependency description information of multiple workflows is parsed, and the parsed scheduling task dependency description information is inserted into a database table to obtain a dependency table; a target scheduling task that is not dependent on the dependency table is determined; historical dependency records of the target scheduling task that is not dependent on are searched to determine the workflow that depends on the target scheduling task; the workflow that depends on the target scheduling task is inserted into the dependency table; a task lineage graph is generated according to the dependency table. Compared with the prior art, by parsing the dependency relationship of the scheduling task of each workflow, the dependency relationship between the scheduling tasks of all workflows is generated, the dependency relationship between the complete scheduling tasks is accurately and efficiently determined, and the efficiency and accuracy of task lineage graph generation are improved.

[0071] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0072] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0073] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0074] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0075] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for generating a task lineage graph, characterized in that: include: Obtain the scheduling task dependency description information of multiple workflows from the database; The scheduling task dependency description information is used to describe the dependency relationship between various scheduling tasks or between scheduling tasks and workflows; Parse the scheduling task dependency description information of multiple workflows, insert the parsed scheduling task dependency description information into the database table, and obtain a dependency relationship table; Determine the target scheduling tasks that are not dependent on the dependency table; Find the historical dependency records of the target scheduling tasks that are not dependent on it, and determine the workflow that depends on the target scheduling tasks; Insert the workflow that depends on the target scheduling task into the dependency table; Generate a task lineage graph based on the dependency table.

2. The task lineage graph generation method according to claim 1, characterized in that: Determine the target scheduling tasks that are not dependent on the dependency table, including: Determine a first scheduling task identifier set in the dependency table; the first scheduling task identifier set includes scheduling task identifiers of scheduling tasks that each scheduling task depends on; Compare the second scheduling task identification set of all scheduling tasks in the multiple workflows with the first scheduling task identification set to obtain the scheduling task identification of the non-dependent target scheduling task; The non-dependent target scheduling task is determined according to the scheduling task identifier of the non-dependent target scheduling task.

3. The task lineage graph generation method according to claim 1, characterized in that: Insert the workflow that depends on the target scheduling task into the dependency table, including: Generate a virtual scheduling task node according to the workflow that depends on the target scheduling task; Insert the scheduling task ID of the virtual scheduling task node into the dependency table.

4. The task lineage graph generation method according to claim 1, characterized in that: After finding the historical dependency records of the target scheduling tasks that are not dependent, it also includes: If there is no workflow that depends on the target scheduling task in the historical dependency record of the non-dependent target scheduling task; Determine the workflow to which the target scheduling task belongs; Insert the workflow to which the target scheduling task belongs into the dependency table.

5. A task lineage graph generation device, characterized in that: include: The target scheduling task determination module is used to obtain the scheduling task dependency description information of multiple workflows from the database; The scheduling task dependency description information is used to describe the dependency relationship between various scheduling tasks or between scheduling tasks and workflows; parsing the scheduling task dependency description information of multiple workflows, inserting the parsed scheduling task dependency description information into a database table, and obtaining a dependency table; Determine the target scheduling tasks that are not dependent on the dependency table; The task lineage graph generation module is used to find the historical dependency records of the target scheduling tasks that are not dependent on it, and determine the workflow of the dependent target scheduling tasks; Insert the workflow that depends on the target scheduling task into the dependency table; Generate a task lineage graph based on the dependency table.

6. The task kinship graph generating device according to claim 5, characterized in that: The target scheduling task determination module is specifically used for: Determine a first scheduling task identifier set in the dependency table; the first scheduling task identifier set includes scheduling task identifiers of scheduling tasks that each scheduling task depends on; Compare the second scheduling task identification set of all scheduling tasks in the multiple workflows with the first scheduling task identification set to obtain the scheduling task identification of the non-dependent target scheduling task; The non-dependent target scheduling task is determined according to the scheduling task identifier of the non-dependent target scheduling task.

7. The task kinship graph generating device according to claim 5, characterized in that: The task lineage graph generation module is specifically used for: Generate a virtual scheduling task node according to the workflow that depends on the target scheduling task; Insert the scheduling task ID of the virtual scheduling task node into the dependency table.

8. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 4 is implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images