Scheduling task execution information acquisition method and device

Abstract

The invention discloses a scheduling task execution information obtaining method and device, and relates to the technical field of computers, and the method comprises the following steps: obtaining a dolphin scheduling log; according to the dolphin scheduling log, a first table is generated, and the first table stores a corresponding relation between a first identifier of a dolphin scheduling first task and a second identifier of a second task in a task calculation system; obtaining execution information of a plurality of second tasks in the task calculation system; the execution information comprises a second identifier of a second task in the task computing system; generating a second table according to execution information of a plurality of second tasks in the task calculation system; the second table stores second identifiers of a plurality of second tasks and execution information of the corresponding second tasks; and according to the first table and the second table, determining execution information of the second task in the task calculation system corresponding to the dolphin scheduling the first task. According to the method, the search accuracy, efficiency and comprehensiveness of the scheduling task information can be improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method and device for acquiring scheduling task execution information. Background Art

[0002] This section is intended to provide a background or context to the embodiments of the invention recited in the claims. No admission is made that the description herein is prior art by inclusion in this section.

[0003] In the prior art, after submitting scheduling tasks such as hive, sqoop, spark, and flink through the scheduling service, these scheduling tasks will be executed in the task computing system, such as yarn. Usually, after the scheduling task is executed, we need to view the specific execution information of the specific scheduling task submitted by the scheduling service in the task computing system, such as yarn task resource usage, yarn task running status and other detailed information, so as to evaluate the resource consumption of the scheduling task execution, provide data for optimizing the scheduling task script, and reasonably arrange the task scheduling time.

[0004] In the prior art, in order to obtain the specific execution information of a task in a specific scheduling service in a task computing system, it is necessary to search for information from a large number of task execution logs in the task computing system. For example, for yarn, it is necessary to log in to the yarn web console page, search the logs of yarn tasks one by one, extract keywords from the logs and compare them with the scheduling tasks, find the corresponding relationship between the two, and thus obtain the execution information of the specific scheduling task. However, this manual search method is inefficient, and it is easy to make mistakes by searching through a large amount of log information based on keywords, and the accuracy and efficiency are very low. At the same time, the log retention time of most task computing systems in the prior art is short, and it is easy to find the corresponding task in the task computing system for the historical scheduling task. Summary of the invention

[0005] An embodiment of the present invention provides a method for obtaining execution information of a scheduling task, which is used to obtain execution information of a task in a task computing system corresponding to a first task in a scheduling service, and improve the accuracy and efficiency of searching the execution information of the scheduling task, and the method includes:

[0006] After submitting a plurality of Dolphin scheduling first tasks to the task computing system based on the Dolphin scheduling service, obtaining a Dolphin scheduling log; the Dolphin scheduling log includes a first identifier of the Dolphin scheduling first task and a second identifier of the second task in the task computing system;

[0007] Generate a first table according to the Dolphin scheduling log, wherein the first table stores a correspondence between a first identifier of a first task in the Dolphin scheduling and a second identifier of a second task in the task computing system;

[0008] Acquire execution information of a plurality of second tasks in the task computing system; the execution information includes a second identifier of the second task in the task computing system;

[0009] Generate a second table according to the execution information of the plurality of second tasks in the task computing system; the second table stores the second identifiers of the plurality of second tasks and the corresponding execution information of the second tasks;

[0010] According to the first table and the second table, the execution information of the second task in the task computing system corresponding to the first task of Dolphin Scheduling is determined.

[0011] The embodiment of the present invention further provides a scheduling task execution information acquisition device, which is used to obtain the execution information of the task in the task computing system corresponding to the first task in the scheduling service, and improve the search accuracy and efficiency of the scheduling task execution information. The device includes:

[0012] A scheduling log acquisition module, used to obtain a Dolphin scheduling log after submitting multiple Dolphin scheduling first tasks to the task computing system based on the Dolphin scheduling service; the Dolphin scheduling log includes a first identifier of the Dolphin scheduling first task and a second identifier of the second task in the task computing system;

[0013] A first table generating module, used to generate a first table according to the Dolphin scheduling log, wherein the first table stores a correspondence between a first identifier of a first task of the Dolphin scheduling and a second identifier of a second task in the task computing system;

[0014] A scheduling task execution information API acquisition module is used to acquire execution information of multiple second tasks in the task computing system; the execution information includes a second identifier of the second task in the task computing system;

[0015] A second table generating module, configured to generate a second table according to the execution information of a plurality of second tasks in the task computing system; the second table stores the second identifiers of the plurality of second tasks and the corresponding execution information of the second tasks;

[0016] The table association module is used to determine the execution information of the second task in the task computing system corresponding to the first task of Dolphin Scheduling according to the first table and the second table.

[0017] An embodiment of the present invention further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned method for acquiring execution information of scheduled tasks when executing the computer program.

[0018] An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned method for acquiring execution information of a scheduling task is implemented.

[0019] An embodiment of the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the method for acquiring the execution information of the scheduling task is implemented.

[0020] In the embodiment of the present invention, a first table is generated through the Dolphin scheduling log, and the first table stores the corresponding relationship between the first identifier of the first task of Dolphin scheduling and the second identifier of the second task in the task computing system, so that the corresponding relationship between the first task of Dolphin scheduling and the second task in the task computing system can be confirmed; secondly, the execution information of multiple second tasks in the task computing system is directly obtained, and the obtained execution information of the second tasks is comprehensive, which improves the comprehensiveness of the search of subsequent scheduling task execution information, and then a second table is generated according to the obtained execution information of the second task, and the second table stores the second identifiers of multiple second tasks and the corresponding execution information of the second tasks; thus, the execution information of the second task in the task computing system corresponding to the first task of Dolphin scheduling can be obtained by associating the first table and the second table, and the acquisition of scheduling task execution information is realized. Compared with the prior art, the search accuracy, efficiency and comprehensiveness of the execution information of the scheduling task are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0022] Figure 1 A schematic diagram of a process for acquiring scheduling task execution information in an embodiment of the present invention;

[0023] Figure 2 A specific example diagram of a method for acquiring scheduling task execution information in an embodiment of the present invention;

[0024] Figure 3 A specific example diagram of a method for acquiring scheduling task execution information in an embodiment of the present invention;

[0025] Figure 4 A specific example diagram of a method for acquiring scheduling task execution information in an embodiment of the present invention;

[0026] Figure 5Schematic diagram of a device for acquiring scheduling task execution information in an embodiment of the present invention. DETAILED DESCRIPTION

[0027] To make the purpose, technical solution and advantages of the embodiments of the present invention more clear, the embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings. Here, the exemplary embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0028] First, technical terms involved in the embodiments of the present invention are explained.

[0029] Task computing system: equivalent to a resource scheduling platform, providing server computing resources, applications and distributed systems for computing programs, integrating cluster resources to maximize resources.

[0030] Yarn: Hadoop's computing framework, a typical task computing system. Yarn is mainly composed of ResourceManager, NodeManager, ApplicationMaster, and Container components. Each component has different functions. ResourceManager is responsible for receiving jobs and implementing job task scheduling.

[0031] Hadoop: is an open source big data processing framework for efficiently storing and processing large data sets ranging from GB to PB.

[0032] Dolphin Scheduling: An open source scheduling product, it is a distributed and easily extensible visual DAG workflow task scheduling open source system. It is suitable for enterprise-level scenarios and provides a solution for visual operation tasks, workflows, and full life cycle data processing.

[0033] The applicant found that in the prior art, in order to obtain the specific execution information of a task in a specific scheduling service in Yarn, it is necessary to search the logs of the Yarn tasks one by one, extract keywords from the logs and compare them with the tasks in the scheduling service to find the corresponding relationship between the two, so as to obtain the scheduling task execution information. This has low accuracy and efficiency. To this end, the applicant proposed a method for obtaining scheduling task execution information.

[0034] It should be noted that the acquisition, storage, use, and processing of data in the technical solution of this application are in compliance with the relevant provisions of national laws and regulations.

[0035] Figure 1 FIG. 1 is a flow chart of a method for acquiring execution information of a scheduling task in an embodiment of the present invention. Figure 1 As shown, the method includes:

[0036] Step 101: after submitting multiple Dolphin Scheduling first tasks to the task computing system based on the Dolphin Scheduling Service, obtaining a Dolphin Scheduling log; the Dolphin Scheduling log includes a first identifier of the Dolphin Scheduling first task and a second identifier of the second task in the task computing system;

[0037] Step 102: Generate a first table according to the Dolphin scheduling log, wherein the first table stores a correspondence between a first identifier of a first task in Dolphin scheduling and a second identifier of a second task in the task computing system;

[0038] Step 103: Acquire execution information of a plurality of second tasks in the task computing system; the execution information includes a second identifier of the second task in the task computing system;

[0039] Step 104: Generate a second table according to the execution information of the plurality of second tasks in the task computing system; the second table stores the second identifiers of the plurality of second tasks and the corresponding execution information of the second tasks;

[0040] Step 105: Determine the execution information of the second task in the task computing system corresponding to the first task of Dolphin Scheduling according to the first table and the second table.

[0041] from Figure 1 It can be seen from the process shown that in the embodiment of the present invention, a first table is generated through the Dolphin Scheduling Log, and the first table stores the correspondence between the first identifier of the first task of Dolphin Scheduling and the second identifier of the second task in the task computing system, so that the correspondence between the first task of Dolphin Scheduling and the second task in the task computing system can be confirmed; secondly, the execution information of multiple second tasks in the task computing system is directly obtained, and the obtained execution information of the second tasks is comprehensive. A second table is generated according to the obtained execution information of the second tasks, and the second table stores the second identifiers of multiple second tasks and the corresponding execution information of the second tasks; thus, the execution information of the second task in the task computing system corresponding to the first task of Dolphin Scheduling can be obtained by associating the first table with the second table, thereby realizing the acquisition of the execution information of the scheduling task. Compared with the prior art, the search accuracy, efficiency and comprehensiveness of the execution information of the scheduling task are improved.

[0042] In the specific implementation, first submit hive, sqoop, spark, flink and other tasks to the task computing system, such as Yarn, based on the Dolphin scheduling service. All tasks will be executed in Yarn, and Dolphin scheduling logs will be generated.

[0043] In step 101, after submitting multiple Dolphin Scheduling first tasks to the task computing system based on the Dolphin Scheduling service, a Dolphin Scheduling log is obtained; the Dolphin Scheduling log includes a first identifier of the Dolphin Scheduling first task and a second identifier of the second task in the task computing system.

[0044] For example, the Dolphin Scheduling log will store the first task ID of Dolphin Scheduling and the application ID of the second task in Yarn.

[0045] In step 102, a first table is generated according to the Dolphin scheduling log, wherein the first table stores a correspondence between a first identifier of a first task of Dolphin scheduling and a second identifier of a second task in a task computing system.

[0046] For example, the first task ID of Dolphin Scheduling and the application ID of the second task in Yarn are obtained and saved in a table to form a first table.

[0047] In one embodiment, generating a first table according to the Dolphin scheduling log may include:

[0048] Scan the Dolphin scheduling log, and filter out the first identifiers of multiple Dolphin scheduling first tasks and the second identifiers of the second tasks in the corresponding task computing system;

[0049] A second table is generated according to the first identifiers of the second tasks of the plurality of Dolphin Schedulers and the second identifiers of the second tasks in the corresponding task computing systems.

[0050] For example, use the grep command to scan the Dolphin Scheduling log, obtain the ID of the first task of Dolphin Scheduling in the Dolphin Scheduling log, and the application ID string of the corresponding yarn task, to form a second table. At the same time, the Dolphin Scheduling log can also include the project ID, workflow definition ID, workflow instance ID, and the application ID string of the corresponding yarn task of the first task of Dolphin Scheduling, which can be stored in the second table together. That is, there can be multiple identification information about the first task of Dolphin Scheduling in the Dolphin Scheduling log.

[0051] As shown in the following Dolphin scheduling log sample content, / 11 / 2584 / 2332658 / 11808646, where 11 is the project ID, 2584 is the workflow definition ID, 2332658 is the workflow instance ID, 11808646 is the first task ID of Dolphin scheduling, and application_1693219184701_101117 is the second identifier of the first task of Dolphin scheduling when Yarn runs this time.

[0052]

[0053] Then, in step 103, execution information of a plurality of second tasks in the task computing system is obtained; the execution information includes a second identifier of the second task in the task computing system.

[0054] For example, when Yarn executes a task, the execution information of each task is obtained in real time. Each execution information carries a Yarn task identifier, such as the application ID string of the Yarn task.

[0055] In one embodiment, obtaining execution information of a plurality of second tasks in the task computing system may include:

[0056] The execution information of the plurality of second tasks in the task computing system is periodically obtained by calling an API interface for obtaining the execution information of the second task in the task computing system.

[0057] During implementation, for example, calling the API interface of Yarn to obtain task information and periodically obtaining the task execution information in Yarn, the task execution information can be temporarily stored in the memory.

[0058] In one embodiment, the execution information includes but is not limited to one or any combination of the following:

[0059] Task running status, task execution progress at different times, task execution priority, task type, task execution start time, task execution end time, and task execution resource consumption information.

[0060] It can be seen that the information related to the scheduling task obtained in this example is relatively large and comprehensive, and technicians in this field can obtain the information they want according to actual needs.

[0061] In step 104, a second table is generated according to the execution information of the plurality of second tasks in the task computing system; the second table stores the second identifiers of the plurality of second tasks and the corresponding execution information of the second tasks.

[0062] During implementation, the second table is formed by using the obtained second identifiers of the plurality of second tasks and the corresponding execution information of the second tasks.

[0063] See also Figure 2 , Figure 3 , Figure 2 FIG. 1 is a specific example diagram of a method for acquiring execution information of a scheduling task in an embodiment of the present invention. Figure 2 A schematic diagram of the table structure of the first table is shown. Figure 3 FIG. 1 is a specific example diagram of a method for acquiring execution information of a scheduling task in an embodiment of the present invention. Figure 3 A schematic diagram of the table structure of the second table is shown.

[0064] In one embodiment, determining the execution information of the second task in the task computing system corresponding to the first task of Dolphin Scheduling according to the first table and the second table may include:

[0065] Obtain the second identifier corresponding to the first identifier of the first task of Dolphin Scheduling in the first table through a SQL statement query;

[0066] The second identifier corresponding to the first identifier of the first task of Dolphin Scheduling in the first table is obtained according to the query, and the execution information of the second task in the task computing system corresponding to the first task of Dolphin Scheduling is determined from the second table.

[0067] For example, by querying the application ID in the first table and associating it with the application ID in the second table through an SQL statement, the Yarn task execution information corresponding to the scheduled task can be accurately obtained.

[0068] In one embodiment, after generating the second table according to the execution information of the plurality of second tasks in the task computing system, the following may also be included:

[0069] The first table and the second table are stored in the database periodically.

[0070] For example, the first table and the second table are stored in the database every hour, or other hours or minutes, so as to avoid information loss caused by regular clearing of Yarn task logs, and historical scheduling task information can be obtained at any time.

[0071] Figure 4 FIG. 1 is a specific example diagram of a method for acquiring execution information of a scheduling task in an embodiment of the present invention. Figure 4 As shown, two tables can be established in the database in advance, one table stores the correspondence between the scheduling task ID and the yarn task ID, for example, called table A, and the other table stores the yarn task information, for example, called table B. The two tables obtain information in parallel, and finally associate table A with table B, and use the shell script command to obtain the relationship between the scheduling task and the Yarn task, thereby obtaining the execution information of the scheduling task.

[0072] In summary, the embodiments of the present invention have the following beneficial effects:

[0073] 1. Use shell script commands to automatically obtain the corresponding relationship between the first task ID of the scheduling and the second task ID in the task calculation system, with high search efficiency;

[0074] 2. By scanning the corresponding relationship between the first task ID in the scheduling task log and the second task ID in the task calculation system, the accuracy rate is higher than manual search;

[0075] 3. Regularly save the execution information of the second task in the task calculation system to the database to avoid the problem that the task calculation system automatically deletes historical task information and cannot find the historical scheduling task information.

[0076] The present invention also provides a device for obtaining scheduling task execution information, as described in the following embodiments. Since the principle of the device to solve the problem is similar to that of the method for obtaining scheduling task execution information, the implementation of the device can refer to the implementation of the method for obtaining scheduling task execution information, and the repeated parts will not be repeated.

[0077] Figure 5 FIG. 1 is a schematic diagram of a device for acquiring information on scheduling task execution in an embodiment of the present invention. Figure 5 As shown, the device comprises:

[0078] The scheduling log acquisition module 501 is used to acquire the Dolphin scheduling log after submitting multiple Dolphin scheduling first tasks to the task computing system based on the Dolphin scheduling service; the Dolphin scheduling log includes a first identifier of the Dolphin scheduling first task and a second identifier of the second task in the task computing system;

[0079] A first table generating module 502 is used to generate a first table according to the Dolphin scheduling log, wherein the first table stores a correspondence between a first identifier of a first task of the Dolphin scheduling and a second identifier of a second task in the task computing system;

[0080] The scheduling task execution information API acquisition module 503 is used to acquire the execution information of multiple second tasks in the task computing system; the execution information includes the second identifier of the second task in the task computing system;

[0081] A second table generating module 504 is used to generate a second table according to the execution information of the plurality of second tasks in the task computing system; the second table stores the second identifiers of the plurality of second tasks and the corresponding execution information of the second tasks;

[0082] The table association module 505 is used to determine the execution information of the second task in the task computing system corresponding to the first task of Dolphin Scheduling according to the first table and the second table.

[0083] An embodiment of the present invention further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned method for acquiring execution information of scheduled tasks when executing the computer program.

[0084] An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned method for acquiring execution information of a scheduling task is implemented.

[0085] An embodiment of the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the method for acquiring the execution information of the scheduling task is implemented.

[0086] In the embodiment of the present invention, a first table is generated through the Dolphin scheduling log, and the first table stores the corresponding relationship between the first identifier of the first task of Dolphin scheduling and the second identifier of the second task in the task computing system, so that the corresponding relationship between the first task of Dolphin scheduling and the second task in the task computing system can be confirmed; secondly, the execution information of multiple second tasks in the task computing system is directly obtained, and the obtained execution information of the second tasks is comprehensive, which improves the comprehensiveness of the search of subsequent scheduling task execution information, and then a second table is generated according to the obtained execution information of the second task, and the second table stores the second identifiers of multiple second tasks and the corresponding execution information of the second tasks; thus, the execution information of the second task in the task computing system corresponding to the first task of Dolphin scheduling can be obtained by associating the first table and the second table, and the acquisition of scheduling task execution information is realized. Compared with the prior art, the search accuracy, efficiency and comprehensiveness of the execution information of the scheduling task are improved.

[0087] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0088] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0089] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0090] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0091] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for obtaining scheduling task execution information, characterized in that: include: After submitting multiple Dolphin scheduling first tasks to the task computing system based on the Dolphin scheduling service, obtain the Dolphin scheduling log; The Dolphin Scheduling log includes a first identifier of a first task in the Dolphin Scheduling system and a second identifier of a second task in the task computing system; Generate a first table according to the Dolphin scheduling log, wherein the first table stores a correspondence between a first identifier of a first task in the Dolphin scheduling and a second identifier of a second task in the task computing system; Acquire execution information of a plurality of second tasks in the task computing system; the execution information includes a second identifier of the second task in the task computing system; Generate a second table according to the execution information of the plurality of second tasks in the task computing system; the second table stores the second identifiers of the plurality of second tasks and the corresponding execution information of the second tasks; According to the first table and the second table, the execution information of the second task in the task computing system corresponding to the first task of Dolphin Scheduling is determined.

2. The method according to claim 1, characterized in that According to the Dolphin scheduling log, the first table is generated, including: Scan the Dolphin scheduling log, and filter out the first identifiers of multiple Dolphin scheduling first tasks and the second identifiers of the second tasks in the corresponding task computing system; A second table is generated according to the first identifiers of the second tasks of the plurality of Dolphin Schedulers and the second identifiers of the second tasks in the corresponding task computing systems.

3. The method according to claim 1, characterized in that Acquiring execution information of a plurality of second tasks in the task computing system, including: The execution information of the plurality of second tasks in the task computing system is periodically obtained by calling an API interface for obtaining the execution information of the second task in the task computing system.

4. The method according to claim 1, characterized in that Determining the execution information of the second task in the task computing system corresponding to the first task of Dolphin Scheduling according to the first table and the second table includes: Obtain the second identifier corresponding to the first identifier of the first task of Dolphin Scheduling in the first table through a SQL statement query; The second identifier corresponding to the first identifier of the first task of Dolphin Scheduling in the first table is obtained according to the query, and the execution information of the second task in the task computing system corresponding to the first task of Dolphin Scheduling is determined from the second table.

5. The method according to claim 1, characterized in that The execution information includes one or any combination of the following: Task running status, task execution progress at different times, task execution priority, task type, task execution start time, task execution end time, and task execution resource consumption information.

6. The method according to claim 1, characterized in that After generating the second table according to the execution information of the plurality of second tasks in the task computing system, the method further includes: The first table and the second table are stored in the database periodically.

7. A scheduling task execution information acquisition device, characterized in that: include: A scheduling log acquisition module, used to obtain Dolphin scheduling logs after submitting multiple Dolphin scheduling first tasks to the task computing system based on the Dolphin scheduling service; The Dolphin Scheduling log includes a first identifier of a first task in the Dolphin Scheduling system and a second identifier of a second task in the task computing system; A first table generating module, used to generate a first table according to the Dolphin scheduling log, wherein the first table stores a correspondence between a first identifier of a first task of the Dolphin scheduling and a second identifier of a second task in the task computing system; A scheduling task execution information API acquisition module is used to acquire execution information of multiple second tasks in the task computing system; the execution information includes a second identifier of the second task in the task computing system; A second table generating module, configured to generate a second table according to the execution information of a plurality of second tasks in the task computing system; the second table stores the second identifiers of the plurality of second tasks and the corresponding execution information of the second tasks; The table association module is used to determine the execution information of the second task in the task computing system corresponding to the first task of Dolphin Scheduling according to the first table and the second table.

8. The device according to claim 7, characterized in that The first table generation module is specifically used for: Scan the Dolphin scheduling log, and filter out the first identifiers of multiple Dolphin scheduling first tasks and the second identifiers of the second tasks in the corresponding task computing system; A second table is generated according to the first identifiers of the second tasks of the plurality of Dolphin Schedulers and the second identifiers of the second tasks in the corresponding task computing systems.

9. The device according to claim 7, characterized in that The scheduling task execution information API acquisition module is specifically used for: The execution information of the plurality of second tasks in the task computing system is periodically obtained by calling an API interface for obtaining the execution information of the second task in the task computing system.

10. The device according to claim 7, characterized in that The table association module is specifically used for: Obtain the second identifier corresponding to the first identifier of the first task of Dolphin Scheduling in the first table through a SQL statement query; The second identifier corresponding to the first identifier of the first task of Dolphin Scheduling in the first table is obtained according to the query, and the execution information of the second task in the task computing system corresponding to the first task of Dolphin Scheduling is determined from the second table.

11. The device according to claim 7, characterized in that The execution information includes one or any combination of the following: Task running status, task execution progress at different times, task execution priority, task type, task execution start time, task execution end time, and task execution resource consumption information.

12. The device according to claim 7, characterized in that Also includes: The database processing module is used to store the first table and the second table in the database regularly after the second table generating module generates the second table according to the execution information of the plurality of second tasks in the task computing system.

13. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

15. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images