Task execution method and device, storage medium and electronic equipment

By storing task data in the cluster database and splitting tasks into subtasks, the task execution efficiency problem caused by the main node downtime is solved, efficient task execution and failure recovery are achieved, and task success rate is improved.

CN118964014APending Publication Date: 2024-11-15JINAN INSPUR DATA TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411017937.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-26
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

During cluster task execution, the main node downtime leads to low task execution efficiency, and the existing technology has failed to effectively solve this problem.

Method used

By storing task data in the database of the target cluster, the master node splits the task into a subtask when receiving task instructions and updates the task status during execution. The backup node can continue to execute unfinished subtasks based on the database data to ensure that the task is efficient execution.

Benefits of technology

Improve the efficiency of task execution, avoid task failure caused by master node failure, reduce manual intervention, and improve the possibility of task success.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118964014A_ABST
    Figure CN118964014A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides a task execution method and device, a storage medium and electronic device.The method comprises the steps that under the condition that it is determined that a main node included in a target cluster breaks down, task data of a target task stored in a database included in the target cluster is obtained, under the condition that the main node receives a task instruction used for indicating execution of a target task, the main node determines sub-tasks included in the target task, determines target data required for executing each sub-task, stores the target data in a database, executes each sub-task, and executes each sub-task under the condition that a first sub-task is executed; updating the execution state of the first subtask in the task data; determining a second sub-task in an unexecuted state included in the target task based on the task data; and executing the second subtask based on the task data. Through the method and the device, the problem of low task execution efficiency is solved, and the effect of improving the task execution efficiency is achieved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computers, and more specifically, to a method, device, storage medium, and electronic device for executing a task. Background Art

[0002] In the related art, when a cluster executes a task, it is usually executed in only one node. If the node goes down during the execution of the task, the corresponding task will fail to execute.

[0003] It can be seen from this that the related technology has the problem of low task execution efficiency.

[0004] Currently, no effective solution has been proposed for the above-mentioned problems existing in the related technologies. Summary of the invention

[0005] The embodiments of the present application provide a method, device, storage medium and electronic device for executing a task, so as to at least solve the problem of low task execution efficiency existing in the related art.

[0006] According to one embodiment of the present application, a method for executing a task is provided, comprising: in a case where it is determined that a master node included in a target cluster fails, obtaining task data of a target task stored in a database included in the target cluster, wherein the task data includes target data, and in a case where the master node receives a task instruction for instructing execution of the target task, determining subtasks included in the target task, determining target data required for executing each of the subtasks, storing the target data in the database, executing each of the subtasks, and in a case where a first subtask is completed, updating the execution status of the first subtask in the task data; determining a second subtask included in the target task that is in an unexecuted state based on the task data; and executing the second subtask based on the task data.

[0007] According to another embodiment of the present application, a task execution device is provided, including: an acquisition module, for acquiring task data of a target task stored in a database included in the target cluster when it is determined that a master node included in the target cluster has failed, wherein the task data includes target data, and the master node, when receiving a task instruction for instructing to execute the target task, determines the subtasks included in the target task, determines the target data required to execute each of the subtasks, stores the target data in the database, executes each of the subtasks, and when a first subtask is completed, updates the execution status of the first subtask in the task data; a determination module, for determining a second subtask in the target task that is in an unexecuted state based on the task data; and an execution module, for executing the second subtask based on the task data.

[0008] According to another embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when run.

[0009] According to another embodiment of the present application, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0010] According to another embodiment of the present application, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0011] Through the present application, when the master node in the target cluster receives a task instruction for executing a target task, it can determine the subtasks included in the target task, determine the target data required for executing each subtask, and store the target data in the database of the target cluster, execute each subtask, and when a first subtask is executed and completed, the execution status of the first subtask is rooted in the task data. When the master node fails, the task data stored in the database can be obtained, and the second subtask included in the target task that is in an unexecuted state can be determined based on the task data, and the second subtask is executed according to the task data. Since the task data required for executing each subtask included in the target task can be stored in the database of the target cluster when the master node executes the target task, when the master node fails, the unexecuted second subtask can be determined based on the task data, and the second subtask can be executed based on the task data. Therefore, the problem of low task execution efficiency existing in the related art can be solved, and the effect of improving task execution efficiency can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 is a hardware structure block diagram of a server device according to a method for executing a task in an embodiment of the present application;

[0013] Figure 2 is a flowchart of a method for executing a task according to an embodiment of the present application;

[0014] Figure 3 is a schematic diagram of a background task for batch creation of logical volumes according to an embodiment of the present application;

[0015] Figure 4 is a flowchart illustrating a background task of creating a storage pool according to an embodiment of the present application;

[0016] Figure 5 It is a structural block diagram of a device for executing tasks according to an embodiment of the present application. DETAILED DESCRIPTION

[0017] The embodiments of the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0018] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0019] The method embodiments provided in the embodiments of the present application can be executed in a server device or a similar computing device. Taking running on a server device as an example, Figure 1 1 is a hardware structure block diagram of a server device of a method for executing a task in an embodiment of the present application. Figure 1 As shown, the server device may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the server device may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above server device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0020] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as computer programs corresponding to the execution methods of tasks in the embodiments of the present application. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, that is, to implement the above-mentioned methods. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the server device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0021] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the server device. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0022] In this embodiment, a method for executing a task is provided. Figure 2 is a flowchart of a method for executing a task according to an embodiment of the present application, such as Figure 2 As shown, the process includes the following steps:

[0023] Step S202, in the case where it is determined that a master node included in the target cluster fails, task data of the target task stored in the database included in the target cluster is obtained, wherein the task data includes target data, and the master node, in the case of receiving a task instruction for instructing to execute the target task, determines the subtasks included in the target task, determines the target data required to execute each of the subtasks, stores the target data in the database, executes each of the subtasks, and in the case of completing the execution of a first subtask, updates the execution status of the first subtask in the task data;

[0024] Step S204, determining a second subtask in the target task that is in an unexecuted state based on the task data;

[0025] Step S206: executing the second subtask based on the task data.

[0026] In the above embodiment, the target cluster can be a distributed cluster, a distributed storage cluster, a distributed data cluster, etc. The target cluster can include multiple nodes, including a master node and the others are standby nodes. When the target task needs to be executed in the target cluster, a separate thread can be set for the target task and put it into the background task for execution. During the execution of the target task, the user can also perform other operations to enhance the user experience. Among them, the target can be a setting task or a deletion operation task, or a large-scale task involving more data, which is not limited by the present invention.

[0027] In the above embodiment, the target cluster may also include a database, and each node included in the target cluster may access the database, store data in the database, or read existing data from the database. When the master node receives a task instruction to execute the target task, it may split the target task into multiple subtasks, obtain the task data required to execute each subtask, and store the task data in the database included in the target cluster. Since other nodes may also access the database, even if the master node fails, the standby node used to replace the master node may track the existing task data through the database to determine the currently executed target task and the execution progress of the target task, and may also obtain the required data for executing each subtask from the database, so that the standby node that replaces the master node acts as the master node and continues to execute the target task. Moreover, when the standby node that replaces the master node executes the target task, it continues to execute the target task on the basis of the target task executed by the master node, that is, it only executes the subtasks that have not been completed by the master node, without re-executing the target task, thereby improving the efficiency of task execution.

[0028] In the above embodiment, in order to enable the standby node that replaces the master node to continue to execute the target task on the basis of the master node executing the target task, the master node can update the execution status of the first subtask in the task data in the database when the first subtask is completed. For example, when the first subtask is not executed, the execution status of the first subtask in the task data can be empty, and when the first subtask is completed, the execution status of the first subtask in the task data can be updated from empty to completed.

[0029] In the above embodiment, the master node determines that the subtasks included in the target task may include splitting the steps of executing the target task, collecting the commands to be executed for each step, and determining the commands to be executed for each step as a subtask. During the execution of the target task, after each step is completed and the command is executed, a new state can be given to the task, and the state is synchronized within the cluster; secondly, at the beginning of the task issuance, all parameters required for the execution of the task, that is, task data, are collected, saved to the database, and the parameter information is also synchronized between nodes. After the task starts, the node (master node) that receives the task is responsible for completing all operations of the task. If the node is down for some reason during the execution process, a new node will replace the original master node and become the new master node. After the new master node is determined, it is first necessary to scan the unfinished background tasks in the database and the status of the task. After obtaining the above information, confirm the starting point for the continued execution of the background task, and then obtain all the parameters involved in the task from the database. The new master node integrates the current progress and parameter information of the task and replaces the old master node to complete the remaining steps of the task. This method not only retains the advantages of the background task operation mode, but also allows users to perform other operations on the cluster during the task execution process to avoid unnecessary waiting. At the same time, the introduction of a mechanism that can be executed across nodes after the background task fails can ensure that the task will not fail due to reasons such as downtime, and avoid the need for manual intervention after the failure of some tasks. On the basis of not reducing the user experience, the possibility of successful task execution is improved, and the impact of node failure is solved to a certain extent. Among them, the task data may include the step information of each subtask, and the task table of the task data can be seen in Table 1. The task data can also include the execution order of each subtask, and the task execution step table can be seen in Table 2. The task data can also include the task context, and the task context can also be stored in the memory of the node that executes the target task, and the task context can be seen in Table 3.

[0030] Table 1

[0031]

[0032]

[0033]

[0034] Table 2

[0035]

[0036]

[0037] Table 3

[0038]

[0039] Through the present application, when the master node in the target cluster receives a task instruction for executing a target task, it can determine the subtasks included in the target task, determine the target data required for executing each subtask, and store the target data in the database of the target cluster, execute each subtask, and when a first subtask is executed and completed, the execution status of the first subtask is rooted in the task data. When the master node fails, the task data stored in the database can be obtained, and the second subtask included in the target task that is in an unexecuted state can be determined based on the task data, and the second subtask is executed according to the task data. Since the task data required for executing each subtask included in the target task can be stored in the database of the target cluster when the master node executes the target task, when the master node fails, the unexecuted second subtask can be determined based on the task data, and the second subtask can be executed based on the task data. Therefore, the problem of low task execution efficiency existing in the related art can be solved, and the effect of improving task execution efficiency can be achieved.

[0040] The execution subject of the above steps may be a standby node included in the target cluster and used to replace the master node, but is not limited thereto.

[0041] In an exemplary embodiment, the method of determining the second subtask in the target task that is in an unexecuted state based on the task data includes: determining the target type of the target task; determining a first execution order of the subtasks included in the target task based on the task data when the target type indicates that the target task is a serial task; determining a third subtask in an unexecuted state based on the execution state of each of the subtasks included in the task data; determining a fourth subtask that is executed first included in the third subtask based on the execution order; determining a first execution result of the master node executing the fourth subtask; and determining a first execution result of the master node executing the fourth subtask based on the first execution result. In the case where the master node is instructed to complete the execution of the fourth subtask, the other subtasks included in the third subtask except the fourth subtask are determined as the second subtasks; in the case where the first execution result indicates that the master node has not completed the execution of the fourth subtask, the third subtask is determined as the second subtask; in the case where the target type indicates that the target task is a parallel task, the executed fifth subtask included in the target task is determined based on the execution status of each of the subtasks included in the task data; and the other subtasks included in the target task except the fifth subtask are determined as the second subtask. In this embodiment, the target task may include a serial task and a parallel task. A serial task may be a task in which the execution of the next subtask depends on the previous subtask, and a parallel task may be a task in which each subtask is independent of each other. When the target task is a serial task, the target task may be a task for creating a storage pool, etc. When the target task is a parallel task, the target task may be a task for creating logical volumes in batches, etc.

[0042] In the above embodiment, different types of target tasks can determine the second subtask in an unexecuted state in different ways. Therefore, the target type of the target task can be determined. When the master node receives the task instruction, the target type of the target task can be determined according to the task instruction, and the target type is stored in the database. When the master node fails, the standby node used to replace the master node can determine the target type of the target task according to the type identifier included in the task data for indicating the target type. When the target type indicates that the target task is a serial task, the first execution order of each subtask included in the target task can be determined, wherein the first execution order can be determined according to the execution priority of each subtask included in the task data, and the subtask with a higher execution priority is executed first. It is also possible to determine that the execution status included in the task data is unexecuted or empty subtasks, and the execution status is unexecuted or empty subtasks are determined as the third subtask. The fourth subtask that is first executed in the third subtask may have been partially executed by the master node, so when re-executing, it can be first judged whether the expected results of each command in the subtask, that is, the fourth subtask, have been generated. If the result has been generated, it means that the command has been executed, and the execution of the command can be skipped. Then the other subtasks included in the third subtask except the fourth subtask are determined as the second subtask. If the result is not generated, it means that the fourth subtask has not been executed, and the third subtask is determined as the second subtask.

[0043] In the above embodiment, when the target task is a parallel task, all task sets to be created can be obtained, that is, all subtasks included in the target task, and all subtasks included in the target task are directly subtracted from the fifth subtask that has been executed, and the subtasks in the difference set are determined as the second subtask, and the tasks in the difference set are executed. In a specific embodiment, the executed parallel tasks can be background tasks of batch operation type, and each operation is independent of each other, such as batch creation of logical volumes (equivalent to the target task), and the result of the first logical volume created does not affect the creation of the second logical volume, that is, each subtask included in the target task is independent of each other. Figure 3 The following is a diagram of the background task of creating logical volumes in batches: Figure 3 As shown, the master node can create a logical volume background task, obtain the logical volume parameter information (corresponding to the above task data), and complete the creation of logical volumes one by one. When the master node crashes during the execution of the background task of batch creation of logical volumes, the standby node can take over the tasks of the master node. The standby node obtains all the logical volume parameter information to be created and the set of logical volumes that the master node has completed and recorded from the background task parameters. The difference between the two is the logical volume that has not been completed and needs to be created. The standby node only needs to complete the creation of the remaining logical volumes one by one.

[0044] In the above embodiment, when the target task is a serial task, it is determined whether the fourth subtask that needs to be executed first has been executed by the master node. If the master node has completed the fourth subtask, there is no need to execute the fourth subtask, thereby avoiding repeated execution of subtasks and improving the efficiency of task execution. When the target task is a parallel task, it is not necessary to make too many judgments. It is only necessary to determine the fifth subtask that has been executed, and the other subtasks except the fifth subtask that has been executed are determined as the second subtasks for execution, thereby improving the efficiency of task execution.

[0045] In an exemplary embodiment, determining the execution result of the fourth subtask executed by the master node includes: obtaining a pre-created verification command for verifying whether the fourth subtask is executed; executing the verification command to obtain a verification result; if the verification result indicates that the verification is successful, determining that the execution result is completed; if the verification result indicates that the verification fails, determining that the execution result is not completed. In this embodiment, the pre-created verification command can be used to verify whether the fourth subtask is executed by the master node. When the master node creates each subtask included in the target task, it can simultaneously create the expected execution result of each subtask, that is, the result generated when the subtask is successfully executed. At the same time, a verification command for verifying the expected execution result of each subtask is also created. If the expected execution result can be obtained after executing the verification command, it is determined that the verification is successful. If the expected execution result is not obtained after executing the verification command, it is determined that the verification fails. If the verification is successful, it means that the master node has completed the fourth subtask, but has not updated the execution status. If the verification fails, it means that the master node has not completed the fourth subtask, and the standby node that is used to replace the master node needs to start execution from the fourth subtask. In the above two embodiments, when the target task is to create a storage pool task, the flowchart of the background task of creating a storage pool can be found in the attached Figure 4 ,like Figure 4 As shown in the figure, creating a storage pool can be divided into three steps: constructing a crush structure, creating a storage pool, and setting storage pool related attributes. Among them, setting storage pool related attributes can be setting the number of storage pool copies and the storage pool application scenario and application type after adding the storage pool to the file system.

[0046] In the above embodiment, when constructing the crush structure, the master node can analyze and record the output of the task during normal execution in advance. For example, first obtain the root name with the largest number in the cluster. The current largest root name may be hdd_3. Then, the construction of the crush structure is based on the hdd_3 number plus 1, that is, a root hdd_4 is obtained. When verifying whether the task is executed, the verification command "osd tree|grep-w hdd_4" can be executed to determine whether hdd_4 exists. If the verification is successful, the subsequent tasks are executed. If the verification fails, the crush structure continues to be constructed. Among them, the subsequent task may be to record this task "hdd_4" in the step_result field of the first step of creating the storage pool: "osd tree|grep-whdd_4".

[0047] In the above embodiment, verifying whether the fourth subtask is executed and completed by a pre-created verification command can help ensure that the fourth subtask is correctly executed, thereby improving the accuracy and efficiency of the entire process. Through the verification command, it can be ensured that each step is executed in the expected manner to avoid potential errors and delays. In addition, the verification command can also help record and track the execution of the fourth subtask, avoid the fourth subtask from being repeated or missed, and ensure the efficient and accurate execution of the target task.

[0048] In an exemplary embodiment, after executing the second subtask based on the task data, the method further includes: when the target subtask included in the second subtask is completed, modifying the execution status of the target subtask included in the task data to execution completed.

[0049] In this embodiment, when the target subtask included in the second subtask is executed, the execution status of the target subtask in the database can be refreshed, and the execution status can be modified to execution completion. After the status is refreshed, the database synchronization mechanism between nodes is relied upon to enable other nodes to perceive the execution status of the background task, so that when other nodes become the master node, the execution progress of the background task can be confirmed, and the target task can be continued to be executed according to the execution progress, thereby improving the efficiency of the target task execution. In addition, the problem of task failure due to the downtime of the master node during the task execution process is solved, and the task is prevented from falling into an intermediate state where part of the operation has been completed and the remaining part of the operation has not yet been carried out, thereby eliminating the investment of manual intervention in task execution after the task fails due to downtime and other reasons.

[0050] In an exemplary embodiment, the execution of the second subtask based on the task data includes: when the target task is a serial task, determining the second execution order of each subtask included in the second subtask based on the task data; repeatedly executing the target operation until the second subtask is completed; wherein the target operation includes: determining the sixth subtask included in the second subtask, and when the target operation is executed for the first time, the sixth subtask is the subtask corresponding to the first execution order included in the second execution order; calling the target data corresponding to the sixth subtask included in the task data to execute the sixth subtask; determining the second execution result of the sixth subtask; when the second execution result indicates that the sixth subtask is executed successfully, updating the sixth subtask to the seventh subtask, wherein the seventh subtask is a task whose execution order is second only to the execution order of the sixth subtask. In this embodiment, the second execution order of each subtask included in the second subtask can be determined according to the task data, and each subtask included in the second subtask can be executed in sequence according to the second execution order. For example, the sixth subtask included in the second subtask that needs to be executed first is determined according to the second execution order. If the sixth subtask is executed successfully, the next task of the sixth subtask is executed. If the next task is executed successfully, the next subtask of the currently executed subtask is executed again until all subtasks are executed. In addition, when executing the sixth subtask, the target data corresponding to the sixth subtask included in the task data can be called, and there is no need to obtain the target data again, which improves the speed of task execution.

[0051] In the above embodiment, the subtasks may be executed in sequence according to the second execution order of each subtask included in the second subtask, and when one subtask is executed successfully, the next subtask may be executed, thereby ensuring that the subtasks can be executed in an orderly manner.

[0052] In an exemplary embodiment, after determining the second execution result of the sixth subtask, the method further includes: when the second execution result indicates that the sixth subtask has failed to execute, stopping the execution of the target task. In this embodiment, when the execution status is updated to execution completed, the next task is executed; when the execution fails, the background task is terminated, that is, the subsequent tasks of the subtask are no longer executed. In the above embodiment, since the target task is a serial task, when the sixth subtask fails to execute, continuing to execute subsequent subtasks will cause the other subtasks to fail to execute. Therefore, when the sixth subtask fails to execute, stopping the execution of the target task can effectively avoid unnecessary waste of computing power.

[0053] In an exemplary embodiment, before obtaining the task data of the target task stored in the database included in the target cluster, the method also includes: determining the reception time of the last heartbeat information received, wherein the heartbeat information is information sent by the master node; determining the time interval between the reception time and the current time; and determining that the master node has a fault when the time interval is greater than a predetermined time threshold.

[0054] In the present embodiment, if the master node fails, depending on the cluster master-standby switching mechanism, other nodes in the cluster will take over the role of the original master node and become the new master node; the master-standby switching can be completed by relying on the keepalived component: when the Keepalived service is working normally, the master node continuously sends heartbeat messages to the standby node to notify the standby node that it is still working normally. When the master node fails, the heartbeat message cannot be sent, so the standby node cannot detect the heartbeat information transmitted from the master node, so it calls its own takeover program to take over the IP resources and services of the master node to work. Among them, the master node can send heartbeat information to the standby node in a multicast manner. By sending heartbeat information to the standby node through the master node, the standby node can monitor the working status of the master node in time and take over, ensuring the efficiency of completing the task.

[0055] In an exemplary embodiment, when a master node receives a first task instruction for instructing the execution of a first task, the master node determines a first task type of the first task, and when the first task type indicates that the first task is a parallel task, the first task is split into multiple subtasks, and the multiple subtasks are divided into N groups to obtain N groups of subtasks, and each group of subtasks in the N groups of subtasks is sent to the N standby nodes included in the target cluster respectively, and the N standby nodes respectively execute the received group of subtasks, and after each standby node completes the execution of a subtask, the execution result of the subtask is uploaded to the database included in the target cluster, and the master node determines in the database the execution of each group of subtasks by each standby node. When the master node determines that a target standby node among the standby nodes has not updated the execution status of the subtask in the database within a predetermined time, it can be determined that the target standby node has failed, and the subtasks that have not been executed by the target standby node are sent to other nodes for execution.

[0056] In an exemplary embodiment, when the first task type is executed as a serial task, the master node splits the first task into multiple subtasks; determines the expected result of the execution of each subtask, and when the subtask is a task other than the first subtask included in the first task, determines the expected result of the subtask and the previous task of the subtask as a data group to obtain M data groups, and sends the M data groups to the M standby nodes included in the target cluster. The standby nodes execute the subtasks according to the expected results included in the data groups, and upload the execution results to the database. The master node determines the execution status of the subtasks through the execution results in the database, and sends the subtasks that failed to execute to other nodes for execution.

[0057] In the above embodiment, the first task can be split, and multiple standby nodes respectively execute the subtasks in the first task, thereby improving the execution efficiency of the task. In the above embodiment, all operations involved in the background task are split, and after each step is completed, the background task is updated to a new state, and the state is synchronized to all nodes that may become the master node. In addition, when the task is triggered, the parameter information corresponding to the task needs to be saved and synchronized. Then after the new master node is determined, the new master node needs to actively scan all unfinished tasks and the parameter information corresponding to the task. After obtaining the task status and parameter information, the task is re-triggered by the new node, but during the task execution process, it is necessary to skip the steps that the task has been executed according to the status of the task, and only execute the steps that the task has not been executed. The advantages of the background task operation mode are retained, so that the user can perform other operations on the cluster during the task execution process to avoid unnecessary waiting. At the same time, the mechanism that can be executed across nodes after the background task fails is introduced, which can ensure that the task will not fail due to reasons such as downtime, and avoid the investment of manual intervention after the failure of some tasks. On the basis of not reducing the user experience, the possibility of successful task execution is improved, and the impact of node failure is solved to a certain extent.

[0058] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0059] In the present embodiment, a task execution device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware is also possible and conceived.

[0060] Figure 5 is a structural block diagram of a task execution device according to an embodiment of the present application, such as Figure 5 As shown, the device comprises:

[0061] An acquisition module 52 is used to acquire task data of a target task stored in a database included in the target cluster when it is determined that a master node included in the target cluster fails, wherein the task data includes target data, and when the master node receives a task instruction for instructing to execute the target task, determines the subtasks included in the target task, determines the target data required to execute each of the subtasks, stores the target data in the database, executes each of the subtasks, and updates the execution status of a first subtask in the task data when the first subtask is completed;

[0062] A determination module 54, configured to determine a second subtask in the target task that is in an unexecuted state based on the task data;

[0063] The execution module 56 is configured to execute the second subtask based on the task data.

[0064] In an exemplary embodiment, the determination module 54 can determine the second subtask in the target task that is in an unexecuted state based on the task data by: determining the target type of the target task; in the case where the target type indicates that the target task is a serial task, determining a first execution order of the subtasks included in the target task based on the task data; determining a third subtask in an unexecuted state based on the execution state of each of the subtasks included in the task data; determining a fourth subtask that is executed first included in the third subtask based on the execution order; determining a first execution result of the master node executing the fourth subtask; and determining a first execution result of the master node executing the fourth subtask. When an execution result indicates that the master node has completed the fourth subtask, the other subtasks included in the third subtask except the fourth subtask are determined as the second subtasks; when the first execution result indicates that the master node has not completed the fourth subtask, the third subtask is determined as the second subtask; when the target type indicates that the target task is a parallel task, the executed fifth subtask included in the target task is determined based on the execution status of each of the subtasks included in the task data; and the other subtasks included in the target task except the fifth subtask are determined as the second subtasks.

[0065] In an exemplary embodiment, the determination module 54 can determine the execution result of the fourth subtask executed by the main node in the following manner: obtain a pre-created verification command for verifying whether the fourth subtask is completed; execute the verification command to obtain a verification result; if the verification result indicates that the verification is successful, determine that the execution result is completed; if the verification result indicates that the verification fails, determine that the execution result is not completed.

[0066] In an exemplary embodiment, the apparatus may be configured to modify the execution status of the target subtask included in the task data to execution completed after executing the second subtask based on the task data and when the target subtask included in the second subtask has been executed.

[0067] In an exemplary embodiment, the execution module 56 can execute the second subtask based on the task data in the following manner: when the target task is a serial task, repeatedly perform the following operations: determine the sixth subtask to be executed at the current moment based on the second execution order of the second subtask; determine the second execution result of the sixth subtask; and when the second execution result indicates that the sixth subtask is executed successfully, execute the next subtask of the sixth subtask until all tasks included in the second subtask are executed.

[0068] In an exemplary embodiment, the apparatus may also be configured to, after determining the second execution result of the sixth subtask, stop executing the target task if the second execution result indicates that the sixth subtask has failed to execute.

[0069] In an exemplary embodiment, the device can also be used to determine the reception time of the last heartbeat information received before obtaining the task data of the target task stored in the database included in the target cluster, wherein the heartbeat information is the information sent by the master node; determine the time interval between the reception time and the current time; and determine that the master node has a fault when the time interval is greater than a predetermined time threshold.

[0070] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0071] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when run.

[0072] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0073] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0074] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0075] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any one of the above method embodiments are implemented.

[0076] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0077] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0078] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for executing a task, characterized in that: include: In the case where it is determined that a master node included in the target cluster fails, task data of the target task stored in a database included in the target cluster is obtained, wherein the task data includes target data, and the master node, upon receiving a task instruction for instructing to execute the target task, determines the subtasks included in the target task, determines the target data required to execute each of the subtasks, stores the target data in the database, executes each of the subtasks, and, in the case where a first subtask is executed and completed, updates the execution status of the first subtask in the task data; A second subtask in the target task that is in an unexecuted state is determined based on the task data; and the second subtask is executed based on the task data.

2. The method according to claim 1, characterized in that: The determining, based on the task data, a second subtask in the target task that is in an unexecuted state comprises: Determining a target type of the target task; In a case where the target type indicates that the target task is a serial task, determining a first execution order of subtasks included in the target task based on the task data; determining a third subtask in an unexecuted state based on the execution state of each of the subtasks included in the task data; Determining, based on the execution order, a fourth subtask included in the third subtask to be executed first; Determine a first execution result of the master node executing the fourth subtask; In a case where the first execution result indicates that the master node has completed the execution of the fourth subtask, determining other subtasks included in the third subtask except the fourth subtask as the second subtasks; If the first execution result indicates that the master node has not completed the fourth subtask, determining the third subtask as the second subtask; In a case where the target type indicates that the target task is a parallel task, determining the executed fifth subtask included in the target task based on the execution status of each of the subtasks included in the task data; The other subtasks included in the target task except the fifth subtask are determined as the second subtasks.

3. The method according to claim 2, characterized in that Determining the execution result of the master node executing the fourth subtask includes: Obtaining a pre-created verification command for verifying whether the fourth subtask has been executed; Execute the verification command to obtain a verification result; If the verification result indicates that the verification is successful, determining that the execution result is execution completed; In the case where the verification result indicates that the verification has failed, the execution result is determined to be incomplete.

4. The method according to claim 1, characterized in that: After performing the second subtask based on the task data, the method further includes: When the target subtask included in the second subtask is executed, the execution status of the target subtask included in the task data is modified to execution completed.

5. The method according to claim 1, characterized in that The performing the second subtask based on the task data includes: In the case where the target task is a serial task, determining a second execution order of each subtask included in the second subtask based on the task data; Repeat the target operation until the second subtask is completed; The target operation includes: Determine a sixth subtask included in the second subtask, where, when the target operation is executed for the first time, the sixth subtask is a subtask corresponding to a first-executed sequence included in the second execution sequence; Calling the target data included in the task data and corresponding to the sixth subtask to execute the sixth subtask; determining a second execution result of the sixth subtask; When the second execution result indicates that the sixth subtask is executed successfully, the sixth subtask is updated to a seventh subtask, wherein the seventh subtask is a task whose execution order is next to that of the sixth subtask.

6. The method according to claim 5, characterized in that After determining the second execution result of the sixth subtask, the method further includes: When the second execution result indicates that the sixth subtask fails to be executed, the target task is stopped.

7. The method according to claim 1, characterized in that Before acquiring the task data of the target task stored in the database included in the target cluster, the method further includes: Determine the reception time of the last heartbeat information received, wherein the heartbeat information is information sent by the master node; Determine the time interval between the receiving time and the current time; When the time interval is greater than a predetermined time threshold, it is determined that a failure occurs on the master node.

8. A task execution device, characterized in that: include: An acquisition module, used for acquiring task data of a target task stored in a database included in the target cluster when it is determined that a master node included in the target cluster fails, wherein the task data includes target data, and the master node, when receiving a task instruction for instructing to execute the target task, determines the subtasks included in the target task, determines the target data required to execute each of the subtasks, stores the target data in the database, executes each of the subtasks, and when a first subtask is executed and completed, updates the execution status of the first subtask in the task data; A determination module is used to determine a second subtask in the target task that is in an unexecuted state based on the task data; and an execution module is used to execute the second subtask based on the task data.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the method described in any one of claims 1 to 7 when executed by a processor.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method described in any one of claims 1 to 7 are implemented.