Task scheduling system, method, device, electronic device and storage medium

Abstract

The embodiment of the present invention provides a task scheduling system, method, device, electronic device and storage medium. The system includes a dispatch node and multiple execution nodes, and each execution node has a node label; the dispatch node is used to obtain the task to be processed and the target node label; add the target node label to the task to be processed, and dispatch the task to be processed with the target node label added to the task queue; each execution node is used to obtain the target task from the task queue and process the target task; wherein the target task is: the task to be processed whose target node label is the same as the node label of the execution node. Compared with the prior art, the solution provided by the embodiment of the present invention can ensure the security of the task to be processed.

Description

Technical Field

[0001] The present invention relates to the field of Internet technology, and in particular to a task scheduling system, method, device, electronic equipment and storage medium. Background Art

[0002] Currently, with the continuous development of Internet technology, "cloud" plays an increasingly important role in various fields, such as cloud computing, cloud storage, etc. The so-called cloud is an abstraction of the Internet and underlying infrastructure, which can be divided into public cloud and private cloud.

[0003] Among them, the so-called private cloud is a cloud built for a single customer. Specifically, the customer owns the underlying infrastructure and can control the way applications are deployed on this underlying infrastructure, thereby effectively controlling the data security and service quality in the private cloud.

[0004] In the related art, a private cloud is provided with multiple public execution nodes, which are used to process various pending tasks of customers. For example, multiple public execution nodes in a company's private cloud can process various pending tasks initiated by the company's finance, administration, sales, research and development departments, etc. Specifically, multiple public execution nodes are uniformly managed by a management node, which obtains pending tasks sent by various departments of the customer and assigns the pending tasks to one of the multiple public execution nodes so that it processes the pending tasks.

[0005] However, in the above-mentioned related technologies, the public execution nodes to which the tasks to be processed are assigned have the characteristic of uncertainty, resulting in that the security of the tasks to be processed cannot be guaranteed. Summary of the invention

[0006] The purpose of the embodiments of the present invention is to provide a task scheduling system, method, device, electronic device and storage medium to ensure the security of tasks to be processed. The specific technical solution is as follows:

[0007] In a first aspect, an embodiment of the present invention provides a task scheduling system, the system comprising a dispatch node and a plurality of execution nodes, and each execution node has a node label;

[0008] The dispatch node is used to obtain the pending task and the target node tag; add the target node tag to the pending task, and dispatch the pending task with the target node tag added to the task queue;

[0009] Each execution node is used to obtain a target task from the task queue and process the target task, wherein the target task is a task to be processed whose target node label is the same as the node label of the execution node.

[0010] Optionally, in a specific implementation, there are multiple task queues, and among the multiple execution nodes, there are execution nodes with the same node label, wherein the execution nodes with node labels different from any other execution nodes correspond to all task queues, and among the execution nodes with the same node label, different execution nodes correspond to different task queues;

[0011] Each execution node is specifically used to obtain a target task from a task queue corresponding to the execution node and process the target task.

[0012] Optionally, in a specific implementation, obtaining the target task from the task queue corresponding to the execution node includes:

[0013] When there are multiple task queues corresponding to the execution node, each task queue corresponding to the execution node is traversed in a loop according to a predetermined queue traversal order;

[0014] For each task queue traversed, if the task queue includes the target task and the number of target tasks included is not less than the specified number, then obtain the specified number of target tasks from the task queue in order of task dispatch time from early to late, and traverse the next task queue; if the task queue includes the target task and the number of target tasks included is less than the specified number, obtain the target tasks included in the task queue in order of task dispatch time from early to late, and traverse the next task queue; if the task queue does not include the target task, directly traverse the next task queue.

[0015] Optionally, in a specific implementation, the task queue is a rocketmq message queue; the information type of the task to be processed is a rocketmq message; and adding the target node label to the task to be processed includes:

[0016] The attribute value of the property attribute of the task to be processed is set to the target node label.

[0017] Optionally, in a specific implementation, obtaining the target task from the task queue includes:

[0018] Obtain the target task from the task queue through a preset filtering method.

[0019] In a second aspect, an embodiment of the present invention provides a task scheduling method, which is applied to a dispatching node in a task scheduling system, and the method includes:

[0020] Get pending tasks and target node labels;

[0021] Adding the target node label to the task to be processed;

[0022] The pending task with the target node label added is dispatched to the task queue, so that each execution node in the task scheduling system obtains the target task from the task queue and processes the target task; wherein the target task obtained by each execution node is: the pending task with the target node label being the same as the node label of the execution node.

[0023] In a third aspect, an embodiment of the present invention provides another task scheduling method, which is applied to each execution node in a task scheduling system, and the method includes:

[0024] Obtain the target task from the task queue; wherein each pending task in the task queue is: the pending task with the target node tag added thereto dispatched to the task queue by the dispatching node in the task scheduling system after obtaining the pending task and the target node tag, and the target task is: the pending task with the target node tag being the same as the node tag of the execution node;

[0025] Process the target task.

[0026] In a fourth aspect, an embodiment of the present invention provides a task scheduling device, which is applied to a dispatching node in a task scheduling system, and the device includes:

[0027] Information acquisition module, used to obtain pending tasks and target node labels;

[0028] A label adding module, used for adding the target node label to the task to be processed;

[0029] The task dispatching module is used to dispatch the pending tasks with the target node label added to the task queue, so that each execution node in the task scheduling system obtains the target task from the task queue and processes the target task; wherein the target task obtained by each execution node is: the pending task with the target node label being the same as the node label of the execution node.

[0030] In a fifth aspect, an embodiment of the present invention provides another task scheduling device, which is applied to each execution node in a task scheduling system, and the device includes:

[0031] The task acquisition module is used to acquire the target task from the task queue; wherein each pending task in the task queue is: the pending task with the target node tag added thereto is dispatched to the task queue by the dispatching node in the task scheduling system after acquiring the pending task and the target node tag, and the target task is: the pending task with the target node tag being the same as the node tag of the execution node;

[0032] A task processing module is used to process the target task.

[0033] In a sixth aspect, an embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0034] Memory, used to store computer programs;

[0035] The processor is used to implement the steps of a task scheduling method provided in the second aspect or the third aspect when executing the program stored in the memory.

[0036] In the seventh aspect, an embodiment of the present invention provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of a task scheduling method provided in the second or third aspect are implemented.

[0037] In an eighth aspect, an embodiment of the present invention provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the steps of a task scheduling method provided in the second or third aspect above.

[0038] Beneficial effects of the embodiments of the present invention:

[0039] Applying the scheme provided by the embodiment of the present invention, the dispatching node in the task scheduling system can simultaneously obtain the target node label when obtaining the pending task, wherein the target node label is the label specified for the pending task, and can indicate the execution node used to process the pending task. Then, when the dispatching node dispatches the pending task to the task queue, the target node label can be added to the pending task, so that the pending task dispatched by the dispatching node to the task queue is the pending task with the target node label added. Correspondingly, each execution node in the task scheduling system has a node label, and the node label indicates the pending task that the execution node can process. Then, when the execution node obtains the pending task that needs to be processed from the task queue, the target task obtained is the pending task whose target node label is the same as the node label that the execution node has. Then, the execution node can process the obtained target task.

[0040] Based on this, by applying the solution provided by the embodiment of the present invention, the tasks to be processed and each execution node can be classified by node labels, wherein the tasks to be processed with a certain node label will be acquired by the execution node with the same node label and processed by the execution node. In other words, each type of execution node will process a preset type of tasks to be processed, that is, the execution nodes that process each type of tasks to be processed are determined, so that each type of tasks to be processed will be processed by a preset type of execution node, which can improve the security of processing tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0042] Figure 1 A structural diagram of a task scheduling system provided by an embodiment of the present invention;

[0043] Figure 2 A flowchart of a task scheduling method for a dispatching node in a task scheduling system provided by an embodiment of the present invention;

[0044] Figure 3 A flowchart of a task scheduling method applied to each execution node in a task scheduling system provided by an embodiment of the present invention;

[0045] Figure 4 A schematic diagram of the structure of a task scheduling device for a dispatching node in a task scheduling system provided by an embodiment of the present invention;

[0046] Figure 5 A schematic diagram of the structure of a task scheduling device applied to each execution node in a task scheduling system provided by an embodiment of the present invention;

[0047] Figure 6 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention;

[0048] Figure 7 A schematic diagram of a specific embodiment of a task scheduling system provided for applying an embodiment of the present invention. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] In the related art, a private cloud is provided with multiple public execution nodes, which are used to process various pending tasks of customers. Specifically, the multiple public execution nodes are uniformly managed by a management node, which obtains pending tasks sent by various departments of the customer and assigns the pending tasks to one of the multiple public execution nodes so that it processes the pending tasks. However, in the above-mentioned related art, the public execution nodes to which the pending tasks are assigned have the characteristics of uncertainty, resulting in that the security of the pending tasks cannot be guaranteed.

[0051] In order to solve this technical problem, an embodiment of the present invention provides a task scheduling system. The system includes a dispatching node and multiple execution nodes, and each execution node has a node label;

[0052] The dispatch node is used to obtain the pending task and the target node tag; add the target node tag to the pending task, and dispatch the pending task with the target node tag added to the task queue;

[0053] Each execution node is used to obtain a target task from the task queue and process the target task, wherein the target task is a to-be-processed task whose target node label is the same as the node label of the execution node.

[0054] Among them, the task scheduling system can be an independent system for scheduling and processing various tasks to be processed, or it can be a subsystem included in a private cloud system for scheduling and processing various tasks to be processed, which is reasonable. In addition, the dispatch node in the task scheduling system can be any device that needs to schedule tasks, such as a desktop computer, a laptop computer, etc., and the execution node in the task scheduling system can be a physical machine that needs to obtain and process tasks to be processed, such as a server, etc.; it can also be a virtual machine that needs to obtain and process tasks to be processed, which is also reasonable.

[0055] As can be seen above, by applying the scheme provided by the embodiment of the present invention, the dispatching node in the task scheduling system can simultaneously obtain the target node label when obtaining the pending task, wherein the target node label is the label specified for the pending task, and can indicate the execution node used to process the pending task. Then, when the dispatching node dispatches the pending task to the task queue, the target node label can be added to the pending task, so that the pending task dispatched by the dispatching node to the task queue is the pending task with the target node label added. Accordingly, each execution node in the task scheduling system has a node label, and the node label indicates the pending task that the execution node can process. Then, when the execution node obtains the pending task that needs to be processed from the task queue, the target task obtained is the pending task with the target node label identical to the node label that the execution node has. Then, the execution node can process the target task obtained.

[0056] Based on this, by applying the solution provided by the embodiment of the present invention, the tasks to be processed and each execution node can be classified by node labels, wherein the tasks to be processed with a certain node label will be acquired by the execution node with the same node label and processed by the execution node. In other words, each type of execution node will process a preset type of tasks to be processed, that is, the execution nodes that process each type of tasks to be processed are determined, so that each type of tasks to be processed will be processed by a preset type of execution node, which can improve the security of processing tasks.

[0057] A task scheduling system provided by an embodiment of the present invention is described in detail below with reference to the accompanying drawings.

[0058] Figure 1 A schematic diagram of the structure of a task scheduling system provided by an embodiment of the present invention. Figure 1 As shown, the task scheduling system includes a dispatching node 110 and a plurality of execution nodes 120, and each execution node 120 has a node label;

[0059] The dispatch node 110 is used to obtain the pending task and the target node tag; add the target node tag to the pending task, and dispatch the pending task with the target node tag to the task queue;

[0060] Each execution node 120 is used to obtain a target task from a task queue and process the target task;

[0061] The target task is a task to be processed whose target node label is the same as the node label of the execution node.

[0062] Among them, the target node label obtained by the dispatch node 110 is: a label specified for the obtained pending task, and the target node label can indicate the execution node used to process the pending task, that is, the pending task should be processed by the execution node having the same node label as the target node label.

[0063] Furthermore, the node label of each execution node 120 indicates the pending tasks that the execution node 120 can process, that is, the execution node 120 can process pending tasks whose target node label is the same as the node label of the execution node 120, that is, the target node label of the pending tasks processed by the execution node is the same as the transition of the execution node.

[0064] The dispatching node 110 may obtain the tasks to be processed and the target node labels in a variety of ways, which are not described in detail in the embodiment of the present invention.

[0065] For example, the dispatch node 110 may provide a human-computer interaction interface for user operation, so that the user can send the to-be-processed task and the target node label to the dispatch node 110 by operating the human-computer interaction interface.

[0066] Exemplary: A user may input a task to be processed, and select a target node label specified for the task to be processed from a plurality of node label options provided in the human-computer interaction interface.

[0067] For another example, the dispatching node 110 may obtain pending tasks and target node labels sent by other electronic devices.

[0068] Exemplarily, the dispatch node 110 can be communicatively connected with the user's client, so that the user can publish the pending task and the target node label specified for the pending task on the client used by the user, and the client can send the pending task and the target node label to the dispatch node 110.

[0069] Furthermore, after acquiring the pending task and the target node tag, the dispatching node 110 may add the target node tag to the pending task, and then dispatch the pending task with the target node tag to the task queue.

[0070] In this way, there is a task queue including a plurality of pending tasks at the dispatch node 110, each pending task in the task queue is added with a target node label, and the target node label of each pending task is the target node label specified for the pending task obtained when the dispatch node 110 obtains the pending task. For each execution node 120, the execution node 120 can read the target node label of each pending task in the task queue, thereby obtaining the pending task whose target node label is the same as the node label it has, that is, obtaining the target task. Then, the execution node 120 can process the obtained target task.

[0071] Wherein, optionally, for each execution node 120, the execution node 120 can obtain the target task from the task queue according to a predetermined period. Specifically: when the execution node 120 obtains the target task from the task queue, the execution node 120 first processes the obtained target task, and then, after the obtained target task is processed, waits for the time interval specified by the predetermined period, and then obtains a new target task from the task queue again; when the execution node 120 does not obtain the target task from the task queue, the execution node 120 can read the task queue again after waiting for the time interval specified by the predetermined period to determine whether the target task can be obtained.

[0072] For example, the time interval specified by the predetermined cycle is 5s, and the execution node 120 obtains the target task from the task queue. The execution node 120 first processes the obtained target task. Then, after the obtained target task is processed, the execution node 120 waits for 5s and reads the task queue again to determine whether a new target task can be obtained. At this time, the execution node 120 does not obtain the new target task from the task queue, and the execution node 120 waits for 5s again and reads the task queue again to determine whether a new target task can be obtained.

[0073] In addition, in order to improve the processing efficiency of the tasks to be processed and ensure that each task to be processed can be processed quickly, each execution node 120 can obtain multiple target tasks from the task queue each time the target task is obtained from the task queue.

[0074] Optionally, for each execution node, when the task queue includes target tasks and the number of target tasks included is not less than a specified number, the execution node 120 can obtain a specified number of target tasks from the task queue; when the task queue includes target tasks and the number of target tasks included is less than a specified number, each execution node 120 can obtain all target tasks from the task queue.

[0075] Optionally, in a specific implementation, the task queue may be a rocketmq message queue, and the information type of the task to be processed may be a rocketmq message;

[0076] Among them, RocketMQ is an open source message middleware in the queue model of distributed systems, which has the characteristics of high performance, high reliability, high real-time and distribution.

[0077] Accordingly, in this specific implementation, the dispatching node 110 adds a target node label to the task to be processed, that is, includes:

[0078] The dispatch node 110 sets the attribute value of the property attribute of the task to be processed as the target node label.

[0079] Among them, property is the attribute of each message in the RocketMQ message queue used for message retrieval.

[0080] Further, based on the above specific implementation, optionally, in another specific implementation, each execution node 120 obtains the target task from the task queue, that is, includes:

[0081] Each execution node 120 obtains the target task from the task queue through a preset filtering method.

[0082] Optionally, the preset filtering method may be a bySql filtering method, where bySql is a filtering method used by the message selector, and uses a method similar to MySQL to filter messages.

[0083] Based on this, by applying the solution provided by the embodiment of the present invention, the tasks to be processed and each execution node can be classified by node labels, wherein a task with a certain node label will be acquired by an execution node with the same node label and processed by the execution node. In other words, each type of execution node will process a preset type of tasks to be processed, that is, the execution nodes that process each type of tasks to be processed are determined, so that each type of tasks to be processed will be processed by a preset type of execution node, which can improve the security of processing tasks.

[0084] It is understandable that the dispatch node 110 may obtain a large number of pending tasks in a short period of time, and among these pending tasks, there may be a large number of pending tasks with the same target node label. In other words, the execution node 120 that can process these pending tasks with the same target node label needs to process more pending tasks in a short period of time.

[0085] Based on this, optionally, in a specific implementation, the number of task queues can be multiple, and there are execution nodes 120 with the same node label among the multiple execution nodes 120, wherein the execution node 120 with a node label different from any other execution node corresponds to all task queues, and among the execution nodes 120 with the same node label, different execution nodes 120 correspond to different task queues;

[0086] Each execution node 120 is specifically used to obtain a target task from a task queue corresponding to the execution node 120 , and process the obtained target task.

[0087] In this specific implementation, the dispatch node 110 can dispatch each pending task with a target node tag to one of the above-mentioned multiple task queues according to any preset dispatch rule. For example, the pending tasks with a target node tag can be dispatched to one of the above-mentioned multiple task queues in turn according to the average dispatch rule; for another example, the pending tasks with a target node tag can be dispatched to a task queue with fewer pending tasks among the above-mentioned multiple task queues according to the queue balance dispatch rule. This is all reasonable.

[0088] In addition, in this specific implementation, the multiple execution nodes 120 included in a task scheduling system provided by an embodiment of the present invention can be divided into multiple execution node groups according to the node labels of each execution node 120, wherein each execution node group includes at least one execution node 120, and when an execution node group includes multiple execution nodes 120, the multiple execution nodes 120 included therein have the same node label.

[0089] Furthermore, in order to ensure load balancing between each execution node 120, to ensure that a task scheduling system provided by an embodiment of the present invention can have a high processing efficiency, so that each task to be processed can be processed as soon as possible, for an execution node group including multiple execution nodes 120, at least one task queue among the above-mentioned multiple task queues can be allocated in advance to each execution node 120 in the execution node group. Thus, each task queue among the above-mentioned multiple task queues corresponds to an execution node 120 in the execution node group, and different execution nodes 120 in the execution node group are allocated different task queues. That is, among the execution nodes 120 with the same node label, different execution nodes 120 correspond to different task queues.

[0090] Among them, for an execution node group including multiple execution nodes 120, the multiple task queues can be evenly distributed to each execution node 120 included in the execution node group, or task queues can be allocated to each execution node 120 according to the processing capacity of each execution node 120 included in the execution node group. Both are reasonable.

[0091] Accordingly, for an execution node group including only one execution node 120, the node label of the execution node 120 included in the execution node group is different from the node label of any execution node 120 in the other execution nodes 120 included in the scheduling system provided by the embodiment of the present invention, and therefore, the execution node 120 included in the execution node group needs to obtain the target task from all the task queues in the above-mentioned multiple task queues. That is, the execution node 120 whose node label is different from that of any other execution node corresponds to all task queues.

[0092] Based on this, in this specific implementation, each execution node 120 can be specifically used to: obtain the target task from the task queue corresponding to the execution node, and process the obtained target task.

[0093] Further, based on the above specific implementation, optionally, in a specific implementation, each execution node 120 obtains the target task from the task queue corresponding to the execution node 120, and processes the obtained target task, which may include:

[0094] When there are multiple task queues corresponding to the execution node 120, each task queue corresponding to the execution node 120 is traversed in a loop according to a predetermined queue traversal order;

[0095] For each task queue traversed, if the task queue includes the target task and the number of target tasks included is not less than the specified number, then obtain the specified number of target tasks from the task queue in order of task dispatch time from early to late, and traverse the next task queue; if the task queue includes the target task and the number of target tasks included is less than the specified number, obtain the target tasks included in the task queue in order of task dispatch time from early to late, and traverse the next task queue; if the task queue does not include the target task, directly traverse the next task queue.

[0096] Specifically, in this specific implementation, for an execution node 120 corresponding to multiple task queues, when assigning task queues to the execution node 120, the traversal order of each task queue can be preset.

[0097] In this way, the execution node 120 can cyclically traverse each task queue corresponding to the execution node 120 according to the predetermined queue traversal order.

[0098] Furthermore, when traversing each task queue, the execution node 120 can obtain a specified number of target tasks from the task queue. By traversing each task queue in a loop, the execution node can obtain a specified number of target tasks from the corresponding multiple task queues. Furthermore, when the specified number of target tasks are obtained, the task acquisition process can be terminated, and the execution node 120 stops obtaining new target tasks and starts processing the obtained specified number of target tasks.

[0099] It is understandable that when traversing each task queue, the number of target tasks included in the task queue may not meet the specified number. Also, the number of all target tasks included in each task queue may not meet the specified number.

[0100] Based on this, in this specific implementation, for each task queue traversed:

[0101] If the task queue includes the target task and the number of the target tasks included is not less than the specified number, the execution node 120 can obtain the specified number of target tasks from the task queue in the order of task dispatch time from early to late, and traverse the next task queue;

[0102] If the task queue includes the target task and the number of the target tasks included is less than the specified number, the execution node 120 can obtain the target tasks included in the task queue in the order of task dispatching time from early to late, and traverse the next task queue; that is, if the task queue includes the target task and the number of the target tasks included is less than the specified number, the execution node 120 can obtain all the target tasks included in the task queue in the order of task dispatching time from early to late;

[0103] If the task queue does not include the target task, the execution node 120 can directly traverse the next task queue.

[0104] Furthermore, considering the processing capability of each execution node 120, a predetermined number of tasks to be processed that each execution node 120 can process at one time may be preset.

[0105] In this way, when the number of target tasks acquired by the execution node 120 meets the predetermined number, the execution node 120 can stop traversing the corresponding task queue and start processing the acquired target tasks. Alternatively, when the number of target tasks acquired by the execution node 120 does not meet the predetermined number, but the target node labels of all pending tasks in each task queue that the execution node 120 has read are different from the node labels of the execution node 120, the execution node 120 can also stop traversing the corresponding task queue and start processing the acquired target tasks.

[0106] For example, the task queues corresponding to an execution node 120 are queue 1, queue 2 and queue 3, the predetermined queue traversal order is queue 1, queue 2 and queue 3, the specified number is 1, and the predetermined number is 32.

[0107] In this way, the execution node 120 can first read the target node label of each pending task in queue 1. Since queue 1 includes multiple target tasks, the execution node 120 can obtain one target task from queue 1 in the order of task dispatch time from early to late; then, the execution node 120 reads the target node label of each pending task in queue 2. Since queue 2 does not include target tasks, the execution node 120 can directly read the target node label of each pending task in queue 3. Since queue 3 includes one target task, the execution node can obtain the target task from queue 3; then, the execution node 120 can read the target node label of each pending task in queue 1 again, and according to the number of target tasks included in queue 1, obtain one target task from queue 1 in the order of task dispatch time from early to late.

[0108] And so on, until the execution node 120 obtains 32 target tasks, the execution node 120 can stop traversing the corresponding task queue and start processing the obtained target tasks; or, until the target node labels of all pending tasks read by the execution node from queue 1, queue 2 and queue 3 are different from the node label of the execution node 120, the execution node 120 can stop traversing the corresponding task queue and start processing the obtained target tasks.

[0109] Accordingly, in this specific implementation, when there is one task queue corresponding to the execution node 120, the execution node can obtain the target task from the task queue in the order of task dispatching time from early to late.

[0110] Among them, when the number of target tasks acquired by the execution node 120 meets the predetermined number, the execution node 120 can stop acquiring the target tasks and start processing the acquired target tasks. Alternatively, when the number of target tasks acquired by the execution node 120 does not meet the predetermined number, but the target node labels of all pending tasks in the task queue that the execution node 120 has read are different from the node labels of the execution node 120, the execution node 120 can also stop acquiring the target tasks and start processing the acquired target tasks.

[0111] In order to ensure timely and smooth processing of pending tasks, a backup node can be set for the execution node 120. Therefore, when the execution node 120 fails and cannot normally obtain and process the pending tasks, the backup node can be used to complete the acquisition and processing of the pending tasks.

[0112] Based on this, optionally, in a specific implementation, the number of the above-mentioned task queues is one, and there are multiple execution nodes 120 with the same node label among the multiple execution nodes 120.

[0113] Among them, the multiple execution nodes 120 with the same node label include a main execution node and at least one backup execution node. That is, in this specific implementation, an execution node group may include a main execution node and at least one backup execution node.

[0114] Accordingly, in this specific implementation, for an execution node group including a main execution node and at least one backup execution node, when the main execution node included in the execution node group operates normally, the main execution node is used to obtain the target task from the task queue and process the obtained target task.

[0115] When the main execution node included in the execution node group cannot operate normally, a backup execution node included in the execution node group is started, and the backup execution node is used to obtain the target task from the task queue and process the obtained target task.

[0116] Among them, when the execution node group includes multiple backup execution nodes, the priority of each backup execution node can be set in advance. Thus, when the main execution node cannot operate normally, the backup execution node with the highest priority among the backup execution nodes that can operate normally is started. The started execution node is used to obtain the target task from the task queue and process the obtained target task.

[0117] In order to better understand a task scheduling system provided by an embodiment of the present invention, Figure 7 A specific embodiment is shown for description.

[0118] in, Figure 7 The task message in is the pending task obtained by the dispatching node, the node group attribute is the target node label specified for the pending task, Queue is the task queue, and the job execution machine is the execution node. The node label of each job execution machine in the node group G1 is id=1, and the node label of each job execution machine in the node group G2 is id=2.

[0119] There are 4 task queues, and JOB1-JOB4 are pending tasks belonging to task queue 1-task queue 4. The target node labels of JOB1-JOB4 are ("groupID", 1), ("groupID", 2), ("groupID", 1), and ("groupID", 2).

[0120] Then the job execution machines 1-4 can obtain JOB1-JOB4 respectively from their corresponding task queues 1-4.

[0121] Corresponding to the task scheduling system provided by the above embodiment of the present invention, the embodiment of the present invention further provides a task scheduling method, wherein the method is applied to a dispatching node in the task scheduling system.

[0122] Figure 2 A flowchart of a task scheduling method for dispatching nodes provided by an embodiment of the present invention is shown in FIG. Figure 2 As shown, the method may include the following steps:

[0123] S201: Obtain the task to be processed and the target node label;

[0124] S202: Add a target node label to the task to be processed;

[0125] S203: dispatching the to-be-processed task with the target node tag added to the task queue, so that each execution node in the task scheduling system obtains the target task from the task queue and processes the target task;

[0126] The target task acquired by each execution node is: a task to be processed whose target node label is the same as the node label of the execution node.

[0127] Based on this, by applying the solution provided by the embodiment of the present invention, the tasks to be processed and each execution node can be classified by node labels, wherein the tasks to be processed with a certain node label will be acquired by the execution node with the same node label and processed by the execution node. In other words, each type of execution node will process a preset type of tasks to be processed, that is, the execution nodes that process each type of tasks to be processed are determined, so that each type of tasks to be processed will be processed by a preset type of execution node, thereby improving the security of processing tasks.

[0128] Optionally, in a specific implementation, the task queue is a rocketmq message queue; the information type of the task to be processed is a rocketmq message; the above step S202, adding a target node label to the task to be processed, may include the following steps:

[0129] Set the property value of the task to be processed to the target node label.

[0130] Corresponding to the task scheduling system provided by the above embodiment of the present invention, the embodiment of the present invention further provides a task scheduling method, wherein the method is applied to each execution node in the task scheduling system.

[0131] Figure 3 A flowchart of a task scheduling method applied to each execution node provided by an embodiment of the present invention is shown in FIG. Figure 3 As shown, the method may include the following steps:

[0132] S301: Obtain a target task from a task queue;

[0133] Among them, each pending task in the task queue is: the pending task with the target node tag added to the task queue dispatched by the dispatching node in the task scheduling system after obtaining the pending task and the target node tag, and the target task is: the pending task with the target node tag being the same as the node tag of the execution node;

[0134] S302: Processing target tasks.

[0135] Based on this, by applying the solution provided by the embodiment of the present invention, the tasks to be processed and each execution node can be classified by node labels, wherein the tasks to be processed with a certain node label will be acquired by the execution node with the same node label and processed by the execution node. In other words, each type of execution node will process a preset type of tasks to be processed, that is, the execution nodes that process each type of tasks to be processed are determined, so that each type of tasks to be processed will be processed by a preset type of execution node, thereby improving the security of processing tasks.

[0136] Optionally, in a specific implementation, there are multiple task queues, and there are execution nodes with the same node label among the multiple execution nodes, wherein the execution nodes with node labels different from any other execution nodes correspond to all task queues, and among the execution nodes with the same node label, different execution nodes correspond to different task queues;

[0137] The above step S301, obtaining the target task from the task queue, may include the following steps:

[0138] Get the target task from the task queue corresponding to the execution node.

[0139] Optionally, in a specific implementation, the step of obtaining the target task from the task queue corresponding to the execution node may include the following steps:

[0140] When there are multiple task queues corresponding to the execution node, each task queue corresponding to the execution node is traversed in a loop according to a predetermined queue traversal order;

[0141] For each task queue traversed, if the task queue includes the target task and the number of target tasks included is not less than the specified number, then obtain the specified number of target tasks from the task queue in order of task dispatch time from early to late, and traverse the next task queue; if the task queue includes the target task and the number of target tasks included is less than the specified number, obtain the target tasks included in the task queue in order of task dispatch time from early to late, and traverse the next task queue; if the task queue does not include the target task, directly traverse the next task queue.

[0142] Optionally, in a specific implementation, the above step S301, obtaining from the task queue the to-be-processed task whose added target node label is the node label of the execution node, may include the following steps:

[0143] Get the target task from the task queue through the preset filtering method.

[0144] Corresponding to the task scheduling system provided by the above embodiment of the present invention, the embodiment of the present invention further provides a task scheduling device, wherein the device is applied to a dispatching node in the task scheduling system.

[0145] Figure 4 A schematic diagram of a task scheduling device for dispatching nodes provided in an embodiment of the present invention is shown in FIG. Figure 4 As shown, the device may include the following modules:

[0146] Information acquisition module 410, used to obtain tasks to be processed and target node labels;

[0147] The label adding module 420 is used to add a target node label to the task to be processed;

[0148] The task dispatching module 430 is used to dispatch the to-be-processed tasks with target node tags added to the task queue, so that each execution node in the task scheduling system obtains the target task from the task queue and processes the target task.

[0149] Based on this, by applying the solution provided by the embodiment of the present invention, the tasks to be processed and each execution node can be classified by node labels, wherein the tasks to be processed with a certain node label will be acquired by the execution node with the same node label and processed by the execution node. In other words, each type of execution node will process a preset type of tasks to be processed, that is, the execution nodes that process each type of tasks to be processed are determined, so that each type of tasks to be processed will be processed by a preset type of execution node, thereby improving the security of processing tasks.

[0150] Optionally, in a specific implementation, the task queue is a rocketmq message queue; the information type of the task to be processed is a rocketmq message; the label adding module 420 is specifically used to set the attribute value of the property attribute of the task to be processed to the target node label.

[0151] Corresponding to the task scheduling system provided by the above embodiment of the present invention, the embodiment of the present invention further provides a task scheduling device, wherein the device is applied to each execution node in the task scheduling system.

[0152] Figure 5 A schematic diagram of a task scheduling device applied to each execution node provided by an embodiment of the present invention is shown in FIG. Figure 5 As shown, the device may include the following modules:

[0153] The task acquisition module 510 is used to acquire the target task from the task queue;

[0154] Among them, each pending task in the task queue is: the pending task with the target node tag added to the task queue dispatched by the dispatching node in the task scheduling system after obtaining the pending task and the target node tag, and the target task is: the pending task with the target node tag being the same as the node tag of the execution node;

[0155] The task processing module 520 is used to process the target task.

[0156] Based on this, by applying the solution provided by the embodiment of the present invention, the tasks to be processed and each execution node can be classified by node labels, wherein the tasks to be processed with a certain node label will be acquired by the execution node with the same node label and processed by the execution node. In other words, each type of execution node will process a preset type of tasks to be processed, that is, the execution nodes that process each type of tasks to be processed are determined, so that each type of tasks to be processed will be processed by a preset type of execution node, thereby improving the security of processing tasks.

[0157] Optionally, in a specific implementation, there are multiple task queues, and there are execution nodes with the same node label among the multiple execution nodes, wherein the execution node with a node label different from any other execution node corresponds to all task queues, and among the execution nodes with the same node label, different execution nodes correspond to different task queues; the task acquisition module 510 includes:

[0158] The task acquisition submodule is used to obtain the target task from the task queue corresponding to the execution node.

[0159] Optionally, in a specific implementation, the task acquisition submodule is specifically used to:

[0160] When there are multiple task queues corresponding to the execution node, each task queue corresponding to the execution node is traversed in a loop according to a predetermined queue traversal order;

[0161] For each task queue traversed, if the task queue includes the target task and the number of target tasks included is not less than the specified number, then obtain the specified number of target tasks from the task queue in order of task dispatch time from early to late, and traverse the next task queue; if the task queue includes the target task and the number of target tasks included is less than the specified number, obtain the target tasks included in the task queue in order of task dispatch time from early to late, and traverse the next task queue; if the task queue does not include the target task, directly traverse the next task queue.

[0162] Optionally, in a specific implementation, the task acquisition module is specifically used to:

[0163] Get the target task from the task queue through the preset filtering method.

[0164] Corresponding to a task scheduling system provided by an embodiment of the present invention, an embodiment of the present invention further provides an electronic device, such as Figure 6 As shown, it includes a processor 601, a communication interface 602, a memory 603 and a communication bus 604, wherein the processor 601, the communication interface 602, and the memory 603 communicate with each other through the communication bus 604.

[0165] Memory 603, used for storing computer programs;

[0166] Processor 601 is used to implement the steps of any task scheduling method provided by the above-mentioned embodiments of the present invention for a dispatching node in a task scheduling system when executing the program stored in memory 603, or the steps of any task scheduling method provided by the above-mentioned embodiments of the present invention for each execution node in a task scheduling system.

[0167] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0168] The communication interface is used for communication between the above electronic device and other devices.

[0169] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0170] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0171] In another embodiment provided by the present invention, a computer-readable storage medium is also provided, which stores a computer program. When the computer program is executed by a processor, it implements the steps of the task scheduling method for a dispatch node in a task scheduling system provided by any of the above-mentioned embodiments of the present invention, or the steps of the task scheduling method for each execution node in a task scheduling system provided by any of the above-mentioned embodiments of the present invention.

[0172] In another embodiment provided by the present invention, a computer program product comprising instructions is also provided. When the computer is run on a computer, the computer executes the steps of any task scheduling method provided by the above-mentioned embodiments of the present invention and applied to a dispatch node in a task scheduling system, or the steps of any task scheduling method provided by the above-mentioned embodiments of the present invention and applied to each execution node in a task scheduling system.

[0173] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0174] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0175] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for method embodiments, device embodiments, electronic device embodiments, computer-readable storage medium embodiments, and computer program product embodiments, since they are basically similar to system embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0176] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A task scheduling system, characterized in that: The system includes a dispatch node and a plurality of execution nodes, and each execution node has a node label; The dispatch node is used to obtain the pending task and the target node tag; add the target node tag to the pending task, and dispatch the pending task with the target node tag added to the task queue; Each execution node is used to obtain a target task from the task queue and process the target task, wherein the target task is: a task to be processed whose target node label is the same as the node label of the execution node; There are multiple task queues, and there are execution nodes with the same node label among the multiple execution nodes, wherein the execution nodes with node labels different from any other execution nodes correspond to all task queues, and among the execution nodes with the same node label, different execution nodes correspond to different task queues; the task queues corresponding to each execution node with the same node label are: allocated according to the processing capacity of the execution node; Each execution node is specifically used to: obtain a target task from a task queue corresponding to the execution node, and process the target task; The step of obtaining the target task from the task queue corresponding to the execution node includes: When there are multiple task queues corresponding to the execution node, each task queue corresponding to the execution node is traversed in a loop according to a predetermined queue traversal order; For each task queue traversed, if the task queue includes the target task and the number of target tasks included is not less than the specified number, then obtain the specified number of target tasks from the task queue in order of task dispatch time from early to late, and traverse the next task queue; if the task queue includes the target task and the number of target tasks included is less than the specified number, obtain the target tasks included in the task queue in order of task dispatch time from early to late, and traverse the next task queue; if the task queue does not include the target task, directly traverse the next task queue.

2. The system according to claim 1, characterized in that The task queue is a rocketmq message queue; the information type of the task to be processed is a rocketmq message; and the target node label is added to the task to be processed, including: The attribute value of the property attribute of the task to be processed is set to the target node label.

3. The system according to claim 2, characterized in that The step of obtaining a target task from the task queue includes: Obtain the target task from the task queue through a preset filtering method.

4. A task scheduling method, characterized in that: Applied to a dispatching node in a task scheduling system, the method comprises: Get pending tasks and target node labels; Adding the target node label to the task to be processed; Dispatching the pending task with the target node label added to the task queue, so that each execution node in the task scheduling system obtains the target task from the task queue and processes the target task; wherein the target task obtained by each execution node is: the pending task with the target node label being the same as the node label of the execution node; There are multiple task queues, and there are execution nodes with the same node label among the multiple execution nodes, wherein the execution nodes with node labels different from any other execution nodes correspond to all task queues, and among the execution nodes with the same node label, different execution nodes correspond to different task queues; the task queues corresponding to each execution node with the same node label are: allocated according to the processing capacity of the execution node; The step of obtaining the target task from the task queue comprises the following steps: obtaining the target task from the task queue corresponding to the execution node; The step of obtaining the target task from the task queue corresponding to the execution node comprises the following steps: When there are multiple task queues corresponding to the execution node, each task queue corresponding to the execution node is traversed in a loop according to a predetermined queue traversal order; For each task queue traversed, if the task queue includes the target task and the number of target tasks included is not less than the specified number, then obtain the specified number of target tasks from the task queue in order of task dispatch time from early to late, and traverse the next task queue; if the task queue includes the target task and the number of target tasks included is less than the specified number, obtain the target tasks included in the task queue in order of task dispatch time from early to late, and traverse the next task queue; if the task queue does not include the target task, directly traverse the next task queue.

5. A task scheduling method, characterized in that: Applied to each execution node in the task scheduling system, the method comprises: Obtain the target task from the task queue; wherein each pending task in the task queue is: the pending task with the target node tag added thereto dispatched to the task queue by the dispatching node in the task scheduling system after obtaining the pending task and the target node tag, and the target task is: the pending task with the target node tag being the same as the node tag of the execution node; Processing the target task; There are multiple task queues, and there are execution nodes with the same node label among the multiple execution nodes, wherein the execution nodes with node labels different from any other execution nodes correspond to all task queues, and among the execution nodes with the same node label, different execution nodes correspond to different task queues; the task queues corresponding to each execution node with the same node label are: allocated according to the processing capacity of the execution node; The step of obtaining the target task from the task queue comprises the following steps: obtaining the target task from the task queue corresponding to the execution node; The step of obtaining the target task from the task queue corresponding to the execution node comprises the following steps: When there are multiple task queues corresponding to the execution node, each task queue corresponding to the execution node is traversed in a loop according to a predetermined queue traversal order; For each task queue traversed, if the task queue includes the target task and the number of target tasks included is not less than the specified number, then obtain the specified number of target tasks from the task queue in order of task dispatch time from early to late, and traverse the next task queue; if the task queue includes the target task and the number of target tasks included is less than the specified number, obtain the target tasks included in the task queue in order of task dispatch time from early to late, and traverse the next task queue; if the task queue does not include the target task, directly traverse the next task queue.

6. A task scheduling device, characterized in that: Applied to a dispatching node in a task scheduling system, the device comprises: Information acquisition module, used to obtain pending tasks and target node labels; A label adding module, used for adding the target node label to the task to be processed; The task dispatching module is used to dispatch the pending task with the target node label added to the task queue, so that each execution node in the task scheduling system obtains the target task from the task queue and processes the target task; wherein the target task obtained by each execution node is: the pending task with the target node label being the same as the node label of the execution node; There are multiple task queues, and there are execution nodes with the same node label among the multiple execution nodes, wherein the execution nodes with node labels different from any other execution nodes correspond to all task queues, and among the execution nodes with the same node label, different execution nodes correspond to different task queues; the task queues corresponding to each execution node with the same node label are: allocated according to the processing capacity of the execution node; The step of obtaining the target task from the task queue comprises the following steps: obtaining the target task from the task queue corresponding to the execution node; The step of obtaining the target task from the task queue corresponding to the execution node comprises the following steps: When there are multiple task queues corresponding to the execution node, each task queue corresponding to the execution node is traversed in a loop according to a predetermined queue traversal order; For each task queue traversed, if the task queue includes the target task and the number of target tasks included is not less than the specified number, then obtain the specified number of target tasks from the task queue in order of task dispatch time from early to late, and traverse the next task queue; if the task queue includes the target task and the number of target tasks included is less than the specified number, obtain the target tasks included in the task queue in order of task dispatch time from early to late, and traverse the next task queue; if the task queue does not include the target task, directly traverse the next task queue.

7. A task scheduling device, characterized in that: Applied to each execution node in the task scheduling system, the device comprises: The task acquisition module is used to acquire the target task from the task queue; wherein each pending task in the task queue is: the pending task with the target node tag added thereto is dispatched to the task queue by the dispatching node in the task scheduling system after acquiring the pending task and the target node tag, and the target task is: the pending task with the target node tag being the same as the node tag of the execution node; A task processing module, used for processing the target task; There are multiple task queues, and there are execution nodes with the same node label among the multiple execution nodes, wherein the execution nodes with node labels different from any other execution nodes correspond to all task queues, and among the execution nodes with the same node label, different execution nodes correspond to different task queues; the task queues corresponding to each execution node with the same node label are: allocated according to the processing capacity of the execution node; The task acquisition module includes: a task acquisition submodule, which is used to acquire the target task from the task queue corresponding to the execution node; The task acquisition submodule is specifically used to loop through each task queue corresponding to the execution node according to a predetermined queue traversal order when there are multiple task queues corresponding to the execution node; For each task queue traversed, if the task queue includes the target task and the number of target tasks included is not less than the specified number, then obtain the specified number of target tasks from the task queue in order of task dispatch time from early to late, and traverse the next task queue; if the task queue includes the target task and the number of target tasks included is less than the specified number, obtain the target tasks included in the task queue in order of task dispatch time from early to late, and traverse the next task queue; if the task queue does not include the target task, directly traverse the next task queue.

8. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; The processor is used to implement the method steps described in claim 4 or 5 when executing the program stored in the memory.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in claim 4 or 5 are implemented.

Images