Task scheduling method and device, computer device and computer readable storage medium

Abstract

Embodiments of the present application disclose a task scheduling method and device, computer equipment and a computer readable storage medium. The task data set is obtained, the task data set includes at least one target type task data; the target task data is determined from the task data set, and the distributed node is determined. The task program based on the task data is deployed to execute the target type task, and the target task data is the task data corresponding to the target type task. The task execution state information of the distributed node is obtained; the idle node is filtered according to the task execution state information, and the idle node is the node currently not running the target type task; the target task data is sent to the idle node, so that the idle node processes the target task data. The application can realize accurate allocation of target task data to nodes and improve the utilization efficiency of distributed nodes. The embodiments of the present application can be applied to cloud technology, artificial intelligence, intelligent transportation, auxiliary driving and various scenes.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of communication, in particular to a task scheduling method and device, computer equipment and computer readable storage medium. BACKGROUND

[0002] Distributed task scheduling refers to the process of scheduling tasks under distributed nodes. The method commonly used to determine the node for executing a task from the distributed nodes is to predict the time required by the distributed node to execute the current scheduled task through a task completion time prediction model, assign the current scheduled task to the node requiring the shortest time, and pull up the next task at a time according to the predicted time. There is a certain gap between the completion time predicted by the task completion time prediction model and the actual completion time, so that the current task has not been completed on the node, while the node has pulled up another task, or the node has not pulled up another task after receiving the current task for a period of time, resulting in that the resources of the distributed node are not reasonably utilized. SUMMARY

[0003] The embodiments of the present application provide a task scheduling method, device, computer equipment and computer readable storage medium, which can realize accurate allocation of target task data to nodes and improve the utilization efficiency of distributed nodes.

[0004] The embodiments of the present application provide a task scheduling method, which comprises:

[0005] Obtaining a task data set, the task data set comprising task data of at least one target type task;

[0006] Determining target task data to be scheduled from the task data set, and determining a distributed node for processing the target task data, the distributed node being deployed with a task program for executing the target type task based on the task data, the target task data being task data corresponding to the target type task;

[0007] Obtaining task execution state information of the distributed node;

[0008] Screening an idle node from the distributed node according to the task execution state information of the distributed node, the idle node being a distributed node that is not currently running the target type task;

[0009] Sending the target task data to the idle node, so that the idle node processes the target task data through the task program.

[0010] Correspondingly, the embodiments of the present application also provide a task scheduling device, which comprises:

[0011] acquire a task data set, the task data set comprising task data of at least one target type task;

[0012] determine, from the task data set, target task data to be scheduled, and determine a distributed node for processing the target task data, the distributed node being deployed with a task program for performing the target type task based on the task data, the target task data being task data corresponding to the target type task;

[0013] acquire task execution state information of the distributed node;

[0014] select, from the distributed node, an idle node according to the task execution state information of the distributed node, the idle node being a distributed node that is not currently running the target type task;

[0015] send the target task data to the idle node, so that the idle node processes the target task data through the task program.

[0016] Correspondingly, the embodiment of the application further provides a computer device, comprising a memory and a processor; the memory stores a computer program, and the processor is configured to execute the computer program in the memory to perform any one of the task scheduling methods provided by the embodiments of the application.

[0017] Correspondingly, the embodiment of the application further provides a computer readable storage medium, configured to store a computer program, the computer program being loaded by a processor to execute any one of the task scheduling methods provided by the embodiments of the application.

[0018] The embodiment of the application can acquire a task data set, the task data set comprising task data of at least one target type task; determine, from the task data set, target task data to be scheduled, and determine a distributed node for processing the target task data, the distributed node being deployed with a task program for performing the target type task based on the task data, the target task data being task data corresponding to the target type task; acquire task execution state information of the distributed node; select, from the distributed node, an idle node according to the task execution state information of the distributed node, the idle node being a distributed node that is not currently running the target type task; and send the target task data to the idle node, so that the idle node processes the target task data through the task program. The scheme can accurately allocate target task data to a node and improve the utilization efficiency of the distributed node by determining an idle node from the distributed node based on the task execution state information of the distributed node. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort based on these drawings.

[0020] Figure 1 is a scenario diagram of the task scheduling method provided by the embodiments of the present application;

[0021] Figure 2 is a flowchart of the task scheduling method provided by the embodiments of the present application;

[0022] Figure 3 is another flowchart of the task scheduling method provided by the embodiments of the present application;

[0023] Figure 4 is a schematic diagram of the task scheduling device provided by the embodiments of the present application;

[0024] Figure 5 is a structural schematic diagram of the computer device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0025] The technical solutions in the embodiments of the present application will be described clearly and completely in the following description with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present application.

[0026] The embodiments of the present application provide a task scheduling method, device, computer device and computer readable storage medium. The task scheduling device can be integrated in a computer device, which can be a server or a terminal device.

[0027] The terminal can include but is not limited to a mobile phone, a wearable smart device, a tablet computer, a notebook computer, a smart voice interaction device, a smart home appliance, a personal computer (PC), and a vehicle-mounted computer.

[0028] The server can be a standalone physical server, a server cluster or a distributed system composed of multiple physical servers, a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, and basic cloud computing services such as big data and artificial intelligence platforms.

[0029] For example, as shown inFigure 1 As shown, the server obtains a task data set to be scheduled, the task data set can contain task data required for execution of multiple tasks, determines target task data to be currently scheduled from the task data set, and confirms a distributed node for processing the target task data, the distributed node is deployed with a task program for executing a target type task based on the task data, and the target task data is task data corresponding to the target type task; task execution state information of the distributed node is obtained, an idle node is screened from the distributed node according to the task execution state information, and the target task data is sent to the idle node, so that the idle node processes the target task data based on the deployed task program, and the scheme can realize accurate allocation of the target task data to the node and improve utilization efficiency of the distributed node by determining the idle node from the distributed node based on the task execution state information of the distributed node.

[0030] The following will be described in detail. It should be noted that the order of the following embodiments is not limited as the preferred order of the embodiments.

[0031] This embodiment will be described from the perspective of a task scheduling device, which can be integrated in a computer device, which can be a server, a terminal or the like. The task scheduling method provided by the embodiments of the present application can be as follows: Figure 2 As shown, the specific process of the task scheduling method can be as follows:

[0032] 101, obtain a task data set, the task data set including task data of at least one target type task.

[0033] The task data set can include task data of at least one target type task and task data corresponding to other types of tasks, for example, the task data can include task parameters required for task execution, etc., for example, the task parameters can include request mode, expected result and task execution duration, etc. when executing a task program to execute an interface performance test task, and the task execution duration is a test for a specified duration of interface performance execution.

[0034] The task can implement corresponding functions or provide corresponding services by executing a program (or a group of programs).

[0035] For example, the task data set can be obtained from a memory or from a blockchain, and the task data set can be stored in a database or saved in a cache, a file or a report.

[0036] 102. determining target task data to be scheduled from the task data set, and determining a distributed node for processing the target task data, the distributed node being deployed with a task program for performing a target type task based on the task data, the target task data being task data corresponding to the target type task.

[0037] The target task data can be task data required by the task to be scheduled.

[0038] The distributed node can be a computer device such as a server, or a device cluster, etc. The distributed node is deployed with a task program, and can process task data based on the task program to implement corresponding functions or provide corresponding services.

[0039] For example, a task data can be randomly determined from the task data set as the target task data set. Alternatively, the earliest task data in time sequence can be determined from the task data set as the target task data according to the time sequence of task data submission. Alternatively, the target task data can be determined from the task data set according to the arrangement order of the task data in the task data set.

[0040] In an embodiment, the task data can correspond to a task identifier, and the corresponding task data can be determined from the task data set as the target task data to be scheduled according to the task identifier, i.e., the step of "determining target task data to be scheduled from the task data set". Specifically, the step can include:

[0041] determining a target task identifier from the task identifier set according to a preset strategy;

[0042] obtaining the target task data to be scheduled from the task data set according to the target task identifier.

[0043] The task identifier set can include at least one task identifier, and the task identifier can be used to indicate the corresponding task data. The task identifier set can be in the form of a task identifier list or a task identifier queue, etc.

[0044] The preset strategy can be a strategy for determining the target task identifier from the task identifier set. The preset strategy can be based on time sequence, or based on arrangement order, etc.

[0045] For example, the task identifier stored earlier in time sequence than the task identifier stored later in the task identifier set can be determined as the target task identifier according to the time sequence. Alternatively, the task identifier stored later in time sequence than the task identifier stored earlier in the task identifier set can be determined as the target task identifier according to the time sequence. Alternatively, the task identifier stored at a specified time can be determined as the target task identifier.

[0046] According to the correspondence between the task identifier and the task data, target task data corresponding to the target task identifier is queried from the task data set.

[0047] According to the submission order, the task data generates a corresponding task identifier, and according to the task identifier, the execution order of the task can be determined, which can improve the accuracy of the order of the task data allocation.

[0048] According to the time or order of the task data submission, the task data generates a corresponding task identifier, and according to the task identifier, the order of the task submission can be more conveniently determined; and the task identifier is generated for the task data, which facilitates the tracing of the task data based on the task identifier, and the tracing of the execution result obtained after the task program processes the task data indicated by the task identifier.

[0049] When there are multiple task identifier sets, different task identifier sets can correspond to different priorities, and the task identifier set with high priority can be processed preferentially, which improves the flexibility of task scheduling, and different priorities can be set according to the needs of application scenarios to adapt to different scenarios, that is, the step "determining a target task identifier from the task identifier set table according to a preset strategy" can specifically include:

[0050] According to the priority, a target task identifier set is determined from at least one task identifier set;

[0051] According to the acquisition order of the task identifier, a target task identifier is filtered from the target task identifier set.

[0052] For example, the priority of at least one task identifier set can be determined to determine the task identifier set with the highest priority as the target task identifier set.

[0053] The target task identifier set can be a set obtained by sorting according to the acquisition order of the task identifier, and the target task identifier can be filtered from the target task identifier set according to the acquisition order of the task identifier, for example, the task identifier with the earliest acquisition order in the target task identifier set is taken as the target task identifier.

[0054] The task identifiers in the task identifier set are acquired in sequence according to the priority order, for example, after the target task data corresponding to the task identifier in the task identifier set with the first priority is scheduled, the target task data corresponding to the task identifier in the task identifier set with the second priority is scheduled, and so on.

[0055] When receiving newly added task data, a task identifier can be generated for the newly added task data based on the task identifier set, and the task identifier is stored in the task identifier set, that is, before the step "obtaining a task data set to be scheduled", the task scheduling method further includes:

[0056] obtain an initial task identifier set corresponding to the candidate task data and the initial task data set;

[0057] generate a task identifier corresponding to the candidate task data according to the task identifier in the initial task identifier set;

[0058] add the task identifier of the candidate task data to the initial task identifier set to obtain a task identifier set;

[0059] add the candidate task data to the initial task data set to obtain a task data set.

[0060] The candidate task data can include newly added task data, the initial task data set can be a set that does not include the candidate task data, and the initial task identifier set can be a set that does not include the task identifier corresponding to the candidate task data.

[0061] For example, the task data can be received from a web application corresponding to the task scheduling method, or the task data can be received in a predetermined message format. The task identifier corresponding to the candidate task data can be generated according to the submission time of the task data, and the order of submission of the task data can be determined according to the task identifier, or the task identifier corresponding to the task data can be generated according to the submission order of the task data. For example, the task identifier corresponding to the first submitted task data can be 1, the task identifier corresponding to the second submitted task data can be 2, and so on.

[0062] The generated task identifier is added to the initial task identifier set to obtain a task identifier set, and the candidate task data set is added to the initial task data set to obtain a task data set.

[0063] 103、obtain task execution state information of the distributed nodes.

[0064] The task execution state information can include the current running task program of the distributed node, such as the currently executed task program.

[0065] For example, the distributed node can send its task execution state information to the server at regular intervals, or the server can send an instruction to the distributed node to obtain the task execution state information from the distributed node.

[0066] 104、screen idle nodes from the distributed nodes according to the task execution state information of the distributed nodes, the idle nodes being distributed nodes that are not currently running a target type task.

[0067] The idle node can be a distributed node that does not currently run a target type task, for example, if the data task program for processing the target task is a program for performing a performance test task, the idle node is a node that does not currently perform a task or does not perform a performance test task.

[0068] For example, the idle node can be determined from the distributed nodes according to the task execution state information, and the node is the idle node.

[0069] Compared with the method of predicting the completion time of a task based on a task completion time prediction model, and then determining the next task according to the predicted completion time, the node performing the task is determined according to the task execution state information of the distributed node, the target task data can be sent to the idle node in time, the use efficiency of the node is improved, and the node does not simultaneously perform the same task due to a prediction time deviation, and the execution effect is not accurate.

[0070] To reduce the amount of calculation, the target node can be determined from the distributed nodes, and the task execution state information of the target node is determined to determine whether the target node is an idle node, that is, in an embodiment, the step of "screening an idle node from the distributed nodes according to the task execution state information of the distributed nodes" can specifically include:

[0071] The target node is determined from the distributed nodes.

[0072] If the task execution state information indicates that the target node currently performs a task, the task data of the currently performed task is obtained.

[0073] If the task data of the currently performed task and the target task data are in a non-mutual exclusion relationship, the target node is determined to be an idle node.

[0074] For example, the target node can be determined from the distributed nodes according to the remaining state of the CPU, memory or IO resource of the distributed node, or the target node can be determined from the distributed nodes according to a preset order.

[0075] The task execution state information of the target node is determined to determine whether the target node is an idle node, if the task execution state information indicates that the target node is performing a task, if the task data of the currently performed task and the target task data are independent of each other, that is, in a non-mutual exclusion relationship, the task program corresponding to the currently performed task and the task for performing the target task data are not the same task program, and the task data of the currently performed task and the target task data are independent of each other, and can be processed simultaneously in the same node, the target node is determined to be an idle node.

[0076] Optionally, the task name of the current execution task and the task name in the target task data can be acquired, and if the task name of the current execution task is different from the task name in the target task data, it indicates that the task program corresponding to the current execution task of the target node is not the task program for executing the target task data.

[0077] When the target node is not the idle node, the idle state of the associated node of the target node can be determined, that is, in an embodiment, the task scheduling method can further include:

[0078] If the task data of the current execution task and the target task data are in a mutual exclusion relationship, the associated node of the target node is taken as the target node, and the task data of the current execution task is acquired until all the nodes in the distributed node are traversed.

[0079] After the step "if the task execution state information indicates that the target node has the current execution task, the task data of the current execution task is acquired", the method can further include:

[0080] If the target node has no current execution task, it is determined that the target node is an idle node.

[0081] The associated node can be the next node of the target node in a preset order, or a node determined according to the CPU, memory or IO resources of the node.

[0082] For example, if the task program corresponding to the current execution task and the task for executing the target task data are the same task program, that is, the task data of the current execution task and the target task data are in a mutual exclusion relationship and cannot be processed in the same node, it is determined that the target node is not an idle node.

[0083] Optionally, the task name of the current execution task and the task name in the target task data can be acquired, and if the task name of the current execution task is different from the task name in the target task data, it indicates that the task program corresponding to the current execution task of the target node is not the task program for executing the target task data.

[0084] If the task data of the current execution task and the target task data are in a mutual exclusion relationship, the associated node of the target node is taken as the target node, and the task data of the current execution task is acquired until all the nodes in the distributed node are traversed.

[0085] If the task execution state information indicates that the target node has no current execution task, it is determined that the target node is an idle node.

[0086] If traversing all nodes in the distributed nodes reveals no idle nodes, idle nodes in the distributed nodes can be identified again after a period of time. In one embodiment, the task scheduling method may further include:

[0087] If it is determined through traversing the distributed nodes that there are no idle nodes, then the waiting conditions are determined according to the preset waiting strategy.

[0088] When the waiting conditions are met, return to determine the target node from the distributed nodes. If the task execution status information indicates that there is a currently executing task on the target node, then obtain the task data of the currently executing task.

[0089] The preset waiting strategy can provide fixed waiting conditions, such as a fixed waiting time. When there are no idle nodes in the distributed nodes, wait 30 seconds and then search for an idle node again. Alternatively, the waiting conditions can be determined based on the gradient. For example, when there are no idle nodes in the distributed nodes, wait 100 seconds and then search for an idle node again. If there are still no idle nodes, wait 50 seconds and then search for an idle node again, and so on, decreasing the waiting time each time.

[0090] The waiting condition can be a waiting time, such as waiting for 30 seconds, or a specified time, such as 21:00, or other conditions.

[0091] For example, if it is determined that there are no idle nodes among the distributed nodes after traversing the distributed nodes, the waiting condition is set to wait for 30 seconds according to the preset waiting strategy. After waiting for 30 seconds, the process of determining an idle node from the distributed nodes is executed again.

[0092] Adding a scanning mechanism to detect the task execution status of distributed nodes can identify idle nodes in the distributed nodes in real time. Compared with the method of predicting the completion time by model and starting the task at regular intervals, it can accurately allocate the target task data to idle nodes, and has better flexibility and higher node utilization efficiency. At the same time, it avoids the same task being repeatedly executed on the same node within the same short period of time due to inaccurate prediction time.

[0093] 105. Send the target task data to the idle node so that the idle node can process the target task data through the task program.

[0094] For example, the target task data could be sent to an idle node, which would then determine the corresponding task program based on the target task data and execute that program to process the target task data.

[0095] From the above, the embodiment of the application can obtain a task data set, the task data set including task data of at least one target type task; determine target task data to be scheduled from the task data set, and determine a distributed node for processing the target task data, the distributed node being deployed with a task program for executing a target type task based on the task data, the target task data being task data corresponding to the target type task; obtain task execution state information of the distributed node; filter an idle node from the distributed node according to the task execution state information of the distributed node, the idle node being a distributed node that is currently not running the target type task; and send the target task data to the idle node, so that the idle node processes the target task data through the task program. The scheme determines an idle node from the distributed node based on the task execution state information of the distributed node, which can accurately allocate target task data to a node and improve the utilization efficiency of the distributed node. The embodiment of the application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, intelligent transportation, and various scenarios such as assisted driving.

[0096] On the basis of the above embodiment, further detailed description will be made by way of example.

[0097] The embodiment will be described from the perspective of a task scheduling device, which can be integrated in a server, which can be a service cluster. The server cluster can include a task scheduling node and a distributed node (i.e., a task execution node), and the task scheduling node can be used to allocate a target task to the distributed node.

[0098] The embodiment of the application provides a task scheduling method, as shown in Figure 3 The specific process of the task scheduling method can be as follows:

[0099] 201. The task scheduling node determines a target task identifier from the task identifier queue, and obtains target task data from the database according to the target task identifier.

[0100] The task identifier queue can be a queue adopting a first-in first-out strategy, and the task data corresponding to the task identifier submitted first is scheduled first. The task identifier queue can include at least one task identifier. The task identifier can be generated according to the submission order of the task data to indicate the submission order, and the task identifier can uniquely correspond to one task data.

[0101] The first-in first-out strategy (FIFO) can be a data caching method, which allows the earliest task data to be processed first.

[0102] For example, the task scheduling node can determine the target task identifier from the task identifier queue according to the task identifier, or determine the target task identifier according to the arrangement order of the task identifier queue, and obtain the target task data from the database according to the target task identifier.

[0103] When there are multiple task identifier queues, different task identifier queues can correspond to different priorities, and the task identifier queue with a high priority can be processed preferentially, the flexibility of task scheduling is improved, and different priorities can be set according to the needs of application scenarios to adapt to different scenarios.

[0104] For example, the priority of at least one task identifier queue can be determined to determine the task identifier queue with the highest priority as the target task identifier queue.

[0105] The target task identifier queue can be a queue sorted according to the acquisition order of the task identifier, and the target task identifier can be filtered from the target task identifier queue according to the acquisition order of the task identifier, for example, the task identifier with the earliest acquisition order in the target task identifier queue is taken as the target task identifier.

[0106] The task identifiers in the task identifier queue are acquired in sequence according to the priority order, for example, after the target task data corresponding to the task identifier in the task identifier queue with the first priority is scheduled, the target task data corresponding to the task identifier in the task identifier queue with the second priority is scheduled, and so on.

[0107] 202, the task scheduling node determines a distributed node for processing the target task data, and determines a target node from the distributed node.

[0108] For example, the servers in the service cluster except the task scheduling server can be determined as the distributed node, or the distributed node for processing the target task data can be determined according to the node list, and the target node can be determined according to the node identifier in the node list.

[0109] The node list can include the node identifier of the distributed node for processing the target task data, the corresponding distributed node can be found according to the node identifier, and the node list can sort the distributed nodes based on the node identifier.

[0110] 203, the task scheduling node obtains the task execution state information of the target node.

[0111] For example, the task scheduling node can send an instruction to the distributed node through an operation and maintenance tool to obtain the task execution state information from the distributed node, for example, a "service AA status" instruction, AA can indicate the target node.

[0112] The operation and maintenance tool can include SaltStack, ansible, puppet, and the like, and is not limited herein. SaltStack is a centralized management platform for server infrastructure, and has functions of configuration management, remote execution, and detection. SaltStack is divided into a master scheduling node (equivalent to a task scheduling node) and a minios agent node (equivalent to a distributed node). The master node can issue tasks to all minios nodes and receive the results returned by the minios nodes.

[0113] By deploying SaltStack, the task scheduling node can execute a command for detecting a service or a process on the specified distributed node, such as “service xx status”, and the result will be returned to the task scheduling node.

[0114] After receiving the instruction, the target node returns the task execution state information of the target node to the task scheduling node. The task execution state information includes the services and processes currently running on the target node.

[0115] 204. The task scheduling node determines whether the target node has a current execution task according to the state task information. If yes, step 205 is performed; if no, step 206 is performed.

[0116] For example, the distributed node can pull up a task by a process. If the target task node determines that the target task node does not currently run any process or service according to the task execution state information, the target node does not have a current execution task, and the task scheduling node performs step 206.

[0117] If the target task node determines that the target task node currently runs any process or service according to the task execution state information, the target node has a current execution task, and the task scheduling node performs step 205.

[0118] 205. The task scheduling node determines whether the task data of the current execution task of the target node and the target task data are in a mutual exclusion relationship. If yes, step 207 is performed; if no, step 206 is performed.

[0119] For example, the task name of the current execution task and the task name in the target task data can be obtained. If the task names are different, it indicates that the task program corresponding to the current execution task of the target node is not the task program for processing the target task data, and the task data of the current execution task of the target node and the target task data are not in a mutual exclusion relationship, and step 206 is performed.

[0120] If the task names are the same, it indicates that the task program corresponding to the current execution task of the target node is a task program for processing the target task data, and the task data of the current execution task of the target node and the target task data are in mutual exclusion, and step 207 is performed.

[0121] 206, the task scheduling node determines that the target node is an idle node, and distributes the target task data to the idle node, so that the idle node processes the target task data through a task program.

[0122] For example, specifically, the task data of the current execution task of the target node and the target task data are not in mutual exclusion, the task scheduling node determines that the target node is an idle node, and distributes the target task data to the idle node, so that the idle node determines the corresponding task program according to the task name contained in the target task data, and processes the target task data through the task program.

[0123] 207, if the task scheduling node does not traverse the distributed nodes, determines the associated nodes of the target node according to the node list, takes the associated nodes as the target node, and returns to step 203.

[0124] For example, specifically, the task data of the current execution task of the target node and the target task data are in mutual exclusion, the target node is determined to be a non-idle node, the next node in the node list is taken as the target node, and step 203 is returned, until all nodes contained in the distributed nodes are traversed.

[0125] 208, if the task scheduling node traverses the distributed nodes and determines that there is no idle node, waits for a preset waiting time, and returns to step 202.

[0126] For example, specifically, if it is determined that there is no idle node in all nodes contained in the distributed nodes, the idle node in the distributed nodes can be determined again after a preset waiting time, that is, step 202 is returned.

[0127] The preset waiting time can be a fixed waiting time, or a waiting time determined according to a preset waiting strategy, for example, when there is no idle node in the distributed nodes, the idle node is searched again after waiting for 100s, if there is still no idle node, the idle node is searched again after waiting for 50s, if there is still no idle node in the distributed nodes, the idle node is searched again after waiting for 25s, and the waiting time is gradually decreased to speed up the determination of the idle node for processing the target task data from the distributed nodes.

[0128] 209, when the task scheduling node receives the candidate task data, a corresponding task identifier is generated for the candidate task data.

[0129] For example, the task scheduling node can receive task data sent by a web application corresponding to the task scheduling method, or receive task data sent in an agreed message format. A task identifier is generated according to a submission time of the candidate task data, and the task identifier can be used to determine the order in which the task data is submitted, or a task identifier is generated according to the order in which the task data is submitted. For example, the task identifier corresponding to the first submitted task data can be 1, the task identifier corresponding to the second submitted task data can be 2, and so on.

[0130] 210. The task scheduling node saves the task identifier to the task identifier queue and saves the candidate task data to the task data set.

[0131] For example, the generated task identifier can be added to the initial task identifier queue to obtain the task identifier queue, and the candidate task data can be added to the initial task data set in the database to obtain the task data set.

[0132] As can be seen from the above, the task scheduling node according to the embodiments of the present application determines a target task identifier from the task identifier queue, and obtains target task data from the database according to the target task identifier; determines a distributed node that processes the target task data, and determines a target node from the distributed node according to the node list; obtains task execution state information of the target node; the task scheduling node determines whether the target node has a current execution task according to the state task information, if so, determines whether the task data of the current execution task of the target node and the target task data have a mutual exclusion relationship, if so, determines an associated node of the target node according to the node list, takes the associated node as the target node, returns to execute the task execution state information of the target node, if not, determines that the target node is an idle node, and distributes the target task data to the idle node; if not, determines that the target node is an idle node, and distributes the target task data to the idle node; when the task scheduling node receives candidate task data, a corresponding task identifier is generated for the candidate task data; the task scheduling node saves the task identifier to the task identifier queue and saves the candidate task data to the task data set. This scheme can timely distribute target task data to nodes according to task execution state information, improve the utilization efficiency of the nodes, and avoid the same task being repeatedly executed in the same node within the same time due to inaccurate prediction time, so that the nodes do not execute multiple same tasks at the same time, and the execution effect of the task is improved.

[0133] To better implement the task scheduling method provided by the embodiments of the present application, a task scheduling device is also provided in an embodiment. The meanings of the terms are the same as those in the above task scheduling method, and specific implementation details can be referred to the description in the method embodiment.

[0134] The task scheduling apparatus can be integrated in a computer device, as shown in the figure, the task scheduling apparatus can include a data acquisition unit 301, a determination unit 302, an information acquisition unit 303, a screening unit 304 and a scheduling unit 305, specifically as follows: Figure 4

[0135] (1) Data acquisition unit 301: used for acquiring a task data set, the task data set including task data of at least one target type task.

[0136] For example, the task data set can be acquired from the memory or from the blockchain, and the task data set can be stored by means of a database or saved by means of caching, files or reports.

[0137] (2) Determination unit 302: used for determining target task data to be scheduled from the task data set, and determining a distributed node for processing the target task data, the distributed node being deployed with a task program for executing a target type task based on the task data, the target task data being task data corresponding to the target type task.

[0138] For example, one task data can be randomly determined from the task data set as the target task set, optionally, the earliest task data in time sequence can be determined from the task data set as the target task data according to the time sequence of task data submission, and optionally, the target task data can be determined from the task data set according to the arrangement order of the task data in the task data set.

[0139] Optionally, the determination unit 302 can include an identification determination subunit and a data acquisition subunit, specifically:

[0140] The identification determination subunit is used for determining a target task identification from a task identification set according to a preset strategy.

[0141] The data acquisition subunit is used for acquiring target task data to be scheduled from the task data set according to the target task identification.

[0142] For example, the task identification stored in the task identification set earlier in time sequence than the task identification stored later in the task identification set, the target task identification can be determined from the task identification set according to the time sequence, or the target task identification can be determined from the task identification set according to the time sequence, or the target task identification can be determined from the task identification set according to the time sequence.

[0143] According to the correspondence between the task identification and the task data, the target task data corresponding to the target task identification is queried from the task data set.

[0144] ​In an embodiment, the identifier determining subunit can include a set determining module and an identifier screening module, specifically:

[0145] The set determining module is configured to determine a target task identifier set from the at least one task identifier set according to the priority;

[0146] The identifier screening module is configured to screen a target task identifier from the target task identifier set according to the acquisition order of the task identifier.

[0147] For example, the priority of the at least one task identifier set can be determined to determine the task identifier set with the highest priority as the target task identifier set.

[0148] The target task identifier set can be a set obtained by sorting according to the acquisition order of the task identifier, and the target task identifier can be screened from the target task identifier set according to the acquisition order of the task identifier, for example, the task identifier with the earliest acquisition order in the target task identifier set is taken as the target task identifier.

[0149] Optionally, the task scheduling apparatus can further include a candidate task acquisition unit, an identifier generating unit, an identifier set generating unit and a data set generating unit, specifically:

[0150] The task acquisition unit is configured to acquire candidate task data and an initial task identifier set corresponding to an initial task data set;

[0151] The identifier generating unit is configured to generate a task identifier corresponding to the candidate task data according to the task identifier in the initial task identifier set;

[0152] The identifier set generating unit is configured to add the task identifier of the candidate task data to the initial task identifier set to obtain a task identifier set;

[0153] The data set generating unit is configured to add the candidate task data to the initial task data set to obtain a task data set.

[0154] For example, the task data can be received from a web application corresponding to the task scheduling method, or the task data can be received in a conventional message format. The task identifier corresponding to the candidate task data can be generated according to the submission time of the task data, and the order of submission of the task data can be determined according to the task identifier, or the task identifier corresponding to the task data can be generated according to the submission order of the task data, for example, the task identifier corresponding to the first submitted task data can be 1, the task identifier corresponding to the second submitted task data can be 2, and so on.

[0155] The generated task identifier is added to the initial task identifier set to obtain a task identifier set, and the candidate task data set is added to the initial task data set to obtain a task data set.

[0156] (3) The information obtaining unit 303 is configured to obtain the task execution status information of the distributed nodes.

[0157] For example, the distributed nodes can send the task execution status information to the server at a certain time, or the server can send an instruction to the distributed nodes to obtain the task execution status information.

[0158] (4) The screening unit 304 is configured to screen the idle nodes from the distributed nodes according to the task execution status information of the distributed nodes, where the idle nodes are the distributed nodes that do not currently run the target type of task.

[0159] For example, the idle nodes can be determined from the distributed nodes according to the task execution status information, and the nodes that do not execute the target type of task are the idle nodes.

[0160] In an embodiment, the screening unit 304 can include a node determining subunit, a task data obtaining subunit, and an idle node determining subunit, and specifically:

[0161] The node determining subunit is configured to determine the target node from the distributed nodes.

[0162] The task data obtaining subunit is configured to obtain the task data of the current execution task if the task execution status information indicates that the target node currently executes a task.

[0163] The idle node determining subunit is configured to determine the target node as an idle node if the task data of the current execution task and the target task data are in a non-mutual exclusion relationship.

[0164] For example, the target node can be determined from the distributed nodes according to the remaining state of the CPU, memory, or IO resources of the distributed nodes, or the target node can be determined from the distributed nodes according to a preset order.

[0165] According to the task execution status information of the target node, it is determined whether the target node is an idle node. If the task execution status information indicates that the target node is executing a task, it is determined whether the task data of the current execution task and the target task data are independent of each other, i.e., in a non-mutual exclusion relationship. If the task program corresponding to the current execution task and the task for executing the target task data are not the same task program, the task data of the current execution task and the target task data are independent of each other and can be processed simultaneously in the same node, and the target node is determined as an idle node.

[0166] In an embodiment, the task scheduling apparatus can further include a loop unit and an idle node determining unit, and specifically:

[0167] The circulating unit is configured to: if the task data of the current task and the target task data are in mutual exclusion, taking the associated node of the target node as the target node, and returning to execute the following steps until all the nodes included in the distributed nodes are traversed: if the task execution state information indicates that the target node has the current task, obtaining the task data of the current task; and if the task execution state information indicates that the target node does not have the current task, determining that the target node is an idle node.

[0168] The idle node determination unit is configured to: if the target node does not have the current task, determining that the target node is an idle node.

[0169] For example, the task program corresponding to the current task and the task for executing the target task data can be the same task program, that is, the task data of the current task and the target task data are in mutual exclusion, and cannot be processed in the same node, and it is determined that the target node is not an idle node.

[0170] If the task data of the current task and the target task data are in mutual exclusion, the associated node of the target node is obtained, the associated node is taken as the target node, and whether the target node is an idle node is determined according to the task execution state information of the target node. If the target node is not an idle node, the associated node of the target node is judged until all the nodes included in the distributed nodes are traversed.

[0171] If the task execution state information indicates that the target node does not have the current task, it is determined that the target node is an idle node.

[0172] In an embodiment, the task scheduling apparatus can include a condition determination unit and a detection unit, specifically:

[0173] The condition determination unit is configured to: if it is determined that there is no idle node in the distributed nodes by traversing the distributed nodes, determining a waiting condition according to a preset waiting strategy.

[0174] The detection unit is configured to: when the waiting condition is met, returning to execute the following steps: determining the target node from the distributed nodes, and if the task execution state information indicates that the target node has the current task, obtaining the task data of the current task.

[0175] For example, if it is determined that there is no idle node in the distributed nodes by traversing the distributed nodes, the waiting condition is determined to be waiting for 30 seconds according to the preset waiting strategy, and when waiting for 30 seconds, the process of determining the idle node from the distributed nodes is executed again.

[0176] (5) The scheduling unit 305 is configured to send the target task data to the idle node, so that the idle node processes the target task data through the task program.

[0177] For example, the target task data can be sent to the idle node, and the idle node determines the corresponding task program according to the target task data, and processes the target task data by executing the task program.

[0178] From the above, the task scheduling apparatus in the embodiments of the present application can acquire a task data set by the data acquisition unit 301, the task data set comprising task data of at least one target type task; the target task data to be scheduled and the distributed node for processing the target task data are determined by the determination unit 302 from the task data set, the distributed node being deployed with a task program for executing the target type task based on the task data, the target task data being the task data corresponding to the target type task; the task execution state information of the distributed node is acquired by the information acquisition unit 303; the idle node is selected from the distributed node by the screening unit 304 according to the task execution state information of the distributed node, the idle node being the distributed node that is not currently running the target type task; the target task data is sent to the idle node by the scheduling unit 305, so that the idle node processes the target task data through the task program. The scheme determines the idle node from the distributed node through the task execution state information of the distributed node, which can realize accurate allocation of the target task data to the node and improve the utilization efficiency of the distributed node.

[0179] The embodiments of the present application also provide a computer device, which can be a terminal or a server, as shown in Figure 5 The computer device is shown in the structural schematic diagram of the computer device related to the embodiments of the present application, and specifically:

[0180] The computer device can include a processor 1001 with one or more processing cores, a memory 1002 with one or more computer readable storage media, a power supply 1003, an input unit 1004, and the like. Those skilled in the art can understand that the computer device structure shown in the Figure 5 The computer device structure shown in the

[0181] The processor 1001 is the control center of the computer device, which connects all parts of the computer device through various interfaces and lines, executes the software programs and / or modules stored in the memory 1002 and calls the data stored in the memory 1002, performs various functions and processes data of the computer device, and thus performs overall detection of the computer device. Optionally, the processor 1001 can include one or more processing cores; preferably, the processor 1001 can integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, computer program, etc., and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 1001.

[0182] The memory 1002 can be used to store software programs and modules, and the processor 1001 executes various function applications and data processing by running the software programs and modules stored in the memory 1002. The memory 1002 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one computer program required by a function (such as a sound playing function, an image playing function, etc.), and the like; and the data storage area can store data created according to the use of the computer device, etc. In addition, the memory 1002 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device. Accordingly, the memory 1002 can also include a memory controller to provide access for the processor 1001 to the memory 1002.

[0183] The computer device further includes a power supply 1003 for supplying power to various components. Preferably, the power supply 1003 can be logically connected to the processor 1001 through a power management system, so as to realize functions such as management of charging, discharging, and power consumption management through the power management system. The power supply 1003 can also include one or more than one direct current or alternating current power supply, a recharging system, a power supply fault detection circuit, a power supply converter or inverter, a power supply state indicator, and the like.

[0184] The computer device can further include an input unit 1004, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function controls.

[0185] Although not shown, the computer device can also include a display unit and the like, which will not be described here. Specifically, in the embodiment, the processor 1001 in the computer device loads one or more executable files corresponding to the processes of the computer program into the memory 1002 according to the following instructions, and runs the computer program stored in the memory 1002 by the processor 1001, so as to realize various functions, as follows:

[0186] Obtaining a task data set, the task data set including task data of at least one target type task;

[0187] Determining target task data to be scheduled from the task data set, and determining a distributed node for processing the target task data, the distributed node being deployed with a task program for executing a target type task based on the task data, the target task data being task data corresponding to the target type task;

[0188] obtain task execution state information of the distributed nodes; filter idle nodes from the distributed nodes according to the task execution state information of the distributed nodes, the idle nodes being the distributed nodes that are not currently running the target type task;

[0189] send the target task data to the idle nodes, so that the idle nodes process the target task data through the task program.

[0190] The specific implementation of each operation can refer to the foregoing embodiments, and will not be described herein.

[0191] As can be seen from the above, the computer device provided in the embodiments of the present application can obtain a task data set, the task data set including task data of at least one target type task; determine target task data to be dispatched from the task data set, and determine a distributed node for processing the target task data, the distributed node being deployed with a task program for executing the target type task based on the task data, the target task data being task data corresponding to the target type task; obtain task execution state information of the distributed nodes; filter idle nodes from the distributed nodes according to the task execution state information of the distributed nodes, the idle nodes being the distributed nodes that are not currently running the target type task; and send the target task data to the idle nodes, so that the idle nodes process the target task data through the task program. The scheme can accurately allocate the target task data to the nodes and improve the utilization efficiency of the distributed nodes by determining the idle nodes from the distributed nodes based on the task execution state information of the distributed nodes.

[0192] According to an aspect of the present application, a computer program product or computer program is provided, which includes computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to enable the computer device to perform the method provided in various optional implementation manners in the above embodiments.

[0193] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by a computer program, or by relevant hardware controlled by a computer program, which can be stored in a computer readable storage medium and loaded and executed by a processor.

[0194] To this end, the embodiments of the present application provide a computer readable storage medium, which stores a computer program. The computer program can be loaded by a processor to execute any one of the task scheduling methods provided in the embodiments of the present application.

[0195] The specific implementation of each operation can refer to the foregoing embodiments, and will not be described herein.

[0196] The computer readable storage medium can include a read only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0197] The computer program stored in the computer readable storage medium can perform any of the task scheduling methods provided by the embodiments of the present application, thus achieving the beneficial effects of any of the task scheduling methods provided by the embodiments of the present application. Details are shown in the foregoing embodiments, which will not be repeated here.

[0198] The above describes in detail the task scheduling method, device, computer device and computer readable storage medium provided by the embodiments of the present application. The principles and implementation manners of the present application are described by applying specific examples. The above embodiment is only used to help understand the method and core idea of the present application. Meanwhile, for those skilled in the art, the specific implementation manner and application range can be changed according to the idea of the present application. In summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A task scheduling method, characterized by, The method comprises: obtaining a task data set, the task data set comprising task data of at least one target type task; determining target task data to be scheduled from the task data set, comprising: querying target task data corresponding to a target task identifier from the task data set according to a correspondence between a task identifier and task data, wherein the task identifier is generated according to a time or sequence of task data submission, and the execution sequence of the task is determined according to the task identifier; determining a distributed node for processing the target task data, the distributed node being deployed with a task program for executing the target type task based on the task data, and the target task data being task data corresponding to the target type task; obtaining task execution state information of the distributed node; determining a target node from the distributed node; if the task execution state information indicates that the target node has a current execution task, obtaining task data of the current execution task; if the task data of the current execution task and the target task data are in a non-mutual exclusion relationship, and the task program corresponding to the current execution task and the task program for executing the target task data are not the same task program, determining that the target node is an idle node; if the task data of the current execution task and the target task data are in a mutual exclusion relationship, taking an associated node of the target node as the target node, and returning to execute if the task execution state information indicates that the target node has a current execution task, obtaining task data of the current execution task, until the distributed node is traversed; if the target node has no current execution task, determining that the target node is an idle node; sending the target task data to the idle node, so that the idle node processes the target task data through the task program.

2. The method of claim 1, wherein, The method further comprises: if no idle node is determined in the distributed node after traversing the distributed node, determining a waiting condition according to a preset waiting strategy; when the waiting condition is met, returning to execute determining a target node from the distributed node, and if the task execution state information indicates that the target node has a current execution task, obtaining task data of the current execution task.

3. The method according to any one of claims 1 to 2, characterized in that, The task data set corresponds to at least one task identifier set, and the determining target task data to be scheduled from the task data set comprises: determining a target task identifier from the task identifier set according to a preset strategy; obtaining target task data to be scheduled from the task data set according to the target task identifier.

4. The method of claim 3, wherein, The at least one task identifier set corresponds to different priorities respectively, and the determining a target task identifier from the task identifier set according to a preset strategy comprises: determining a target task identifier set from the at least one task identifier set according to the priorities; screening a target task identifier from the target task identifier set according to an acquisition sequence of the task identifier.

5. The method of claim 3, wherein, Before obtaining the task data set to be scheduled, the method further comprises: Obtaining an initial task identifier set corresponding to the candidate task data and the initial task data set; According to the task identifier in the initial task identifier set, the corresponding task identifier of the candidate task data is generated; The task identifier of the candidate task data is added to the initial task identifier set to obtain the task identifier set; The candidate task data is added to the initial task data set to obtain the task data set.

6. A task scheduling apparatus characterized by comprising: Comprise: Data acquisition unit, for obtaining a task data set, the task data set includes at least one target type task data; The determination unit is used for determining the target task data to be scheduled from the task data set, including: according to the corresponding relationship between the task identifier and the task data, the target task data corresponding to the target task identifier is inquired from the task data set;Wherein, the task identifier is generated according to the time or the order of submitting the task data, and the execution order of the task is determined according to the task identifier; The determination unit is also used for determining a distributed node for processing the target task data, the distributed node is deployed with a task program for executing the target type task based on the task data, and the target task data is the task data corresponding to the target type task; Information acquisition unit, for obtaining the task execution state information of the distributed node; The device comprises a screening unit, and the screening unit comprises a node determination subunit, a task data acquisition subunit and an idle node determination subunit, specifically: The determination subunit is used for determining a target node from the distributed node; The task data acquisition subunit is used for acquiring the task data of the current execution task if the task execution state information indicates that the target node exists the current execution task; The idle node determination subunit is used for determining the target node as an idle node if the task data of the current execution task and the target task data are in a non-mutual exclusion relationship, and the task program corresponding to the current execution task and the task program for executing the target task data are not the same task program; The device further comprises a loop unit, specifically: The loop unit is used for taking the associated node of the target node as the target node if the task data of the current execution task and the target task data are in a mutual exclusion relationship, and returning to execute the task data acquisition of the current execution task if the task execution state information indicates that the target node exists the current execution task, until the distributed node is traversed; The idle node determination subunit is used for determining the target node as an idle node if the target node does not exist the current execution task; The scheduling unit is used for sending the target task data to the idle node, so that the idle node processes the target task data through the task program.

7. The apparatus of claim 6, wherein, The device further comprises a condition determination unit and a detection unit, specifically: The condition determination unit is used for determining a waiting condition according to a preset waiting strategy if it is determined that there is no idle node in the distributed node by traversing the distributed node. The detection unit is configured to return to determine a target node from the distributed nodes when the waiting condition is met, and to obtain task data of a current execution task if the task execution state information indicates that the target node exists.

8. The apparatus of any one of claims 6-7, wherein, The task data set corresponds to at least one task identifier set, and the determination unit includes an identifier determination subunit and a data acquisition subunit, specifically: The identifier determination subunit is configured to determine a target task identifier from the task identifier set according to a preset strategy. The data acquisition subunit is configured to acquire target task data to be scheduled from the task data set according to the target task identifier.

9. The apparatus of claim 8, wherein, The at least one task identifier set corresponds to different priorities, and the identifier determination subunit includes a set determination module and an identifier screening module, specifically: The set determination module is configured to determine a target task identifier set from the at least one task identifier set according to the priority. The identifier screening module is configured to screen a target task identifier from the target task identifier set according to an acquisition order of task identifiers.

10. The apparatus of claim 8, wherein, The device further includes a candidate task acquisition unit, an identifier generation unit, an identifier set generation unit, and a data set generation unit, specifically: The candidate task acquisition unit is configured to acquire candidate task data and an initial task identifier set corresponding to an initial task data set. The identifier generation unit is configured to generate a corresponding task identifier for the candidate task data according to a task identifier in the initial task identifier set. The identifier set generation unit is configured to add the task identifier of the candidate task data to the initial task identifier set to obtain the task identifier set. The data set generation unit is configured to add the candidate task data to the initial task data set to obtain the task data set.

11. A computer device, characterized by The computer readable storage medium is configured to store a computer program, and the computer program is loaded by a processor to execute the task scheduling method of any one of claims 1 to 5.

12. A computer-readable storage medium, characterized in that, The computer readable storage medium is configured to store a computer program, and the computer program is loaded by a processor to execute the task scheduling method of any one of claims 1 to 5.

Images