Task scheduling method, device and equipment, readable storage medium and program product

Abstract

The invention relates to a task scheduling method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the steps of obtaining a submitted target task, and adding the target task to a task priority queue; sorting all the tasks in the task priority queue based on the priority and deadline demand of each task in the task priority queue; task scheduling is conducted on the basis of the sequence of all tasks in the task priority queue, for the current scheduling task, a target computing node is determined on the basis of the transmission duration of data on which the current scheduling task depends and the earliest idle moment of each computing node, and the target computing node is used for completing the current scheduling task. By adopting the method, the task completion efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of cloud computing technology, and in particular to a task scheduling method, apparatus, device, computer-readable storage medium, and computer program product. Background Art

[0002] In task scheduling in heterogeneous systems, since computing nodes in heterogeneous systems may use different types of processors (such as CPU, GPU, etc.), different computing nodes have different computing and storage capabilities. For computing tasks that rely on large-scale data sets, how to schedule tasks to appropriate computing nodes to achieve the optimization goal of minimizing task completion time is a key issue.

[0003] In traditional methods, task scheduling in heterogeneous systems generally only considers the task requirements and the capabilities of the computing nodes. Since a large amount of data involved in computing tasks is stored in the cloud for persistence, when the computing tasks are executed, the latency of accessing the cloud data transmission is high, which affects the completion time of the task. Therefore, traditional task scheduling methods have the problem of low task completion efficiency. Summary of the invention

[0004] Based on this, it is necessary to provide a task scheduling method, device, computer equipment, computer-readable storage medium and computer program product that can improve the efficiency of task completion in response to the above technical problems.

[0005] In a first aspect, the present application provides a task scheduling method, comprising:

[0006] Get the submitted target task and add it to the task priority queue;

[0007] Sort all tasks in the task priority queue based on the priority and deadline requirements of each task in the task priority queue;

[0008] Task scheduling is performed based on the order of tasks in the task priority queue. For the current scheduled task, the target computing node is determined based on the transmission duration of the data on which the current scheduled task depends and the earliest idle time of each computing node. The target computing node is used to complete the current scheduled task.

[0009] In one embodiment, all tasks in the task priority queue are sorted based on the priority and deadline requirements of each task in the task priority queue, including:

[0010] For each task in the task priority queue, determine the task priority score based on the task priority and deadline requirements;

[0011] Sort all tasks in the task priority queue according to their priority scores.

[0012] In one embodiment, determining a priority score of a task according to the priority and deadline requirement of the task includes:

[0013] The priority of the task is weighted and summed with the inverse of the data corresponding to the deadline requirement of the task to obtain the priority score of the task.

[0014] In one embodiment, determining a target computing node based on the transmission duration of data on which the current scheduling task depends and the earliest idle time of each computing node includes:

[0015] For each computing node, when the data on which the current scheduling task depends exists in the cache data queue of the computing node, the completion time of the current scheduling task on the computing node is determined according to the earliest idle time of the computing node;

[0016] When the data that the current scheduled task depends on does not exist in the cache data queue of the computing node, the data arrival time is obtained by summing the transmission time and the task start time of the current scheduled task. The completion time of the current scheduled task on the computing node is determined based on the maximum value of the data arrival time and the earliest idle time of the computing node.

[0017] According to the completion time of the current scheduled tasks on each computing node, a target computing node is determined among the multiple computing nodes.

[0018] In one embodiment, determining a target computing node from among a plurality of computing nodes according to the completion time of the currently scheduled task at each computing node respectively includes:

[0019] Get the node type of each computing node;

[0020] Determine, from the plurality of computing nodes, a plurality of candidate computing nodes whose completion times satisfy the deadline constraint and whose node types satisfy the node type requirements;

[0021] A target computing node that meets the optimization goal is determined from multiple candidate computing nodes.

[0022] In one embodiment, the task scheduling method further includes:

[0023] If there is remaining storage space in the cache data queue in the target computing node, and the remaining storage space is greater than or equal to the amount of data on which the current scheduling task depends, the data on which the current scheduling task depends is cached to the target computing node;

[0024] When there is remaining storage space in the cache data queue in the target computing node, and the remaining storage space is less than the amount of data on which the current scheduling task depends, or when there is no remaining storage space in the cache data queue in the target computing node, data is removed from the cache data queue in the target computing node according to preset rules, and the data on which the current scheduling task depends is cached to the target computing node.

[0025] In a second aspect, the present application also provides a task scheduling device, comprising:

[0026] The acquisition module is used to obtain the submitted target task and add the target task to the task priority queue;

[0027] A sorting module is used to sort all tasks in the task priority queue based on the priority and deadline requirements of each task in the task priority queue;

[0028] The scheduling module is used to schedule tasks based on the order of tasks in the task priority queue. For the current scheduling task, the target computing node is determined based on the transmission duration of the data on which the current scheduling task depends and the earliest idle time of each computing node. The target computing node is used to complete the current scheduling task.

[0029] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0030] Get the submitted target task and add it to the task priority queue;

[0031] Sort all tasks in the task priority queue based on the priority and deadline requirements of each task in the task priority queue;

[0032] Task scheduling is performed based on the order of tasks in the task priority queue. For the current scheduled task, the target computing node is determined based on the transmission duration of the data on which the current scheduled task depends and the earliest idle time of each computing node. The target computing node is used to complete the current scheduled task.

[0033] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0034] Get the submitted target task and add it to the task priority queue;

[0035] Sort all tasks in the task priority queue based on the priority and deadline requirements of each task in the task priority queue;

[0036] Task scheduling is performed based on the order of tasks in the task priority queue. For the current scheduled task, the target computing node is determined based on the transmission duration of the data on which the current scheduled task depends and the earliest idle time of each computing node. The target computing node is used to complete the current scheduled task.

[0037] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0038] Get the submitted target task and add it to the task priority queue;

[0039] Sort all tasks in the task priority queue based on the priority and deadline requirements of each task in the task priority queue;

[0040] Task scheduling is performed based on the order of tasks in the task priority queue. For the current scheduled task, the target computing node is determined based on the transmission duration of the data on which the current scheduled task depends and the earliest idle time of each computing node. The target computing node is used to complete the current scheduled task.

[0041] The above-mentioned task scheduling method, apparatus, computer equipment, computer-readable storage medium and computer program product, by adding the submitted target task to the task priority queue, all tasks in the task priority queue can be sorted according to the priority and deadline requirements of each task, which is conducive to giving priority to scheduling tasks with high importance; in the task scheduling process, for the current scheduling task, based on the transmission duration of the data on which the current scheduling task depends and the earliest idle time of each computing node, the target computing node is determined, which fully considers the transmission delay of the data on which the current scheduling task depends and the computing resource conditions of the computing node, which is conducive to determining the optimal target computing node, using the target computing node to complete the current scheduling task, and is conducive to improving the efficiency of task completion. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 An application environment diagram of a task scheduling method in one embodiment;

[0044] Figure 2 A flowchart of a task scheduling method in one embodiment;

[0045] Figure 3 The figure is a schematic diagram of the overall process of a task scheduling method in one embodiment;

[0046] Figure 4 is a schematic diagram of the overall flow of a task scheduling method in another embodiment;

[0047] Figure 5 is a structural block diagram of a task scheduling device in one embodiment;

[0048] Figure 6 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0050] The task scheduling method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. This embodiment uses the method applied to the server 104 as an example. It can be understood that the method can also be applied to the terminal 102, and can also be applied to a system including the terminal 102 and the server 104, and is implemented through the interaction between the terminal 102 and the server 104. The server 104 obtains the submitted target task and adds the target task to the task priority queue; based on the priority and deadline requirements of each task in the task priority queue, all tasks in the task priority queue are sorted; based on the order of each task in the task priority queue, the task is scheduled, and for the current scheduled task, the target computing node is determined based on the transmission duration of the data on which the current scheduled task depends and the earliest idle time of each computing node. The target computing node is used to complete the current scheduled task. The terminal 102 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car devices, projection devices, etc. Portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. Head-mounted devices may be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. The server 104 may be an independent physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides cloud computing services.

[0051] In an exemplary embodiment, Figure 2 As shown, a task scheduling method is provided, which is applied to Figure 1 The server 104 in the example is used as an example to illustrate the method, which includes the following steps 202 to 206. Among them:

[0052] Step 202: Obtain the submitted target task and add the target task to the task priority queue.

[0053] The server may be a server in a heterogeneous system, which includes multiple computing nodes, and each computing node may use a different type of processor (such as a CPU, a GPU, etc.).

[0054] The target task refers to the task submitted by the user. For example, in a cloud computing scenario, the target task can be a computing task. The task priority queue refers to a queue used to store tasks. When the target task is added to the task priority queue, there can be at least one task in the task priority queue, or there can be no task.

[0055] Step 204: sort all tasks in the task priority queue based on the priority and deadline requirements of each task in the task priority queue.

[0056] Among them, tasks in the task priority queue are usually scheduled in a first-in-first-out order. However, if the importance of the target task is relatively high, since the target task is the last one added to the task priority queue, it may cause the target task to be scheduled late, thereby affecting the efficiency of task completion. The embodiment of the present application sorts the tasks in the task priority queue to ensure that tasks with higher importance are scheduled first, so as to improve system efficiency.

[0057] Priority refers to the priority that the user presets for each task, for example, priorities include high priority, medium priority, low priority, etc. Deadline requirement refers to the time required to complete the task.

[0058] The server may sort the tasks according to at least one of the priority and deadline requirements of each task. For example, each task may be sorted according to high priority, medium priority and low priority, and tasks of the same priority may be scheduled in a first-in-first-out order. For another example, each task may be sorted in a descending order of deadline requirements. For another example, each task may be sorted according to the comprehensive priority and deadline requirements.

[0059] Step 206, perform task scheduling based on the order of tasks in the task priority queue, and determine the target computing node for the current scheduled task based on the transmission duration of the data on which the current scheduled task depends and the earliest idle time of each computing node. The target computing node is used to complete the current scheduled task.

[0060] The currently scheduled task refers to the task that is currently scheduled when scheduling is performed according to the order of tasks in the task priority queue.

[0061] The data that the current scheduling task depends on refers to the data required to complete the current scheduling task. In cloud computing scenarios, the data that each task depends on is usually stored in the cloud.

[0062] Transmission duration refers to the time it takes for data to be transmitted from the cloud to the computing node. In some embodiments, the time it takes for data to be output from the cloud to each computing node may be different. Therefore, for the current scheduling task, the server needs to obtain the transmission time it takes for the data that the current scheduling task depends on to be transmitted from the cloud to each computing node. Transmission duration is used to characterize the transmission delay of the data that the task depends on.

[0063] The earliest idle time refers to the completion time of the last task among all tasks that the computing node needs to complete, and is used to characterize the computing resource status of the computing node.

[0064] In a heterogeneous system, some computing nodes with weak computing power will have a negative impact on the overall performance of the system. At the same time, computing nodes with strong computing power may cause the task waiting time to be too long due to excessive load, affecting the completion time of the task. Computing nodes with weak computing power are idle and do not perform any tasks. This situation may lead to load imbalance, that is, some nodes are overloaded while other nodes are idle, thereby reducing the efficiency of the entire system. For computing-intensive tasks submitted by users, which involve a large amount of data processing and require computing power, scheduling computing tasks to computing nodes with strong computing power according to the computing resources in the computing stage is conducive to reducing task execution time.

[0065] In addition, computing tasks are also dependent on data. Usually in cloud computing scenarios, data is persistently stored in cloud storage, and the latency in accessing cloud data is high. When making task scheduling decisions in heterogeneous systems, comprehensive consideration of the computing resources of the computing nodes and the data transmission latency is beneficial to determining the optimal target computing node and completing the current scheduling task in the target computing node, which helps reduce task completion time and improve system efficiency.

[0066] In the above task scheduling method, by adding the submitted target task to the task priority queue, all tasks in the task priority queue can be sorted according to the priority and deadline requirements of each task, which is conducive to giving priority to the scheduling of tasks with high importance; in the task scheduling process, for the current scheduling task, the target computing node is determined based on the transmission duration of the data on which the current scheduling task depends and the earliest idle time of each computing node, which fully considers the transmission delay of the data on which the current scheduling task depends and the computing resource conditions of the computing node, which is conducive to determining the optimal target computing node, using the target computing node to complete the current scheduling task, and improving the efficiency of task completion.

[0067] In an exemplary embodiment, all tasks in the task priority queue are sorted based on the priority and deadline requirements of each task in the task priority queue, including: for each task in the task priority queue, determining the priority score of the task according to the priority and deadline requirements of the task; and sorting all tasks in the task priority queue according to the priority scores of each task.

[0068] In the process of calculating the priority score, the priority of the task can be converted into corresponding data, and the priority score can be calculated based on the data corresponding to the priority and the data corresponding to the deadline requirement.

[0069] The priority score is a score used to indicate the order of tasks. For example, the tasks may be ordered in descending order according to their priority scores, and the task with the highest priority score is ranked first in the task priority queue.

[0070] In this embodiment, since the calculation of the priority score of the task comprehensively considers the priority and deadline requirements of the task, sorting the tasks according to their priority scores is conducive to prioritizing the scheduling of tasks with higher importance, thereby achieving efficient scheduling of tasks.

[0071] In an exemplary embodiment, determining the priority score of a task according to the priority and deadline requirement of the task includes: performing a weighted summation of the priority of the task and the inverse of the data corresponding to the deadline requirement of the task to obtain the priority score of the task.

[0072] Among them, tasks with higher priorities and tasks with smaller deadline requirements are scheduled first, which is conducive to improving task scheduling efficiency. Therefore, the priority of the task and the inverse of the data corresponding to the deadline requirement of the task can be weighted and summed to obtain the priority score of the task.

[0073] In some embodiments, multiple tasks , Indicates The calculation formula for the priority score of a task can be:

[0074]

[0075] in, Indicates the task The priority score of Indicates the task The priority of Indicates the task The deadline requirement, and is a weight factor used to balance the impact of priority and deadline requirements.

[0076] In this embodiment, by performing a weighted summation of the task priority and the inverse of the data corresponding to the task deadline requirement, it is helpful to ensure that tasks with higher priority and tasks with smaller deadline requirements are scheduled first, thereby ensuring efficient task scheduling.

[0077] In an exemplary embodiment, a target computing node is determined based on the transmission duration of the data on which the current scheduling task depends and the earliest idle time of each computing node, including: for each computing node, when the data on which the current scheduling task depends exists in the cache data queue of the computing node, the completion time of the current scheduling task on the computing node is determined according to the earliest idle time of the computing node; when the data on which the current scheduling task depends does not exist in the cache data queue of the computing node, the transmission duration is summed with the task start time of the current scheduling task to obtain the data arrival time, and the completion time of the current scheduling task on the computing node is determined based on the maximum value of the data arrival time and the earliest idle time of the computing node; and the target computing node is determined among multiple computing nodes according to the completion time of the current scheduling task on each computing node.

[0078] In some embodiments, the data that some tasks depend on are pre-stored in the cache data queue of the computing node, thereby reducing the impact of data transmission delay caused by accessing the cloud. If the data that the current scheduled task depends on exists in the cache data queue of the computing node, the completion time of the current scheduled task on the computing node can be calculated based on the earliest idle time of the computing node.

[0079] If the data that the current scheduling task depends on does not exist in the cache data queue of the computing node, the completion time of the current scheduling task on the computing node needs to be determined comprehensively based on the transmission time of the data that the current scheduling task depends on and the earliest idle time of the computing node. Specifically, the data arrival time is obtained by summing the transmission time and the task start time. The data arrival time represents the time when the data that the current scheduling task depends on arrives at the computing node, and the completion time is determined based on the maximum value of the data arrival time and the earliest idle time of the computing node. Since the data that the current scheduling task depends on exists in the cloud, the completion time comprehensively considers the data transmission delay and the computing resource situation of the computing node, which is conducive to determining a more accurate completion time.

[0080] In some embodiments, the computing node Earliest free time It can be expressed as:

[0081]

[0082] in, Represents a compute node The set of tasks to be executed on Represents a compute node The last task among all tasks The moment of completion.

[0083] Current scheduled tasks On the compute node The completion time It can be expressed as:

[0084]

[0085]

[0086] in, , , Respectively represent the current scheduling tasks The computing requirements, data requirements, and task start time, , , Represents the computing nodes computing power, cache data queues, bandwidth, Respectively represent the current scheduling tasks Data relied on Transmitted from the cloud to the computing nodes The transmission time on .

[0087] After the completion time of the current scheduling task on each computing node is determined, the shortest completion time can be used as the optimization goal to determine the target computing node.

[0088] In this embodiment, different methods are used to determine whether the data on which the current scheduling task depends exists in the cache data queue of the computing node, which fully considers the data transmission delay and the computing resource situation of the computing node, which is conducive to determining a more accurate completion time, calculating the completion time of the current scheduling task on each computing node, determining the optimal target computing node, and using the target computing node to complete the current scheduling task, which is conducive to improving the efficiency of task completion.

[0089] In an exemplary embodiment, a target computing node is determined from multiple computing nodes based on the completion time of the current scheduled task at each computing node, including: obtaining the node type of each computing node; determining from the multiple computing nodes multiple candidate computing nodes whose completion times meet the deadline constraints and whose node types meet the node type requirements; and determining from the multiple candidate computing nodes a target computing node that meets the optimization goal.

[0090] The node type of the computing node includes but is not limited to the CPU type, GPU type, etc. Computing nodes of different node types have different computing capabilities. The node type requirement refers to the node type required by the user, and the node type of the computing node needs to meet the node type requirement to ensure the efficiency of task completion. The completion time satisfies the deadline constraint condition when the difference between the completion time and the task start time of the currently scheduled task is less than or equal to the deadline requirement.

[0091] In some embodiments, the conditions for determining the candidate computing nodes may be expressed as follows:

[0092]

[0093] in, Represents a candidate computing node The node type, , Respectively represent the current scheduling tasks Node type requirements, deadline requirements, Indicates the current scheduled task On the compute node The moment of completion.

[0094] The server can include multiple candidate computing nodes The computing node with the shortest completion time is used as the target computing node. In some embodiments, the optimization target can be expressed as follows:

[0095]

[0096] in, Indicates the current scheduled task On the compute node The completion time of .

[0097] In this embodiment, multiple candidate computing nodes whose completion times meet the deadline constraints and whose node types meet the node type requirements are determined from multiple computing nodes. The determination of the candidate computing nodes takes into account the computing requirements of the task, the node type requirements, and the computing resources, storage resources, data cache queue conditions, etc. of the computing nodes. Multiple candidate computing nodes are determined for each task, and multiple candidate computing nodes are used for decision-making on the target computing node, which is conducive to improving system efficiency.

[0098] In an exemplary embodiment, the task scheduling method also includes: when there is remaining storage space in the cache data queue of the target computing node, and the remaining storage space is greater than or equal to the number of data on which the current scheduling task depends, caching the data on which the current scheduling task depends to the target computing node; when there is remaining storage space in the cache data queue of the target computing node, and the remaining storage space is less than the number of data on which the current scheduling task depends, or, when there is no remaining storage space in the cache data queue of the target computing node, removing data from the cache data queue in the target computing node according to preset rules, and caching the data on which the current scheduling task depends to the target computing node.

[0099] Among them, when the target computing node is executing the current scheduling task, if the data that the current scheduling task depends on exists in the cache data queue of the target computing node, the current scheduling task can be executed directly. If the data that the current scheduling task depends on does not exist in the cache data queue of the target computing node, the corresponding data needs to be cached from the cloud to the computing node.

[0100] The specific caching method needs to determine whether the target computing node has remaining storage space and whether the remaining storage space is greater than the amount of data that the current scheduling task depends on, so as to determine whether the data that the current scheduling task depends on is directly stored in the cache data queue in the target computing node; or after removing part of the data from the cache data queue, the data that the current scheduling task depends on is directly stored in the cache data queue in the target computing node.

[0101] In some embodiments, the preset rule may be a LRU (Least Recently Used) rule.

[0102] In some embodiments, the caching method may be expressed as follows:

[0103]

[0104] in, Indicates the target computing node of remaining storage space, Indicates the current scheduled task The data relied on, Indicates update After the queue, It is a data queue that is evicted through the LRU rule. is the remaining storage space in the queue after the eviction operation.

[0105] In this embodiment, the data that the current scheduling task depends on is cached from the cloud to the target computing node through the cache data queue in the target computing node, which is beneficial to improve user transmission delay, reduce task completion time, and improve system efficiency.

[0106] To explain the task scheduling method and effect in this solution in detail, a most detailed embodiment is described below:

[0107] In heterogeneous systems, a heuristic dynamic task scheduling solution is provided. The scheduler is defined according to the task priority queue and node candidate set, taking into account the task deadline constraints and node storage resource constraints, and making task scheduling decisions based on the heuristic method of cache data affinity. Figure 3 The figure is a schematic diagram of the overall process of a task scheduling method in one embodiment.

[0108] 1. Task priority queue

[0109] Task set multiple tasks , Indicates tasks, including Respectively represent tasks priorities, node type requirements, computing requirements, and data requirements, For the task deadline requirements.

[0110] In order to comprehensively consider the priority and deadline requirements, a comprehensive priority score is defined to sort tasks, maintain the task priority queue, and achieve efficient scheduling. The calculation formula of the priority score can be:

[0111]

[0112] in, Indicates the task The priority score, Indicates the task The priority of Indicates the task The deadline requirement, and is a weight factor used to balance the impact of priority and deadline requirements.

[0113] According to the comprehensive priority score of the task, a task priority queue is maintained, which is implemented through the maximum heap and managed by the scheduler. When the user submits a new task to the scheduling system, the new task is added to the priority queue and waits for scheduling.

[0114] 2. Node Candidate Set

[0115] The node set is defined as , Indicates Compute nodes, of which Represents the computing nodes The node type, computing power, remaining storage space, bandwidth, and cache data queues.

[0116] Task On the compute node The waiting time on the node depends on the number of tasks being processed on the node and the completion time of the tasks. The set of tasks to be executed is , then the computing node Earliest free time Equal to the completion time of the last task in the task set, expressed as:

[0117]

[0118] Task On the compute node The completion time It can be expressed as:

[0119]

[0120]

[0121] in, , , Respectively represent the current scheduling tasks The computing requirements, data requirements, and task start time, , , Represents the computing nodes computing power, cache data queues, bandwidth, Respectively represent the current scheduling tasks Data relied on Transmitted from the cloud to the computing nodes The transmission time on .

[0122] Task The node candidate set , where the computing nodes Satisfy the node type requirements of the task and meet the deadline constraints for a long time when completed:

[0123]

[0124] 3. Task scheduler based on cache data affinity

[0125] According to the completion time of the task on the computing node, the optimization goal of minimizing the task completion time is expressed as:

[0126]

[0127] When the data that the task depends on is not in the cache data queue of the target computing node, that is, , the data The cache is stored on the target computing node and needs to meet the storage resource constraints of the target computing node:

[0128]

[0129] in, Indicates the target computing node of remaining storage space, Indicates the current scheduled task The data relied on, Indicates update After the queue, It is a data queue that is evicted through the LRU rule. is the remaining storage space in the queue after the eviction operation. The space is not full and the remaining storage space is greater than or equal to the amount of data When the cache data queue is not full and the remaining storage space is less than the amount of data, the data can be cached to the cache data queue of the target computing node; when the cache data queue is not full and the remaining storage space is less than the amount of data When, or when the cache data queue is full, there is more than the amount of data in the queue Data queue , select the data queue according to the LRU rule Remove and save the data Cache to node.

[0130] like Figure 4 The overall flow diagram of the task scheduling method in another embodiment is shown. The scheduler evaluates the completion time of the task on each candidate computing node, selects the minimum completion time as the best target computing node to complete the task scheduling according to the evaluation index and constraint conditions, and updates the node status at the same time.

[0131] The above-mentioned task scheduling method, by adding the submitted target task to the task priority queue, all tasks in the task priority queue can be sorted according to the priority and deadline requirements of each task, which is conducive to giving priority to the scheduling of tasks with high importance; in the task scheduling process, for the current scheduling task, based on the transmission time of the data on which the current scheduling task depends and the earliest idle time of each computing node, the target computing node is determined, which fully considers the transmission delay of the data on which the current scheduling task depends and the computing resource conditions of the computing node, which is conducive to determining the optimal target computing node, using the target computing node to complete the current scheduling task, and is conducive to improving the efficiency of task completion.

[0132] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0133] Based on the same inventive concept, the embodiment of the present application also provides a task scheduling device for implementing the task scheduling method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more task scheduling device embodiments provided below can refer to the limitations on the task scheduling method above, and will not be repeated here.

[0134] In an exemplary embodiment, Figure 5 As shown, a task scheduling device 100 is provided, comprising: an acquisition module 120, a sorting module 140 and a scheduling module 160, wherein:

[0135] The acquisition module 120 is used to acquire the submitted target task and add the target task to the task priority queue;

[0136] A sorting module 140, for sorting all tasks in the task priority queue based on the priority and deadline requirements of each task in the task priority queue;

[0137] The scheduling module 160 is used to schedule tasks based on the order of tasks in the task priority queue. For the current scheduling task, the target computing node is determined based on the transmission duration of the data on which the current scheduling task depends and the earliest idle time of each computing node. The target computing node is used to complete the current scheduling task.

[0138] The above-mentioned task scheduling device, by adding the submitted target task to the task priority queue, can sort all the tasks in the task priority queue according to the priority and deadline requirements of each task, which is conducive to giving priority to the scheduling of tasks with high importance; in the task scheduling process, for the current scheduling task, based on the transmission duration of the data on which the current scheduling task depends and the earliest idle time of each computing node, the target computing node is determined, which fully considers the transmission delay of the data on which the current scheduling task depends and the computing resource conditions of the computing node, which is conducive to determining the optimal target computing node, using the target computing node to complete the current scheduling task, and improving the efficiency of task completion.

[0139] In one embodiment, all tasks in the task priority queue are sorted based on the priority and deadline requirements of each task in the task priority queue. The sorting module 140 is also used to: for each task in the task priority queue, determine the priority score of the task according to the priority and deadline requirements of the task; and sort all tasks in the task priority queue according to the priority scores of each task.

[0140] In one embodiment, the priority score of the task is determined according to the priority and deadline requirement of the task, and the sorting module 140 is further used to: perform a weighted sum of the reciprocal of the data corresponding to the priority of the task and the deadline requirement of the task to obtain the priority score of the task.

[0141] In one embodiment, the target computing node is determined based on the transmission duration of the data on which the current scheduling task depends and the earliest idle time of each computing node. The scheduling module 160 is also used to: for each computing node, when the data on which the current scheduling task depends exists in the cache data queue of the computing node, determine the completion time of the current scheduling task on the computing node according to the earliest idle time of the computing node; when the data on which the current scheduling task depends does not exist in the cache data queue of the computing node, sum the transmission duration and the task start time of the current scheduling task to obtain the data arrival time, and determine the completion time of the current scheduling task on the computing node based on the maximum value of the data arrival time and the earliest idle time of the computing node; determine the target computing node among multiple computing nodes according to the completion time of the current scheduling task on each computing node.

[0142] In one embodiment, a target computing node is determined from a plurality of computing nodes according to the completion time of the current scheduling task at each computing node respectively, and the scheduling module 160 is also used to: obtain the node type of each computing node; determine from a plurality of computing nodes a plurality of candidate computing nodes whose completion times meet the deadline constraints and whose node types meet the node type requirements; and determine from a plurality of candidate computing nodes a target computing node that meets the optimization goal.

[0143] In one embodiment, the task scheduling device 100 also includes a cache module, which is used to: when there is remaining storage space in the cache data queue of the target computing node, and the remaining storage space is greater than or equal to the number of data on which the current scheduling task depends, cache the data on which the current scheduling task depends to the target computing node; when there is remaining storage space in the cache data queue of the target computing node, and the remaining storage space is less than the number of data on which the current scheduling task depends, or when there is no remaining storage space in the cache data queue of the target computing node, remove the data from the cache data queue in the target computing node according to preset rules, and cache the data on which the current scheduling task depends to the target computing node.

[0144] Each module in the above task scheduling device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0145] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 6 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store target tasks, transmission duration, earliest idle time, etc. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a task scheduling method is implemented.

[0146] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0147] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0148] Get the submitted target task and add it to the task priority queue; sort all tasks in the task priority queue based on the priority and deadline requirements of each task in the task priority queue; schedule tasks based on the order of each task in the task priority queue, and determine the target computing node for the current scheduled task based on the transmission duration of the data on which the current scheduled task depends and the earliest idle time of each computing node. The target computing node is used to complete the current scheduled task.

[0149] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0150] For each task in the task priority queue, determine the priority score of the task according to the priority and deadline requirements of the task; and sort all tasks in the task priority queue according to the priority scores of each task.

[0151] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0152] The priority of the task is weighted and summed with the inverse of the data corresponding to the deadline requirement of the task to obtain the priority score of the task.

[0153] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0154] For each computing node, when the data on which the current scheduling task depends exists in the cache data queue of the computing node, the completion time of the current scheduling task on the computing node is determined according to the earliest idle time of the computing node; when the data on which the current scheduling task depends does not exist in the cache data queue of the computing node, the data arrival time is obtained by summing the transmission duration and the task start time of the current scheduling task, and the completion time of the current scheduling task on the computing node is determined based on the maximum value of the data arrival time and the earliest idle time of the computing node; according to the completion time of the current scheduling task on each computing node, the target computing node is determined among multiple computing nodes.

[0155] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0156] Obtain the node type of each computing node; determine multiple candidate computing nodes from multiple computing nodes whose completion times meet the deadline constraint and whose node types meet the node type requirements; and determine a target computing node that meets the optimization goal from the multiple candidate computing nodes.

[0157] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0158] When there is remaining storage space in the cache data queue of the target computing node, and the remaining storage space is greater than or equal to the number of data on which the current scheduling task depends, the data on which the current scheduling task depends is cached to the target computing node; when there is remaining storage space in the cache data queue of the target computing node, and the remaining storage space is less than the number of data on which the current scheduling task depends, or when there is no remaining storage space in the cache data queue of the target computing node, the data is removed from the cache data queue of the target computing node according to preset rules, and the data on which the current scheduling task depends is cached to the target computing node.

[0159] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0160] Get the submitted target task and add it to the task priority queue; sort all tasks in the task priority queue based on the priority and deadline requirements of each task in the task priority queue; schedule tasks based on the order of each task in the task priority queue, and determine the target computing node for the current scheduled task based on the transmission duration of the data on which the current scheduled task depends and the earliest idle time of each computing node. The target computing node is used to complete the current scheduled task.

[0161] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0162] For each task in the task priority queue, determine the priority score of the task according to the priority and deadline requirements of the task; and sort all tasks in the task priority queue according to the priority scores of each task.

[0163] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0164] The priority of the task is weighted and summed with the inverse of the data corresponding to the deadline requirement of the task to obtain the priority score of the task.

[0165] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0166] For each computing node, when the data on which the current scheduling task depends exists in the cache data queue of the computing node, the completion time of the current scheduling task on the computing node is determined according to the earliest idle time of the computing node; when the data on which the current scheduling task depends does not exist in the cache data queue of the computing node, the data arrival time is obtained by summing the transmission duration and the task start time of the current scheduling task, and the completion time of the current scheduling task on the computing node is determined based on the maximum value of the data arrival time and the earliest idle time of the computing node; according to the completion time of the current scheduling task on each computing node, the target computing node is determined among multiple computing nodes.

[0167] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0168] Obtain the node type of each computing node; determine multiple candidate computing nodes from multiple computing nodes whose completion times meet the deadline constraint and whose node types meet the node type requirements; and determine a target computing node that meets the optimization goal from the multiple candidate computing nodes.

[0169] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0170] When there is remaining storage space in the cache data queue of the target computing node, and the remaining storage space is greater than or equal to the number of data on which the current scheduling task depends, the data on which the current scheduling task depends is cached to the target computing node; when there is remaining storage space in the cache data queue of the target computing node, and the remaining storage space is less than the number of data on which the current scheduling task depends, or when there is no remaining storage space in the cache data queue of the target computing node, the data is removed from the cache data queue of the target computing node according to preset rules, and the data on which the current scheduling task depends is cached to the target computing node.

[0171] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0172] Get the submitted target task and add it to the task priority queue; sort all tasks in the task priority queue based on the priority and deadline requirements of each task in the task priority queue; schedule tasks based on the order of each task in the task priority queue, and determine the target computing node for the current scheduled task based on the transmission duration of the data on which the current scheduled task depends and the earliest idle time of each computing node. The target computing node is used to complete the current scheduled task.

[0173] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0174] For each task in the task priority queue, determine the priority score of the task according to the priority and deadline requirements of the task; and sort all tasks in the task priority queue according to the priority scores of each task.

[0175] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0176] The priority of the task is weighted and summed with the inverse of the data corresponding to the deadline requirement of the task to obtain the priority score of the task.

[0177] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0178] For each computing node, when the data on which the current scheduling task depends exists in the cache data queue of the computing node, the completion time of the current scheduling task on the computing node is determined according to the earliest idle time of the computing node; when the data on which the current scheduling task depends does not exist in the cache data queue of the computing node, the data arrival time is obtained by summing the transmission duration and the task start time of the current scheduling task, and the completion time of the current scheduling task on the computing node is determined based on the maximum value of the data arrival time and the earliest idle time of the computing node; according to the completion time of the current scheduling task on each computing node, the target computing node is determined among multiple computing nodes.

[0179] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0180] Obtain the node type of each computing node; determine multiple candidate computing nodes from multiple computing nodes whose completion times meet the deadline constraint and whose node types meet the node type requirements; and determine a target computing node that meets the optimization goal from the multiple candidate computing nodes.

[0181] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0182] When there is remaining storage space in the cache data queue of the target computing node, and the remaining storage space is greater than or equal to the number of data on which the current scheduling task depends, the data on which the current scheduling task depends is cached to the target computing node; when there is remaining storage space in the cache data queue of the target computing node, and the remaining storage space is less than the number of data on which the current scheduling task depends, or when there is no remaining storage space in the cache data queue of the target computing node, the data is removed from the cache data queue of the target computing node according to preset rules, and the data on which the current scheduling task depends is cached to the target computing node.

[0183] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0184] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0185] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0186] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A task scheduling method, characterized in that: The method comprises: Obtain the submitted target task and add the target task to the task priority queue; Sorting all tasks in the task priority queue based on the priority and deadline requirements of each task in the task priority queue; Task scheduling is performed based on the order of tasks in the task priority queue. For the current scheduled task, a target computing node is determined based on the transmission duration of the data on which the current scheduled task depends and the earliest idle time of each computing node. The target computing node is used to complete the current scheduled task.

2. The method according to claim 1, characterized in that The step of sorting all the tasks in the task priority queue based on the priority and deadline requirements of each task in the task priority queue includes: For each task in the task priority queue, determining a priority score of the task according to the priority and deadline requirement of the task; All tasks in the task priority queue are sorted according to the priority score of each task.

3. The method according to claim 2, characterized in that Determining the priority score of the task according to the priority and deadline requirement of the task includes: The priority of the task and the inverse of the data corresponding to the deadline requirement of the task are weightedly summed to obtain the priority score of the task.

4. The method according to claim 1, characterized in that: The determining of the target computing node based on the transmission duration of the data on which the current scheduling task depends and the earliest idle time of each computing node includes: For each computing node, when the data on which the currently scheduled task depends exists in the cache data queue of the computing node, determine the completion time of the currently scheduled task on the computing node according to the earliest idle time of the computing node; In the case that the data on which the currently scheduled task depends does not exist in the cache data queue of the computing node, the data arrival time is obtained by summing the transmission duration and the task start time of the currently scheduled task, and the completion time of the currently scheduled task on the computing node is determined based on the maximum value of the data arrival time and the earliest idle time of the computing node; According to the completion time of the current scheduled task at each computing node, a target computing node is determined from the plurality of computing nodes.

5. The method according to claim 4, characterized in that The determining a target computing node from among the plurality of computing nodes according to the completion time of the currently scheduled task at each computing node respectively includes: Get the node type of each computing node; Determine, from the plurality of computing nodes, a plurality of candidate computing nodes whose completion times satisfy the deadline constraint and whose node types satisfy the node type requirements; A target computing node that meets the optimization goal is determined from multiple candidate computing nodes.

6. The method according to claim 1, characterized in that The method further comprises: If there is remaining storage space in the cache data queue in the target computing node, and the remaining storage space is greater than or equal to the amount of data on which the current scheduling task depends, cache the data on which the current scheduling task depends to the target computing node; When there is remaining storage space in the cache data queue in the target computing node, and the remaining storage space is less than the amount of data on which the current scheduling task depends, or when there is no remaining storage space in the cache data queue in the target computing node, data is removed from the cache data queue in the target computing node according to preset rules, and the data on which the current scheduling task depends is cached to the target computing node.

7. A task scheduling device, characterized in that: The device comprises: An acquisition module, used for acquiring the submitted target task and adding the target task to the task priority queue; A sorting module, used to sort all tasks in the task priority queue based on the priority and deadline requirements of each task in the task priority queue; The scheduling module is used to schedule tasks based on the order of tasks in the task priority queue. For the current scheduled task, the target computing node is determined based on the transmission duration of the data on which the current scheduled task depends and the earliest idle time of each computing node. The target computing node is used to complete the current scheduled task.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images