Task distribution method and device, storage medium, equipment and distributed task system

Abstract

The invention relates to a task distribution method, a task distribution device, a storage medium, equipment and a distributed task system. The method comprises the steps of determining a current taskto be distributed from a queue to be distributed; traversing resource configuration information of each sub-node on a main node according to a preset first traversal time according to resources needed by a current task to be distributed to determine whether the sub-node which can meet the resources needed by the current task to be distributed exists; when the sub-node capable of meeting the resources needed by the current task to be distributed does not exist, transferring the current task to be distributed to a waiting resource queue from the queue to be distributed, thus allowing the task to wait for distribution in the waiting resource queue. Therefore, the blocked task in the queue to be distributed can be timely transferred, so that the condition that the other tasks in the queue tobe distributed can be timely and sequentially distributed is ensured, task distribution efficiency in the queue to be distributed is improved, and task concurrent processing capability of the distributed task system and a resource utilization rate of each sub-node are enhanced.

Description

technical field [0001] The present disclosure relates to the technical field of the Internet, and in particular, to a task allocation method, device, storage medium, equipment, and distributed task system. Background technique [0002] A distributed task system usually consists of a master node, multiple sub-nodes, and at least one client, wherein the resources (CPU resources, graphics processing unit (Graphics Processing Unit, GPU) resources, memory resources, hard disk resources) between different sub-nodes etc.) The configuration may be different, and the resource configuration information of each sub-node is stored on the master node. When the master node allocates tasks, it usually first continuously adds tasks to the task queue of the master node, and the master node takes out the tasks to be assigned from the task queue, and finds the tasks that can meet the requirements by traversing the resource configuration information of each sub-node. The sub-nodes of the resou...

AI Technical Summary

Problems solved by technology

That is to say, when there is no sub-node that can satisfy all the resources required by the task to be allocated, the master node will always traverse the resource configuration information of each sub-node, even if there are resources required by other tasks in the task queue. When the sub-nodes can be satisfied, these other tasks cannot be assigned, that is, task blocking occurs

If the task blocking is not controlled, the concurrent task processing capability of the distributed task system and the resource utilization of each sub-node will be greatly reduced

Images