Task queuing method and device, computer equipment and storage medium

Abstract

The invention relates to a task queuing method and device, computer equipment and a storage medium. The method comprises the following steps: after tasks from different sources are received, summarizing the tasks from the same source into the same task group; putting the tasks of the different task groups into a task regulation and control queue in turn, and outputting the tasks of the multiple task groups through the task regulation and control queue; and monitoring the output tasks, obtaining the number of tasks in a preset state in the same task group, determining the corresponding task group as a target task group based on the task number range in which the task number is located, and stopping putting the tasks of the target task group into the task regulation and control queue. After the method is adopted, no matter how many task sources exist, excessive computing resources cannot be occupied, the task regulation and control queue can output multiple queues, efficiency can be guaranteed, the capacity upper limit of the system cannot be exceeded, and the performance and stability of the system are high.

Description

technical field [0001] The present application relates to the field of data processing, in particular to a task queuing method, device, computer equipment and storage medium. Background technique [0002] With the development of data processing technology, multiple modules may share unified computing resources to achieve full use of computing resources. In order to make full use of computing resources, the technology of independent task queues has emerged. In each independent task queue, its task scheduling is independent, independent of each other, and there is no need to wait for tasks in other queues to finish before they can be executed. [0003] When using the independent task queue technology, each task queue and execution thread are independent of each other, and users can execute tasks in a timely manner after submitting tasks without being forced to wait for tasks from other tenants; even if there are too many tasks that need to be queued, Users can also see that s...

AI Technical Summary

Problems solved by technology

[0004] However, its disadvantage is that when many modules are deployed in the environment, a large number of queues and execution threads need to be used. When multiple modules submit a large number of tasks at the same time, the load of the system will increase instantly, and the occupied resources may exceed the capacity of the system. The upper limit will seriously affect the performance and stability of the system, and may even cause the system to crash

Images