Automatic task arrangement method and device, computer equipment and storage medium

Abstract

The embodiment of the invention discloses an automatic task arrangement method and device, computer equipment and a storage medium. The method comprises the steps: obtaining the current idle resourcecondition of a big data cluster if a submitting instruction of a plurality of tasks is received; obtaining one of the plurality of tasks as a target task, and determining a resource demand for executing the target task; judging whether the current idle resource condition of the big data cluster meets the resource demand required for executing the target task or not; If the current idle resource condition of the big data cluster cannot meet the resource demand required for executing the target task, selecting another task from the plurality of tasks as the target task, and returning to executethe step of determining the resource demand for executing the target task. According to the method, the tasks can be reasonably arranged, it is guaranteed that the tasks are smoothly executed under the condition of limited resources, and the task execution efficiency is improved.

Description

technical field [0001] The present invention relates to the technical field of computers, in particular to a task automatic arrangement method, device, computer equipment and storage medium. Background technique [0002] In the existing big data system, the core function of the task scheduling system is to submit tasks to the big data cluster for execution. No matter whether the task is temporarily submitted by the user or regularly submitted, the task will eventually be submitted to the big data cluster for execution. Data clusters have certain resource limitations. If the submission of tasks is not properly arranged, a large number of tasks will compete for a small amount of running resources, and eventually all tasks will run very slowly, making it difficult to meet the expected demand. Contents of the invention [0003] In view of this, the embodiments of the present invention provide an automatic task arrangement method, device, computer equipment and storage medium, ...

AI Technical Summary

Problems solved by technology

[0002] In the existing big data system, the core function of the task scheduling system is to submit tasks to the big data cluster for execution. No matter whether the task is temporarily submitted by the user or regularly submitted, the task will eventually be submitted to the big data cluster for execution. Data clusters have certain resource limitations. If the submission of tasks is not properly arranged, a large number of tasks will compete for a small amount of running resources, and eventually all tasks will run very slowly, making it difficult to meet the expected needs.

Images