A method and device for cluster management task scheduling

Abstract

The present invention relates to the field of node allocation, and in particular to a method for cluster management task scheduling, which is arranged and allocated by reference to multiple dimensions such as task priority, number of user rights, running time, user utilization rate, etc., so as to achieve the reasonable allocation of limited computing power resources between users and tasks as much as possible, and realize the balanced allocation of internal resources. The beneficial effect of the present invention is that compared with traditional scheduling, the problem of docking and coupling between scheduling modules and other modules in traditional modules is overcome through the design of independent scheduling calculation modules, making the program more flexible and easy to expand. The two-in-one mode of interruption and startup logic is adopted. From the scheduling level, there is no need to care about the task status, whether it is a new task, an interrupted task or a restarted task, only the calculation result is needed, and the independence of the task scheduling module is improved from the bottom-level design, which provides conditions for more users, larger clusters and more complex platform service scenarios in the future.

Description

Technical Field

[0001] The present invention relates to the field of computing clusters, and in particular to a method and device for cluster management task scheduling. Background Art

[0002] With the development of artificial intelligence technology, more and more companies, universities, and research institutes are taking artificial intelligence as a new direction and new driving force for research and exploration. At present, the main field of development of artificial intelligence technology is deep learning, which combines low-level features to form more abstract high-level representation attribute categories or features to discover the distributed feature representation of data, thereby solving many problems that traditional machine learning algorithms could not solve. Deep learning models rely on large-scale sample training and computer hardware support. With the increase in business volume and the continuous improvement of model complexity, large-scale GPU clusters in computer rooms are needed to provide sufficient computing power for the training of numerous models.

[0003] When using a large-scale computing cluster for model training, it is necessary to consider the issues of computing power allocation and task scheduling. In the prior art, a single first-in-first-out queue scheduling or a static scheduling method of priority scheduling is usually adopted. The first-in-first-out method is to use a waiting queue, in order of submission time, and the first submitted task will be given priority to obtain the running permission. Priority scheduling is usually based on the priority of the task marked by the user or administrator, and the task with the highest priority in the task queue will be run first.

[0004] Traditional scheduling methods have many defects. Static scheduling means that it is impossible to respond to cluster conditions, including changes in user quotas, tasks, and cluster resources, and dynamically adjust task scheduling. Simple first-in-first-out can easily lead to low flexibility in the scheduler, and some urgent tasks cannot be scheduled; priority scheduling will cause some tasks to occupy resources for a long time, and cannot improve the overall efficiency of cluster use. In addition, the traditional scheduling method distinguishes between new and suspended tasks. The scheduler is used once each time a new task is created. Logically, the scheduling module is highly coupled with other modules, which is not conducive to the flexible expansion of the scheduling module. These problems need to be solved. Summary of the invention

[0005] In view of the above-mentioned defects existing in the prior art, the present invention provides a method and device for cluster management task scheduling, which can realize the most reasonable allocation of limited computing power resources between users and tasks by arranging and allocating them according to multiple dimensions such as task priority, number of user rights, running time, user utilization rate, etc., and achieve balanced distribution of internal resources to solve the problems existing in the existing scheduling technology.

[0006] The technical solution for achieving the purpose of the present invention is:

[0007] A method for cluster management task scheduling, triggering the following scheduling operation steps every time a scheduling time interval:

[0008] (1) Obtain training parameters for all tasks;

[0009] (2) Obtain information about available nodes in the cluster;

[0010] (3) Arrange in descending order of priority. The highest priority task in the priority list will be given priority to obtain the running permission. If there are still available nodes, the next priority task will be scheduled until the available nodes or task allocation are completed.

[0011] (4) Scheduling is completed and the task starts running.

[0012] Preferably, the task scheduling in the priority list described in step (3) comprises the following steps:

[0013] (3.1) Obtain the node quota of each user and calculate the number of nodes required by the user based on the number of nodes occupied by the tasks submitted by the user;

[0014] (3.2) If the sum of available nodes is greater than or equal to the sum of all user node requirements, then there are enough nodes, and the number of user node rights is equal to the number of user node requirements. According to the number of user node rights, available nodes are allocated so that the tasks corresponding to the number of user rights are allowed to run, and the allocation of tasks of this priority level is completed;

[0015] (3.3) If the number of available nodes is less than the sum of all user node requirements, the user node rights are the value obtained by allocating the available nodes among users according to the node quota ratio. According to the user node rights, the available nodes are allocated so that the tasks within the user rights are allowed to run.

[0016] (3.4) Among the remaining tasks, for the tasks whose continuous running time is less than the marked time, the task with the shorter continuous running time will be given priority in obtaining the running permission;

[0017] (3.5) If there are still available nodes, obtain the usage rate of each user, and the remaining available nodes are preferentially used for tasks submitted by users with low usage rates.

[0018] Preferably, the user usage rate in step (3.5) is the ratio of the nodes actually occupied by a user to the user's node quota.

[0019] Preferably, the method for calculating the number of user node rights described in step (3.3) is to approximate the scaling factor P through an iterative method, and P needs to reach the highest precision of floating point numbers. The number of user node rights = min (available nodes * the user node quota / the sum of all user node requirements, the user node requirement), and at the same time, the sum of all user node rights is equal to the number of idle nodes.

[0020] Preferably, the users involved are sorted in ascending order according to the user usage rate described in step (3.5), and each user can run and only run one task in sequence, and then the above rotation process is repeated until the available nodes and tasks are allocated, or the number of remaining task nodes is greater than the number of available nodes.

[0021] Preferably, during the rotation of each user's usage rate described in step (3.5), each user can only run one task in each round, and the user's tasks in the running or suspended state are given priority to obtain the running permission. After the allocation of the running and suspended state tasks is completed, when the user usage rate is rotated later, the tasks in the waiting state are selected to obtain the running permission.

[0022] As a preferred feature, the order in which each user submits a task to obtain a running permission follows the first-in-first-out principle, and the tasks submitted first are satisfied first, followed by the tasks submitted later in the order of their submission time.

[0023] Preferably, the scheduling time is set to 1 second.

[0024] Preferably, it is characterized in that the marking duration in step (3.4) is 15 minutes.

[0025] A computer device applying the above method comprises a memory, a processor, a bus, and a computer program stored in the memory and executable on the processor.

[0026] The beneficial effect of the present invention is to provide a method and device for cluster management task scheduling, realize the task allocation and scheduling management in cluster training, and make reference to multiple dimensions such as task priority, number of user rights, running time, user usage rate, etc. to arrange and allocate, so as to reasonably allocate limited computing resources between users and tasks as much as possible, and realize the balanced allocation of internal resources. Through user node quota, dynamic priority, first-in-first-out, priority allocation and other methods, the number of inefficient round-robin scheduling is reduced, the scheduling algorithm is made more stable, the waste of system resources is reduced, and the allocation efficiency is improved.

[0027] By triggering the scheduling algorithm once at each interval, the utilization rate of the computing cluster can be greatly improved. The response speed to changes in user node quotas, task training parameters, and cluster resources can be greatly improved, and dynamic adaptation to changes can be achieved, ensuring the overall efficient operation of the cluster computers.

[0028] Compared with traditional scheduling, the independent scheduling module design overcomes the problem of docking and coupling between scheduling modules and other modules in traditional modules, making the program more flexible and easy to expand. The two-in-one mode of interruption and startup logic is adopted. From the scheduling level, there is no need to care about the task status, whether it is a new task, an interrupted task or a restarted task. You only need to calculate the result, which improves the independence of the task scheduling module from the bottom-level design. It provides conditions for more users, larger clusters and more complex platform service scenarios in the future. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a schematic flow diagram of the method of the present invention;

[0030] Figure 2 is a schematic diagram of the task scheduling process in the same priority level of the present invention, DETAILED DESCRIPTION

[0031] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0032] In order to facilitate the implementation of the solution of the present application, the system platform presets a scheduling module, and implements the method S101-S107 of task scheduling through the scheduling algorithm in the scheduling module, so as to realize the reception of tasks and the reasonable allocation of node resources. The scheduling algorithm allocates and schedules resources for all tasks submitted by users, including tasks in waiting, running, and suspended states in addition to completed tasks. The scheduling algorithm runs once at every trigger time interval. In the event of new task creation and addition, task error, exit, etc., the scheduling algorithm is not triggered immediately, but waits until the interval trigger time, and then runs the scheduling algorithm once more.

[0033] In a specific embodiment, the scheduling trigger time can be set to 1 second.

[0034] like Figure 1 As shown, a method for implementing cluster management task scheduling is as follows: the following scheduling operation steps are triggered every time a scheduling trigger time is set:

[0035] S101 obtains all task training parameters; in specific implementation, the task can be a training program for machine learning algorithm code through a computer cluster, such as a neural network model training program (RNN), etc. After the user completes the code development of the training program through the client, it can be uploaded to the cluster training cloud service platform, create a training task, queue in the task list to wait for the allocation of computing node resources, deploy the environment and run the training task.

[0036] After the user submits a task, the task status is divided into four types: waiting to run, running, suspended, and completed. After a new task is submitted, it is in the waiting to run state and enters the waiting to run queue. After the node is allocated, the task that obtains the running permission enters the running state. During the running process, the task that needs to be interrupted due to the resource scheduling algorithm is in the suspended state. After being suspended, the scheduling algorithm can be used to re-obtain the running permission. The task that has completed the running is in the completed state.

[0037] In the specific implementation, all tasks involved in scheduling include tasks submitted by all users in the system that are in the waiting, running and suspended states.

[0038] In one embodiment, the task training parameters include the number of nodes occupied by the task and the task running priority;

[0039] Specifically, when a user creates a training task, he or she will submit the training parameters of the task at the same time. These training parameters submitted by the user include the machine training environment selection for the task to run, as well as the number of nodes occupied by the task and the task training priority. Based on the selected machine training environment components, the training environment container and container image are created through containerization technology, and the training environment such as the operating system, driver, configuration file, project training framework, etc. are synchronized to the assigned node computer to complete the automatic deployment of the training environment. The number of nodes occupied by the task indicates how many nodes are required to run the task. The task running priority is the priority setting in the task running. In a specific embodiment, the task running is divided into 6 priorities. High-priority tasks have priority running rights. After all high-priority tasks are assigned, they enter the task scheduling assignment of the lower priority.

[0040] S102 obtains information about available nodes in the cluster.

[0041] The available node information includes the number of available node resources. In this embodiment, the cluster includes a plurality of nodes, and the node refers to a computer server, including one or more GPUs, CPUs, etc. The GPU is used to receive the training task message distributed by the scheduler, load the corresponding algorithm model into the GPU memory according to the training task message, and execute the training task. The available nodes are nodes in all cluster nodes that are in normal operating state, not locked, not disabled, and not executing tasks. Among all cluster node computer servers, some node servers can be manually disabled, and some node servers are executing tasks. There will be a certain delay in the process of determining the task assignment to start running the training task, during which the node assigned to execute the task will be locked.

[0042] In this embodiment, a node computer server is assigned tasks as a whole, that is, a node computer server can only be assigned to one task at a time, and can be scheduled to be assigned to multiple tasks in different time periods.

[0043] S103 arranges the tasks in descending order of priority, and the tasks with the highest priority in the priority list are given the running permission first;

[0044] In this embodiment, the task with the highest priority is given priority to obtain the running permission, and the available nodes are allocated to run the task. The method for allocating and scheduling tasks within the same priority level is as follows: Figure 2 As shown, including:

[0045] 301 Get the node quota for each user.

[0046] In the specific implementation, each user is pre-configured with a certain number of node quotas, and the number of nodes used by all tasks submitted by the user in actual operation cannot exceed the user's node quota. It is worth noting that the node quota given to each user is pre-specified, and can also be adjusted in the background according to the actual situation during the training task operation.

[0047] 302 Calculate the number of user node requirements according to the number of nodes occupied by tasks submitted by the user;

[0048] In the specific implementation, the number of nodes required for the user can be calculated if the number of nodes occupied by the tasks submitted by the user and the node quota of the user are met. It is worth noting that tasks whose number of nodes occupied by the remaining tasks exceeds the user's node quota will not be run. For example, the user submitted 2 tasks, each of which occupies 4 nodes, and the user's node quota is 6. Then the user only has one task that is allowed to run, and the number of nodes required is 4. The user submitted 2 tasks, each of which occupies 2 nodes, and the user's node quota is 6. Then the number of nodes required is 4, and the user has not used up his quota;

[0049] It should be pointed out that the calculation of the number of user node requirements here should include all tasks submitted by the user that are in the waiting, running, and suspended states.

[0050] In this embodiment, since the user node quota is adjustable during operation, the required number of nodes may also change during the task operation.

[0051] 303: Whether the sum of available nodes is greater than or equal to the sum of the number of nodes required by all users; if so, proceed to step 304; if not, proceed to step 307.

[0052] 304 obtains the number of rights of each user node, which is equal to the number of user node requirements;

[0053] The number of user node rights refers to the number of nodes that the user can actually allocate under the current available node resources. When the number of available nodes is greater than or equal to the sum of the number of node requirements of all users, the number of node rights of each user is equal to the number of node requirements.

[0054] 305 Allocate available nodes according to the user's node rights so that tasks corresponding to the user's rights obtain running permission;

[0055] At this time, the available nodes are sufficient, because the number of node rights of each user is equal to the number of node requirements of the user. At this time, the available nodes are sufficient to ensure the operation of the tasks of this priority. Therefore, in this priority, each user submits all tasks that need to be run and can obtain a run license. Node resources are allocated according to the number of node rights of the user and the corresponding task, and the priority task allocation is completed in step 306.

[0056] 307 Obtaining the node rights of each user, where the node rights are obtained by allocating available nodes among users according to the node quota ratio;

[0057] When the number of available nodes is less than the sum of all user node requirements, the available nodes are insufficient to fully meet the operation of all tasks, and the available nodes need to be allocated among users according to the node quota ratio. Specifically, the scaling factor P is approximated by the iterative method, and P needs to reach the highest precision of floating point numbers. The number of user node rights = min (available nodes * the user node quota / the sum of all user node requirements, the user node requirement), and at the same time, the sum of all user node rights is equal to the number of idle nodes.

[0058] 308 Allocate available nodes according to the user's node rights so that tasks corresponding to the user's rights are allowed to run;

[0059] In the case that the available nodes cannot meet all user tasks of this priority, all tasks that do not exceed the rights of the user node can obtain running permissions. Through this allocation method, priority is given to ensuring that the resource requirements within the rights of the user node are met;

[0060] In this embodiment, the order in which each user submits tasks to obtain execution permission follows the first-in-first-out principle. According to the time sequence of task submission, the tasks submitted first are satisfied first, followed by the tasks submitted later.

[0061] 309 obtains tasks whose continuous running time is less than the marked time;

[0062] Specifically, for tasks other than the node rights, which are in the running state and whose continuous running time is less than the marked time, they are sorted in ascending order of continuous running time. The continuous running time refers to the continuous running time from the last time the task resumed running to the time when the scheduling algorithm was triggered. For tasks that have not been interrupted or suspended, the continuous running time is the time from the start of the task to the time when the scheduling algorithm was triggered;

[0063] In one embodiment, the marking duration may be 15 minutes.

[0064] 310 During continuous operation, the shorter the task, the higher the priority in obtaining the operation permission;

[0065] In ascending order of running time, tasks with short running time are given priority to be assigned available nodes and obtain running permission;

[0066] In addition to tasks with user node rights, tasks with short runtimes are prioritized. Specifically, tasks with continuous runtimes shorter than the marked duration are sorted in ascending order of continuous runtimes and are allocated runtime permissions in sequence. Tasks with short runtimes are prioritized in allocating available nodes and obtaining runtime permissions. This allocation method ensures that tasks with short runtimes are not interrupted as much as possible.

[0067] In particular, when the number of remaining available nodes cannot meet the number of nodes occupied by the task, the task is abandoned and the next task that meets the number of available nodes is found. In this embodiment, if there are 4 available nodes left at the end of the allocation, but the number of nodes occupied by the next task is 6 nodes, the task is abandoned and the next sequence is entered until the next task with less than 4 nodes is found. It is worth noting that since each user submits a task to obtain permission in accordance with the first-in-first-out time sorting principle, if the remaining available nodes cannot meet the number of nodes occupied by the task at the end of the allocation, the subsequent tasks of the user cannot obtain the running permission.

[0068] 311 checks whether there are any available nodes. If there are no available nodes, the process proceeds to step 306 to complete the priority allocation. If there are available nodes, the process proceeds to step 311 to continue the allocation step.

[0069] 312 Obtain the usage rate of each user;

[0070] In this embodiment, the user usage rate represents the ratio of the nodes actually used by a user to his node quota;

[0071] Specifically, a calculation is performed once at each trigger time, and the user's "instantaneous usage rate" = the number of nodes actually used by the user / "node quota". In particular, if the "graphics card quota" is 0, the "instantaneous usage rate" is also 0;

[0072] In one embodiment, the trigger time may be 1 second;

[0073] "User usage rate" R = the exponential decay average of "instantaneous usage rate" S, and the half-life is tentatively set to 3600 seconds. The calculation formula is as follows, where R is the user usage rate, S is the instantaneous usage rate, t is time, and the Math.pow(a, b) method is used to return the power of parameter a to parameter b;

[0074] Right now:

[0075]

[0076] In particular, when t=0, R0=0.

[0077] In this embodiment, since the user node quota is directly specified at the initial stage, it can also be adjusted in the background according to the actual situation during the training task. In order to avoid the dynamic adjustment of the "node quota" during the operation process, which will produce an excessively large value and affect the user experience, the algorithm of the instantaneous usage rate S can be adjusted to instantaneous usage rate S = min (the actual number of nodes used by the user / node quota, 2.0).

[0078] 313 The remaining available nodes are preferentially used for tasks submitted by users with low user utilization;

[0079] Specifically, the users involved are sorted in ascending order according to the user usage rate, and each user can run and can only run one task in order, and then the rotation process is repeated until the available nodes and tasks are allocated, or the number of remaining task nodes is greater than the number of available nodes. The available nodes are based on the user usage rate, giving priority to user tasks with low user usage rates. ;

[0080] In this embodiment, during the rotation of each user's usage rate, each user can only run one task in each round, and the tasks of the user in the running or suspended state are preferentially selected to obtain the running permission. After the distribution of the running and suspended tasks is completed, when the user usage rate is rotated later, the tasks in the waiting state are selected to obtain the running permission. The order in which each user obtains the running permission follows the first-in-first-out principle, and the permission is obtained in the order of the time when the tasks are submitted.

[0081] In particular, if there are 4 available nodes left at the end of the allocation, but the number of nodes occupied by the next task is 6, then the task will be abandoned and the next user will be transferred until the next task with less than 4 nodes is found. It is worth noting that since each user submits a task to obtain permission in accordance with the first-in-first-out time sorting principle, if the remaining available nodes cannot meet the number of nodes occupied by the task at the end of the allocation, the subsequent tasks of the user will not be able to obtain running permission;

[0082] In ascending order of user usage rate, each user can run only one task until all tasks of this priority level are assigned or the number of remaining task nodes required is greater than the number of available nodes;

[0083] In this way, the overall cluster utilization is improved, resource allocation is fairer, the number of invalid rotations is reduced, and the stability of task scheduling is increased.

[0084] 306 Priority tasks have been assigned.

[0085] S104 checks whether there are any available nodes for the next priority task to run. If yes, proceed to step S105. If no, proceed to step S107.

[0086] S105 checks whether there are any tasks to be executed or suspended at the next priority level. If yes, the process goes to step S106. If not, the process goes to step S107.

[0087] S106 enters the next priority in the priority list;

[0088] Specifically, when the highest priority task is scheduled, check whether there are any available nodes and whether there are any tasks in the next priority to be run or in a suspended state. If so, enter the next priority task scheduling, so that the highest priority task among the remaining tasks obtains the running permission, and repeat steps 301-312. If not, end the task scheduling.

[0089] S107 task scheduling is completed and starts running;

[0090] In this embodiment, the scheduling algorithm runs once every trigger time, and the tasks that obtain the running permission can be assigned to the available nodes for running. For the tasks that were previously in the waiting or suspended state, the training environment needs to be deployed before starting to run. For the tasks that have been running before, the running state is maintained without interruption;

[0091] Tasks that have not obtained the running permission in the scheduling algorithm will not be assigned to available nodes. For tasks that were previously in the waiting or suspended state, they will continue to wait in the queue in their original state. For tasks that were previously in the running state, the running will be interrupted and enter the suspended state, and the node of the task will be scheduled to be used by other tasks that have obtained the running permission;

[0092] Specifically, for the task to start running, the training environment is deployed on the allocated node through the container. Docker technology is used to achieve unified management of the server environment and overall planning of resources. The system pre-creates multiple training environment component image files, and according to the machine training environment pre-selected by the user, synchronizes the corresponding image files to create container components to complete the training environment deployment. After the deployment is completed, the training framework is loaded and the task training begins.

[0093] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0094] In addition, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0095] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk. It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0096] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention. It should be noted that similar numbers and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0097] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for cluster management task scheduling, characterized in that: Each scheduling time interval triggers the following scheduling operation steps: (1) Obtain training parameters for all tasks; (2) Obtain information about available nodes in the cluster; (3) Arrange in descending order of priority, and the task with the highest priority in the priority list will be given priority to obtain the running permission; if there are still available nodes, proceed to the next priority task scheduling until the available nodes or task allocation are completed; the priority task scheduling includes the following steps: (3.1) Obtain the node quota of each user and calculate the number of nodes required by the user based on the number of nodes occupied by the tasks submitted by the user; (3.2) If the sum of available nodes is greater than or equal to the sum of all user node requirements, then there are enough nodes, and the number of user node rights is equal to the number of user node requirements. According to the number of user node rights, available nodes are allocated so that the tasks corresponding to the number of user rights are allowed to run, and the allocation of tasks of this priority level is completed; (3.3) If the number of available nodes is less than the sum of all user node requirements, the user node rights are the value obtained by allocating the available nodes among users according to the node quota ratio. According to the user node rights, the available nodes are allocated so that the tasks within the user rights are allowed to run. (3.4) Among the remaining tasks, for the tasks whose continuous running time is less than the marked time, the task with the shorter continuous running time will be given priority in obtaining the running permission; (3.5) If there are still available nodes, obtain the usage rate of each user, and the remaining available nodes are preferentially used for tasks submitted by users with low user usage rates; (4) Scheduling is completed and the task starts running.

2. The cluster management task scheduling method according to claim 1, characterized in that: The user usage rate in step (3.5) is the ratio of the nodes actually occupied by a user to the user's node quota.

3. The cluster management task scheduling method according to claim 1, characterized in that: The calculation method of the user node rights number described in step (3.3) is to approximate the scaling factor P through an iterative method. P needs to reach the highest precision of floating point numbers. The user node rights number = min (available nodes * the user node quota / the sum of all user node requirements, the user node requirement number), and at the same time, the sum of all user node rights numbers is equal to the number of idle nodes.

4. The cluster management task scheduling method according to claim 1, characterized in that: The users involved are sorted in ascending order according to the user usage rate described in step (3.5). Each user can run and can only run one task in order, and then the above rotation process is repeated until the available nodes and tasks are allocated, or the number of remaining task nodes is greater than the number of available nodes.

5. The cluster management task scheduling method according to claim 4, characterized in that: During the rotation of each user's usage rate described in step (3.5), each user can only run one task in each round, and the user's tasks in the running or suspended state are given priority to obtain the running permission. After the allocation of the running and suspended state tasks is completed, when the user usage rate is rotated later, the tasks in the waiting state are selected to obtain the running permission.

6. The cluster management task scheduling method according to claim 1, characterized in that: The order in which each user submits a task and obtains the running permission follows the first-in-first-out principle. The tasks submitted first are given priority, followed by the tasks submitted later.

7. The cluster management task scheduling processing method according to claim 1, characterized in that: The scheduling time is set to 1 second.

8. The cluster management task scheduling method according to claim 1, characterized in that: The marking time in step (3.4) is 15 minutes.

9. A computer device using the method according to any one of claims 1 to 8, characterized in that: It includes a memory, a processor, a bus, and a computer program stored in the memory and executable on the processor.

Images