Cluster job scheduling method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses a cluster job scheduling method and device, electronic equipment and a storage medium, relates to the technical field of computers, and is convenient for improving the overall resource utilization rate of a cluster. The method comprises the steps that a job mark carried in a first job is obtained, the job mark is used for indicating the matching scheduling relation between the job and at least two priority queues, and the at least two priority queues comprise the first priority queue and the second priority queue; according to the job mark, judging whether the first priority queue has running resources required for running and processing the first job or not; if not, judging whether the second priority queue has running resources required for running and processing the first job or not; and if the second priority queue has the running resources required for running and processing the first job, distributing the first job to the second priority queue for processing. The method is suitable for the computing cluster.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a cluster job scheduling method, device, electronic equipment and storage medium. Background Art

[0002] With the increasing demand for computing, computing clusters have become an important infrastructure for high-performance computing and large-scale data processing. Computing clusters are usually divided into multiple queues, where a queue is a collection of computers, and the computers in a queue are also called computing nodes. Traditional cluster job scheduling methods usually assign jobs to fixed resource queues. Since jobs can only run on a single queue, resource utilization is not high, especially when resource demand fluctuates greatly, many resources may be idle.

[0003] For example, in exclusive resource queues, some queues may have excess resources while other queues may have tight resources, resulting in insufficient utilization of resources. In shared resource queues, the resource sharing ratio and priority are preset by the administrator, which lacks flexibility and cannot dynamically allocate resources according to actual needs, which may lead to idle resources or failure to meet job requirements, affecting the overall resource utilization of the cluster. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a cluster job scheduling method, device, electronic device and storage medium, which are convenient for improving the overall resource utilization of the cluster.

[0005] In order to achieve the above-mentioned invention object, the following technical scheme is adopted: In a first aspect, an embodiment of the present invention provides a cluster job scheduling method, the method comprising: obtaining a job tag carried in a first job, the job tag comprising an operating resource requirement of the first job and a method for indicating a matching scheduling relationship between the job and at least a first priority queue and a second priority queue; based on the job tag, determining whether the first priority queue has operating resources required to run and process the first job; if not, determining whether the second priority queue has operating resources required to run and process the first job; if the second priority queue has operating resources required to run and process the first job, distributing the first job to the second priority queue for processing.

[0006] In combination with the first aspect, in a first implementation of the first aspect, after determining whether the first priority queue has the running resources required to run and process the first job, the method also includes: if the first priority queue has the running resources required to run and process the first job, distributing the first job to the first priority queue for processing; listening to the job processing completion status fed back by the first priority queue; if a job is currently completed, querying the first priority queue whether there is a second job in a suspended state; if so, controlling the second priority queue to resume running and processing the suspended second job, and obtaining the status of the second job resuming running and processing.

[0007] In combination with the first aspect and the first implementation method of the first aspect, in the second implementation method of the first aspect, after determining that the first priority queue does not meet the running resources required to run and process the first job, the method also includes: controlling the first priority queue to suspend the low-priority job currently being processed or canceling at least one job mark indicating that the job should match the highest priority queue other than the current first priority queue, so as to release occupied resources.

[0008] In combination with the first aspect and the first or second implementation manner of the first aspect, in a third implementation manner of the first aspect, after determining that the first priority queue does not meet the running resources required for running and processing the first job, the method further includes: determining whether the running resources released by suspending the low-priority job currently being processed by the first priority queue are sufficient to meet the processing of the first job; If so, the first priority queue is controlled to suspend the low priority job currently being processed; If not, continue to determine whether the resources released by the first priority queue canceling at least one third job currently being processed are sufficient to process the first job; wherein the highest priority queue matched by the third job is not the current first priority queue; If so, the first job is assigned to the first priority queue for processing, and the status of the third job cancellation and the first job running processing is obtained.

[0009] After determining whether the second priority queue has the operating resources required to run and process the first job, the method further includes: if the second priority queue does not have the operating resources required to run and process the first job, continuing to determine whether the remaining priority queues have the operating resources required to run and process the first job; if so, distributing the first job to the corresponding priority queues for processing; if not, re-running the next round of determination from the first priority queue in sequence to determine whether there are the operating resources required to run and process the first job.

[0010] In combination with the first aspect and the first, second, third or fourth implementation of the first aspect, in the fifth implementation of the first aspect, the matching scheduling relationship includes: the association between the first job and at least two priority queues; and, indicating that when the first priority queue cannot meet the resource conditions required for the first job, the first job is allowed to be transferred to other priority queues for resource condition judgment; and, indicating the priority order of matching scheduling of the first job between multiple priority queues.

[0011] In a second aspect, an embodiment of the present invention provides a cluster job scheduling device, comprising: an acquisition program unit, used to acquire a job tag carried in a first job, the job tag including the running resource requirements of the first job and a matching scheduling relationship indicating the job and at least a first priority queue and a second priority queue; a first judgment program unit, used to judge whether the first priority queue has the running resources required to run and process the first job based on the job tag; a second judgment program unit, used to judge whether the second priority queue has the running resources required to run and process the first job if not; a first distribution program unit, used to distribute the first job to the second priority queue for processing if the second priority queue has the running resources required to run and process the first job.

[0012] In combination with the second aspect, in a first implementation of the second aspect, the device also includes: a second distribution unit, for, after determining whether the first priority queue has the running resources required to run and process the first job, if the first priority queue has the running resources required to run and process the first job, then distributing the first job to the first priority queue for processing; a first listening program unit, for listening to the job processing completion status fed back by the first priority queue; a first query program unit, for querying the first priority queue whether there is a second job in a suspended state if a job is currently completed; and a first control program unit, for controlling the second priority queue to resume running and processing the suspended second job, if so, and obtaining the status of resuming running and processing of the second job.

[0013] In combination with the second aspect and the first implementation of the second aspect, in the second implementation of the second aspect, the device also includes: a second control program unit, which is used to control the first priority queue to suspend the low-priority job currently being processed or cancel at least one job mark indicating that the job should match the highest priority queue other than the current first priority queue, so as to release occupied resources after determining that the first priority queue does not meet the running resources required to run and process the first job.

[0014] In combination with the second aspect and the first or second implementation of the second aspect, in the third implementation of the second aspect, the first judgment program unit is also used to judge whether the running resources released by the first priority queue suspending the low priority job currently being processed are sufficient to process the first job after judging that the first priority queue does not meet the running resources required for running and processing the first job; the second control program unit is used to control the first priority queue to suspend the low priority job currently being processed if so; the third judgment program unit is used to continue to judge whether the resources released by the first priority queue canceling at least one third job currently being processed are sufficient to process the first job if not; wherein the highest priority queue matched by the third job is not the current first priority queue; the third distribution program unit is used to assign the first job to the first priority queue for processing if so, and obtain the status of the third job cancellation and the first job running processing.

[0015] In combination with the second aspect and the first, second or third implementation of the second aspect, in the fourth implementation of the second aspect, the first distribution program unit is also used to continue to determine whether the remaining priority queues have the operating resources required to run and process the first job if the second priority queue does not have the operating resources required to run and process the first job; if so, distribute the first job to the corresponding priority queue for processing; if not, re-run the next round of determination from the first priority queue in sequence to determine whether there are the operating resources required to run and process the first job.

[0016] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: one or more processors; a memory; one or more executable programs are stored in the memory, and the one or more processors read the executable program code stored in the memory to run the program corresponding to the executable program code, so as to run any of the methods described in the first aspect.

[0017] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement any method described in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1a This is a typical queue organization configuration structure diagram; Figure 1b This is another typical queue organization configuration structure diagram; Figure 2 It is a flow chart of an embodiment of a cluster job scheduling method in the prior art; Figure 3 It is a schematic diagram of the structure of an embodiment of a computing cluster or an electronic device for job processing of the present invention; Figure 4 A schematic diagram of a process flow of an embodiment of a cluster job scheduling method of the present invention; Figure 5 A schematic diagram of a flow chart of another embodiment of the cluster job scheduling method provided by the present invention; Figure 6 A schematic block diagram of the architecture of an embodiment of a cluster job scheduling device provided by the present invention; Figure 7 The figure is a schematic diagram of the structure of an embodiment of the electronic device of the present invention. DETAILED DESCRIPTION

[0020] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0021] It should be clear that the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] A computing cluster is a computing system consisting of multiple computers connected by a high-speed network. The computers in the cluster system work together to provide more powerful computing power and reliability than a single computer. Computing clusters solve large-scale problems through parallel processing, aiming to shorten job running time and improve job running efficiency.

[0023] Computing clusters are usually divided into multiple queues. A queue is a collection of computers, and the computers in a queue are also called computing nodes. Different queues serve different types of jobs. For example, some jobs require multiple computing cores but not much memory, some jobs require a large memory space but require fewer computing cores, some jobs require a short waiting time but a short running time, and some jobs can wait in line for a long time but have a long running time. The cluster system will configure different host queues according to the job type's demand for running resources.

[0024] Figure 1aThe diagram illustrates a queue organization configuration structure of a computing cluster, which is a typical way in which queues monopolize host resources. This will result in some queues being heavily loaded and some queues being lightly loaded. When the loads of different queues are unbalanced, since jobs are running on a single queue, it is difficult to allocate resources of idle queues to process jobs running on busy queues, resulting in a waste of resources. Figure 1b This diagram shows another queue organization configuration structure of a computing cluster, which is a queue sharing host resource method. Although the host resources of the idle queue can be allocated to run the jobs that the busy queue needs to run to a certain extent, since the proportion of shared resources is determined in advance by the administrator, it is impossible to achieve full on-demand dynamic sharing and allocation of resources, and there will still be idle resources. For the above two queue organization configuration structures, the current typical job scheduling process is as follows: Figure 2 As shown in the figure, since the job runs on a single queue, or shares running resources in two or more designated queues, after determining that the fixed designated queue does not have enough running resources, it will be in a waiting state. Even if other queues may have idle resources, since it is pre-specified to run in a fixed queue, the current job will not run until the designated queue has enough running resources. This static scheduling method will cause idle resources in the entire computing cluster.

[0025] In order to improve the overall resource utilization of the cluster, the present application provides a cluster job scheduling method. When the job is submitted, a job tag is configured for the job in advance, and the job tag is used to indicate the matching relationship between the job and multiple priority queues. In this way, when the job is received, according to the job tag and the available resources of the multiple priority queues, dynamic scheduling and allocation are performed among the multiple priority queues indicated in the job tag, ensuring efficient utilization of cluster resources and priority running of jobs, thereby improving the overall resource utilization of the cluster.

[0026] In a specific implementation process, the queue configuration of the computing cluster involved in the present invention may be a typical exclusive resource queue or a shared resource queue. Figure 3 A schematic diagram of the computing cluster structure provided by an embodiment of the present invention is shown. Figure 3 In some embodiments, the computing cluster 200 may include: a cluster job scheduling system 201 and a plurality of host queues, for example, Figure 3Two host queues are shown in FIG. 2 , including a first priority queue 210 and a second priority queue 220. Of course, this is only for illustration, and the number of host queues can be specifically set according to the scale of the computing cluster 200. Each host queue includes at least one entry node controller 211 (221) and multiple computing nodes 212 (222). The computing nodes may include hardware resources such as servers, storage nodes, network devices, and IO interfaces for running jobs. The cluster job scheduling system 201 is responsible for receiving job requests from the user terminal device 100, and generating a job tag for the job indicating the priority of each job, the corresponding matching multiple priority queues, and resource requirements according to the user operation instructions; in addition, the cluster job scheduling system 201 is also used to monitor the usage status of queue resources in real time to achieve dynamic allocation of running resources; specifically, it is also used to run a specific job dynamic scheduling strategy according to the job priority indicated by the job tag and the matching priority queue resource status, and according to the available status of the running resources of the multiple priority queues with matching relationships, according to the job priority indicated in the job tag, the first job is allocated to the corresponding priority queue for running.

[0027] See also Figure 4 In some embodiments, the cluster job scheduling method provided by the embodiment of the present invention can efficiently schedule jobs between multiple priority queues to ensure effective allocation and reasonable use of resources. The method includes: S110: Obtain a job tag carried in the first job.

[0028] In this embodiment, the cluster job scheduling system 201 receives a first job from a client or a job submission platform. The first job carries a job tag, which at least includes the resource requirement information of the first job and is used to indicate the matching scheduling relationship between the job and multiple priority queues. Specifically, the matching scheduling relationship may include: the association between the first job and at least two priority queues; for example, in this embodiment, the job tag will list the running resource requirements required by the first job and the first priority queue 210 and the second priority queue 220 associated with the first job. If the identifier of the first priority queue is Memory1, indicating a large memory host queue, and the identifier of the second priority queue is Compute2, indicating a large computing power host queue, the job tag will list multiple associated queues including the large memory host queue and the large computing power host queue, so as to schedule between multiple host queues. And, it indicates that when the first priority queue cannot meet the resource conditions required by the first job, the first job is allowed to be transferred to other priority queues for resource condition judgment, that is, the cluster manager is informed to allow the above multiple associated queues to be scheduled to another priority queue to judge whether the resource conditions required by the job are met when one priority queue cannot meet the resource conditions required by the first job. For example, for a chip performance test job, the test is first performed in the memory test queue. If the memory resources are insufficient, the job can be transferred to the CPU test queue for subsequent testing according to the above content indicated by the job tag. And, the first job is instructed to match the scheduling priority order between multiple priority queues. When there are multiple queues, the scheduling order is clarified between multiple associated queues. Still taking the aforementioned chip performance test as an example, for a chip performance test job, when the running resources required by the job to be tested are high in memory resource requirements, the first priority queue can be indicated in the job tag as a large memory host queue. If the memory resources of the first priority queue are insufficient, the job is transferred to the second priority queue according to the scheduling priority order of the job tag to continue to judge whether the resource conditions are met. Multiple priority queues include at least: a first priority queue 210 and a second priority queue 220. Specifically, the information carried by the job tag may include the priority of the job, the running resource requirements, such as memory, CPU, GPU, etc., and the matching dependency association relationship with the priority queue. Of course, when the job to be run and processed is determined, the job tag may not carry the priority of the job.

[0029] Exemplarily, when submitting a job, the user can add the job tag, and mark the various priority queues that match the job according to the running resource requirements of the job and the resource characteristics of different queues. For example, for a job that requires large memory resources, the first priority queue 210 is a large memory queue, and the second priority queue 220 can be a computing power priority queue with a relatively large number of CPU cores. The first priority queue 210 is the main queue or the highest priority queue configured for the job, and other queues including the second priority queue 220 are host queues added to utilize idle resources. Among them, the method of adding the job tag is agreed according to the implementation method of the job scheduling system, and can be implemented in the form of formatted text, command line, etc. By indicating the matching scheduling relationship between the job and multiple priority queues in the job tag, after obtaining the job tag, the job can be dynamically scheduled between different priority queues according to the job tag to achieve reasonable utilization of resources.

[0030] S120 , judging whether the first priority queue 210 has running resources required to meet the first job according to the job tag.

[0031] The scheduling system determines whether the first priority queue 210 in the cluster has sufficient resources to process the first job based on the running resource requirements in the job tag carried by the first job and the associated first priority queue and second priority queue. For example, the available resources in the first priority queue 210, such as CPU, memory, GPU and other running resources, can be evaluated and compared with the resources required for the first job to determine whether the running resources required for the first job are met.

[0032] If the resources in the first priority queue 210 meet the needs of the job, the scheduling system assigns the job to the first priority queue 210 for processing. Otherwise, the scheduling system proceeds to the next step to determine whether the second priority queue 220 has sufficient resources to process the first job based on the scheduling matching relationship with the second priority queue 220 indicated in the job tag.

[0033] S130 , if the first priority queue 210 has insufficient resources, determine whether the second priority queue 220 has operating resources required to meet the first job.

[0034] Specifically, if the resources in the first priority queue 210 are insufficient, the scheduling system will determine whether the second priority queue 220 has available resources based on the job tag. The resource scheduling requirements of the second priority queue 220 are usually low, so the system will determine whether the first job can be allocated to run based on the resource usage of the second priority queue 220. If the second priority queue 220 can meet the resources required by the first job, the scheduling system will distribute the first job to the second priority queue 220 for processing. Even if the priority of the queue is low, due to the sufficiency of resources, the system will still give priority to scheduling the job to the second priority queue 220 with more sufficient available resources, thereby avoiding the idleness of some queue running resources and improving the overall resource utilization.

[0035] For example, still taking the performance test job of a server chip as an example, the test process of the chip involves multiple key links, one of which has strict requirements on storage resources. When the user submits the test job, he adds a job tag, including the requirements for the storage bandwidth resources of the host queue, and associates multiple host queues. Among them, the first priority queue 210 is set as a high-bandwidth storage test dedicated queue, and its internal host is equipped with a memory with extremely high data read and write speeds, marked as StorageBWTest1, which is mainly used to execute data exchange tests between the chip's built-in cache and external storage, high-concurrency storage read and write stress tests, and other sub-tasks that are sensitive to storage bandwidth.

[0036] After receiving the test job, the scheduling system parses the job tag and allocates the test job to the first priority queue 210 with priority. Before allocation, it determines whether the first priority queue meets the running resource conditions required for the first job based on the running resource requirements carried by the job tag. When the first priority queue identified by StorageBWTest1 runs the storage test subtask, if it is continuously monitored that the storage device is overheating and slowing down, or the bandwidth is occupied due to the simultaneous execution of multiple high-load storage tasks, it will feedback that there are not enough running resources required for the job. The scheduling system allows the first job to be transferred to other priority queues for resource condition judgment according to the information indicated in the matching scheduling relationship. The second priority queue 220 is a multi-core computing capability test queue. The host is equipped with a multi-core high-performance CPU and is equipped with a large-bandwidth memory to ensure fast data exchange. Its identity is CPUCoreTest2. At this time, the scheduling system matches the scheduling priority order among multiple priority queues according to the first job indicated by the job tag. When the current first priority queue does not have sufficient running resources to run the first job, the job preparation scheduling is transferred to the second priority queue marked by CPUCoreTest2 according to the scheduling priority order carried in the job tag, and it is judged again whether the currently idle running resources of the second priority queue can meet the processing of the first job, so as to realize scheduling among multiple priority queues, so that the job will not be in a waiting state due to insufficient resources of a single host queue, thereby improving the overall resource utilization.

[0037] S140: If the second priority queue 220 has the required running resources to run and process the first job, the first job is distributed to the second priority queue 220 for processing.

[0038] After determining that the resources of the second priority queue 220 meet the requirements of the first job, the scheduling system distributes the first job to the second priority queue 220 to start running. In addition, the scheduling system notifies the second priority queue 220 to allocate corresponding running resources for the first job and start the job running process.

[0039] The cluster job scheduling method provided by an embodiment of the present invention carries a job tag in the first job to indicate that multiple priority queues are allowed to be dynamically scheduled to run and process the first job, making the scheduling process more intelligent and efficient. When the operating resources of the first priority queue 210 are tight, the second priority queue 220 or other priority queues that still have available resources are allowed to be scheduled to run the operation and processing of the first job, avoiding idle waste of resources or failure to meet job requirements, thereby ensuring maximum utilization of cluster resources and facilitating improving the overall resource utilization of the cluster.

[0040] Figure 5The cluster job scheduling method provided by an embodiment of the present invention is briefly illustrated; see Figure 5 In some embodiments, after determining whether the first priority queue 210 has the operating resources required to run and process the first job (S120), the method further includes: if the first priority queue 210 has the operating resources required to run and process the first job, the first job is distributed to the first priority queue 210 for processing.

[0041] In this embodiment, if the first priority queue 210 has sufficient resources, the scheduling system will distribute the first job to the first priority queue 210 for processing. The first priority queue 210 will run the matching resource node to which the job is assigned until completion, and report the job completion status to the monitoring module of the job scheduling system.

[0042] Monitor the job processing completion status fed back by the first priority queue 210; if a job is currently running and completed, query the first priority queue 210 whether there is a second job in a suspended state.

[0043] When a job in the first priority queue 210 receiving feedback is completed, the queue is queried to see if there is a second job in a suspended state. The suspended job is usually a job that fails to run in time due to insufficient resources, unsatisfied dependencies, or scheduling priority queue matching problems. In this way, through the job running processing status feedback mechanism, it can be decided whether to resume processing of the suspended job according to the priority and resource requirements of the suspended second job.

[0044] If there is a suspended second job, the first priority queue 210 is controlled to resume processing the suspended second job, and the status of the second job resuming processing is obtained. If a suspended job is found, for example, the second job is suspended, and the second job meets the current resource allocation conditions, the second job is resumed. Of course, the scheduling system will continue to monitor the resumed operation status of the second job to perform overall resource scheduling.

[0045] In this embodiment, by real-time monitoring of the job running status and automatic recovery of suspended jobs, it is ensured that the jobs in the first priority queue 210 can run efficiently, and the waste caused by idle resources is reduced, thereby improving the overall job scheduling efficiency.

[0046] In order to give priority to running the job in the first priority queue 210 to improve resource utilization, if the resources of the first priority queue 210 are insufficient, further measures can be taken to run the job in the highest priority queue corresponding to the first job.

[0047] Therefore, in some embodiments, after determining that the first priority queue 210 does not meet the running resources required to run and process the first job, the method further includes: controlling the first priority queue 210 to suspend the low-priority job currently being processed or canceling at least one job mark indicating that the job should match the highest priority queue other than the current first priority queue 210, so as to release occupied resources.

[0048] In this embodiment, when the first priority queue 210 cannot provide sufficient resources, the job scheduling system determines whether resources can be released by suspending the currently running low-priority job, and selects a job with a lower priority, or the highest priority queue indicated in the job tag is not the current first priority queue 210 for suspension. The job status after suspension is recorded and fed back to the job scheduling system. The occupied processor resources, I / O resources and software licenses can be released through the suspension operation, but the memory resources may not be released immediately. Therefore, if the resource requirements of the first job include memory resources, the first priority queue 210 can quickly release some resources through the suspension operation, but it still cannot meet the running resources required for running and processing the first job.

[0049] Therefore, in other embodiments, after determining that the first priority queue 210 does not meet the running resources required to run and process the first job, the method further includes: determining whether the running resources released by the first priority queue 210 suspending the low priority job currently being processed are sufficient to meet the needs of processing the first job; if so, controlling the first priority queue 210 to suspend the low priority job currently being processed; if not, continuing to determine whether the resources released by the first priority queue 210 canceling at least one third job currently being processed are sufficient to process the first job; wherein the highest priority queue matched by the third job is not the current first priority queue 210; if so, assigning the first job to the first priority queue 210 for processing, and obtaining the status of the third job cancellation and the first job running and processing.

[0050] Specifically, if the suspend operation still cannot meet the resource requirements required for the operation of the first job, it is further determined whether the low priority job can be canceled. A third job with a lower priority or a queue with the highest priority indicated by the mark other than the current queue can be selected for cancellation to free up more resources. The canceled third job can be rearranged to a suitable queue for re-run. After the third job is canceled, the cancellation of the third job is obtained to recalculate the resource status of the cluster 200 to schedule and optimize resource allocation.

[0051] Specifically, after the operation is suspended or canceled, if the resource requirements for the operation and processing of the first job can be met, the first job is distributed to the first priority queue 210 for processing. At the same time, the job scheduling system continuously monitors and manages the status of the job to ensure that the suspended or canceled job is resumed or scheduled when resources permit. Specifically, a job monitoring module can be set in the scheduling system to track the operation status of the job throughout the process, ensure reasonable allocation of resources, and achieve the goal of optimal utilization of cluster resources. In this embodiment, by monitoring the job status of the priority queue operation and processing and dynamically scheduling resources, the resource utilization of the cluster can be improved and the management burden of the IT team can be reduced.

[0052] If after suspending or canceling the currently running job, the released resources still cannot meet the resource requirements for running the first job, it is determined whether the second priority queue 220 has the running resources required for running and processing the first job.

[0053] In the existing cluster job scheduling system 201, jobs are usually assigned to fixed priority queues for processing. However, when the resources in a certain priority queue are insufficient to meet the needs of the current job, the system is often unable to effectively and dynamically adjust the scheduling strategy, causing the job to be in a waiting state for a long time. This static scheduling method not only reduces resource utilization, but may also cause delayed operation of high-priority jobs, affecting the overall scheduling efficiency.

[0054] In some embodiments, after determining whether the second priority queue 220 has the operating resources required to meet the first job, the method further includes: if the second priority queue 220 does not have the operating resources required to run and process the first job, continuing to determine whether the remaining priority queues have the operating resources required to run and process the first job.

[0055] In this embodiment, in the cluster scheduling system, jobs are allowed to run in different priority queues by adding job tags. When the first priority queue 210 cannot meet the resource requirements required for operation, the scheduling system will determine whether there are enough resources in the second priority queue 220 to run the first job. At this time, the system can obtain resource information in the second priority queue 220, including computing resources such as CPU, memory, storage, etc. and other necessary resources such as network bandwidth, GPU, etc. Of course, the specific resource information to be obtained is related to the resource requirements required for the operation and processing of the first job.

[0056] If yes, the first job is distributed to the corresponding priority queue for processing; if no, the next round of judging whether there are sufficient operating resources required for processing the first job is executed again from the first priority queue 210 in sequence.

[0057] Specifically, if there are sufficient resources in the second priority queue 220, the first job will be assigned to the queue and start running. The specific determination may include load monitoring of nodes in the second priority queue 220, resource occupancy, and calculation of remaining resources.

[0058] If the resources of the second priority queue 220 are insufficient to process the first job, the scheduling system will continue to determine whether other priority queues have the resources required to run the job. The specific determination process is as follows: Query the remaining priority queues, such as the third priority queue, the fourth priority queue, etc., and check the third priority queue and the subsequent priority queues in turn to determine whether any of the priority queues has sufficient resources to run the first job. If a queue meets the conditions, the first job is assigned to the priority queue that has sufficient resources to run the first job.

[0059] If all priority queues cannot provide enough resources to run the first job, then return to the first priority queue 210, re-run resource judgment from the beginning, scan resource status, and dynamically capture the moment of resource release to ensure that the job can be scheduled and run as soon as resources are available. Of course. In order to prevent the system from being overloaded, a scanning cycle or resource threshold can be set to optimize the response speed of resource scheduling.

[0060] In this embodiment, when cluster resources are tight, resource conditions are checked cyclically to avoid resource waste and optimize job scheduling efficiency, ultimately achieving flexibility and efficiency in job resource scheduling, thereby achieving overall cluster resource utilization.

[0061] It should be noted that the cluster job scheduling method provided in the embodiment of the present invention can be solidified in a certain manufactured physical product in the form of software, and when the user uses the product, the method flow of the present application can be reproduced.

[0062] The resource scheduling method provided in the embodiment of the present application is introduced above in combination with Figures 1 to 3. Next, the structure of the scheduling device provided in the embodiment of the present application and the structure of the electronic device for processing jobs are introduced in combination with the accompanying drawings.

[0063] See also Figure 6 , shows a schematic diagram of the structure of a cluster job scheduling device, the scheduling device 200 includes: An acquisition program unit 310 is used to acquire a job tag carried in the first job, where the job tag is used to indicate a matching scheduling relationship between the job and at least the first priority queue 210 and the second priority queue 220; A first judgment program unit 320, configured to judge, according to the job tag, whether the first priority queue 210 has the running resources required to run and process the first job; A second determination program unit 330 is configured to determine whether the second priority queue 220 has the required operating resources to run and process the first job if the first priority queue 210 does not have the required operating resources to run and process the first job; The first dispatching program unit 340 is configured to dispatch the first job to the second priority queue 220 for processing if the second priority queue 220 has the required running resources for running and processing the first job.

[0064] In some embodiments, the cluster job scheduling device further includes: a second distribution unit, configured to, after determining whether the first priority queue 210 has the running resources required to run and process the first job, distribute the first job to the first priority queue 210 for processing if the first priority queue 210 has the running resources required to run and process the first job; A first monitoring program unit, configured to monitor the job processing completion status fed back by the first priority queue 210; A first query program unit, configured to query the first priority queue 210 whether there is a second job in a suspended state if a job is currently running and completed; The first control program unit controls the second priority queue 220 to resume processing the suspended second job if it is found that the first priority queue 210 has a second job in a suspended state, and obtains a state in which the second job resumes processing.

[0065] In some embodiments, the cluster job scheduling device also includes: a second control program unit, which is used to control the first priority queue 210 to suspend the low-priority job currently being processed or cancel at least one job mark indicating that the job should match the highest priority queue other than the current first priority queue 210, so as to release occupied resources after determining that the first priority queue 210 does not meet the running resources required to run and process the first job.

[0066] In some embodiments, the first determination program unit 320 is further configured to determine whether the running resources released by suspending the low-priority job currently being processed by the first priority queue 210 are sufficient to process the first job after determining that the first priority queue 210 does not meet the running resources required for running and processing the first job; A second control program unit is used to control the first priority queue 210 to suspend the low priority job currently being processed if the running resources released by the first priority queue 210 suspending the low priority job currently being processed are sufficient to process the first job; A third judgment program unit is used to continue to judge whether the resources released by the first priority queue 210 canceling at least one third job currently being processed are sufficient to process the first job if the running resources released by the first priority queue 210 suspending the low priority job currently being processed are not sufficient to process the first job; wherein the highest priority queue matched by the third job is not the current first priority queue 210; The third dispatcher unit is used to allocate the first job to the first priority queue 210 for processing if it is determined that the resources released by canceling at least one third job currently being processed by the first priority queue 210 are sufficient to process the first job, and obtain the status of the third job cancellation and the first job running processing.

[0067] In some embodiments, the first dispatch program unit 340 is also used to continue to determine whether the remaining priority queues have the operating resources required to run and process the first job if the second priority queue 220 does not have the operating resources required to run and process the first job; if so, distribute the first job to the corresponding priority queue for processing; if not, re-run the next round of determination from the first priority queue 210 in sequence to determine whether there are the operating resources required to run and process the first job.

[0068] The implementation principle and technical effect of the cluster job scheduling device of this embodiment are similar to those of the aforementioned method embodiment, which will not be described in detail here, and can be referenced to each other.

[0069] Figure 7 FIG. 1 is a schematic diagram of a structure of an electronic device for operation processing according to an embodiment of the present invention, which can implement any of the methods described in the embodiments of the present invention. Figure 7 As an optional embodiment, the electronic device may include: a housing 41, a processor 42, a memory 43, a circuit board 44 and a power supply circuit 45, wherein the circuit board 44 is arranged inside the space enclosed by the housing 41, and the processor 42 and the memory 43 are arranged on the circuit board 44; the power supply circuit 45 is used to supply power to various circuits or devices of the electronic device; the memory 43 is used to store executable program codes; the processor 42 runs a program corresponding to the executable program code by reading the executable program code stored in the memory 43, so as to run the cluster job scheduling method described in any of the aforementioned embodiments.

[0070] The specific operation process of the processor 42 for the above steps and the steps further executed by the processor 42 by running the executable program code can be found in the description of the cluster job scheduling method embodiment 1 of the present invention, which will not be repeated here.

[0071] The electronic device exists in various forms, including but not limited to: (1) Mobile communication devices: These devices are characterized by having mobile communication functions and are mainly aimed at providing voice and data communications. Such terminals include: smart phones (such as iPhone), multimedia phones, functional phones, and low-end phones. (2) Ultra-mobile personal computer devices: These devices belong to the category of personal computers, have computing and processing functions, and generally also have mobile Internet access features. Such terminals include: PDA, MID and UMPC devices, such as iPad. (3) Portable entertainment devices: These devices can display and play multimedia content. Such devices include: audio and video playback modules (such as iPod), handheld game consoles, e-books, as well as smart toys and portable car navigation devices. (4) Servers: Devices that provide computing services. The server's composition includes processors, hard disks, memory, system buses, etc. The server is similar to the general computer architecture, but because it needs to provide highly reliable services, it has higher requirements in terms of processing power, stability, reliability, security, scalability, manageability, etc. (5) Other electronic devices with data interaction functions.

[0072] An embodiment of the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the cluster job scheduling method described in any one of the above-mentioned embodiments.

[0073] In summary, according to the description of the above embodiments, the cluster job scheduling method and device disclosed in this embodiment, by adding a job tag to the job to be processed, can dynamically and orderly schedule and select the priority queue that meets the resource requirements among multiple queues according to the matching scheduling relationship between the job indicated by the job tag and multiple priority queues during the job scheduling process. This effectively solves the problem that the job can only run in a single queue in the traditional cluster scheduling method, avoids the idleness of cluster resources, and significantly improves the resource utilization efficiency, thereby maximizing the utilization of cluster resources.

[0074] In addition, the present invention gives users fine-grained control over job running resources by allowing users to add tags to jobs when submitting them. Users can specify the priority queue and resource requirements of jobs when submitting jobs based on their own understanding of job characteristics, which enhances the user's flexibility in resource use and enables them to maximize the effectiveness of cluster resources based on specific job characteristics and requirements. Moreover, users no longer need to communicate frequently with the IT team or wait for resource allocation when submitting jobs, thereby reducing communication costs and time waste and improving the response speed of job scheduling.

[0075] Finally, the present invention simplifies the complexity of cluster resource configuration, especially in the resource scheduling process, avoiding the workload of the operation and maintenance team to frequently manually adjust the resource queue. The traditional cluster scheduling method requires the IT operation and maintenance team to continuously adjust the queue configuration according to the changes and needs of the job, while the job scheduling system of the present invention can automatically determine the idle state of queue resources by adding job tags, and schedule jobs to the appropriate queue without manual intervention. In particular, in the configuration mode of using independent resource queues, when the first priority queue 210 is short of resources, the system can automatically query the resource status of other queues and perform job allocation, thereby realizing automatic scheduling of resources, significantly reducing the workload of the operation and maintenance team, and improving the efficiency of cluster management.

[0076] It should be noted that, in this article, except for the first priority and the second priority, etc., which represent the first and second qualifiers in the queue priority, the remaining relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0077] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0078] For the convenience of description, the above device is described by dividing the functions into various units / modules. Of course, when implementing the present invention, the functions of each unit / module can be implemented in the same or multiple software and / or hardware.

[0079] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is running, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can also be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0080] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A cluster job scheduling method, characterized in that: The method comprises: Acquire a job tag carried in the first job, the job tag including a running resource requirement of the first job and a matching scheduling relationship indicating the first job and at least two priority queues, the at least two priority queues including: a first priority queue and a second priority queue; Determining, according to the job tag, whether the first priority queue has running resources required to run and process the first job; If not, determining whether the second priority queue has the required operating resources to run and process the first job; If the second priority queue has the running resources required to run and process the first job, the first job is distributed to the second priority queue for processing.

2. The cluster job scheduling method according to claim 1, characterized in that: After determining whether the first priority queue has the running resources required to run and process the first job, the method further includes: if the first priority queue has the running resources required to run and process the first job, distributing the first job to the first priority queue for processing; Monitoring the job processing completion status fed back by the first priority queue; If a job is currently running and completed, query whether the first priority queue has a second job in a suspended state; If yes, the second priority queue is controlled to resume processing the suspended second job, and the status of the second job resuming processing is obtained.

3. The cluster job scheduling method according to claim 1, characterized in that: After determining that the first priority queue does not meet the running resources required to run and process the first job, the method also includes: controlling the first priority queue to suspend the low priority job currently being processed or canceling the job indicated by at least one job mark that should match the highest priority queue other than the current first priority queue, so as to release occupied resources.

4. The cluster job scheduling method according to claim 1, characterized in that: After determining that the first priority queue does not have the required operating resources to run and process the first job, the method further includes: determining whether the operating resources released by suspending the currently processed low-priority job in the first priority queue are sufficient to process the first job; If so, the first priority queue is controlled to suspend the low priority job currently being processed; If not, continue to determine whether the resources released by the first priority queue canceling at least one third job currently being processed are sufficient to process the first job; wherein the highest priority queue matched by the third job is not the current first priority queue; If so, the first job is assigned to the first priority queue for processing, and the status of the third job cancellation and the first job running processing is obtained.

5. The cluster job scheduling method according to claim 1, characterized in that: After determining whether the second priority queue has the running resources required to run and process the first job, the method further includes: if the second priority queue does not have the running resources required to run and process the first job, continuing to determine whether the remaining priority queues have the running resources required to run and process the first job; If yes, distribute the first job to the corresponding priority queue for processing; If not, the next round of judging whether there are sufficient operating resources required for processing the first operation is performed again from the first priority queue in sequence.

6. The cluster job scheduling method according to claim 1, characterized in that: The matching scheduling relationship includes: the association between the first job and at least two priority queues; and, indicating that when the first priority queue cannot meet the resource conditions required by the first job, the first job is allowed to be transferred to other priority queues for resource condition judgment; and, indicating the priority order of matching scheduling of the first job among multiple priority queues.

7. A cluster job scheduling device, characterized in that: include: An acquisition program unit is used to acquire a job tag carried in a first job, wherein the job tag includes a running resource requirement of the first job and a matching scheduling relationship indicating the job and at least two priority queues, wherein the at least two priority queues include: a first priority queue and a second priority queue; A first judgment program unit, used for judging whether the first priority queue has running resources required for running and processing the first job according to the job tag; a second judgment program unit, for judging whether the second priority queue has the running resources required to run and process the first job if no; The first dispatching program unit is used to dispatch the first job to the second priority queue for processing if the second priority queue has running resources required for running and processing the first job.

8. The cluster job scheduling device according to claim 7, characterized in that: Also includes: A second distribution unit is used for, after determining whether the first priority queue has the running resources required for running and processing the first job, distributing the first job to the first priority queue for processing if the first priority queue has the running resources required for running and processing the first job; A first monitoring program unit, configured to monitor the job processing completion status fed back by the first priority queue; A first query program unit is used to query whether there is a second job in the first priority queue in a suspended state if a job is currently running and completed; The first control program unit, if any, controls the second priority queue to resume processing of the suspended second job, and obtains the status of the second job resuming processing.

9. The cluster job scheduling device according to claim 7, characterized in that: Also includes: The second control program unit is used to control the first priority queue to suspend the low-priority job currently being processed or cancel the job that should match the highest priority queue other than the current first priority queue indicated by at least one job mark after determining that the first priority queue does not meet the running resources required for running and processing the first job, so as to release the occupied resources.

10. The cluster job scheduling device according to claim 7, characterized in that: The first judgment program unit is further configured to, after determining that the first priority queue does not have the required running resources for running and processing the first job, determine whether the running resources released by suspending the low-priority job currently being processed by the first priority queue are sufficient to meet the needs of processing the first job; a second control program unit, configured to control the first priority queue to suspend the low priority job currently being processed if yes; A third judgment program unit is used to, if not, continue to judge whether the resources released by the first priority queue canceling at least one third job currently being processed are sufficient to process the first job; wherein the highest priority queue matched by the third job is not the current first priority queue; The third dispatching program unit is used for allocating the first job to the first priority queue for processing if yes, and obtaining the status of the third job cancellation and the first job running processing.

11. The cluster job scheduling device according to claim 7, characterized in that: The first dispatching program unit is further configured to, if the second priority queue does not have the required operating resources to run and process the first job, continue to determine whether the remaining priority queues have the required operating resources to run and process the first job; If yes, distribute the first job to the corresponding priority queue for processing; If not, the next round of judging whether there are sufficient operating resources required for processing the first operation is performed again from the first priority queue in sequence.

12. An electronic device, characterized in that: include: One or more processors; Memory; The memory stores one or more executable programs, and the one or more processors read the executable program codes stored in the memory to run the programs corresponding to the executable program codes, so as to run any method described in claims 1 to 5.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the method of any one of claims 1 to 6.

Images