Method, device, equipment, medium and program product for determining backfill scheduling parameters

Abstract

This application relates to a method, apparatus, device, storage medium, and program product for determining backfill scheduling parameters. The method obtains the output logs of the job scheduling system during previous backfill scheduling processes, determines the backfill status of the backfill scheduling process based on the output logs, and adjusts the current backfill scheduling parameters according to the scheduling strategy corresponding to the backfill status. This method enables status monitoring of the job scheduling system during the backfill scheduling process, allowing real-time acquisition of the backfill status. This, in turn, enables dynamic optimization of the current backfill scheduling parameters based on the backfill status, maximizing the scheduling effect of backfill scheduling and thereby improving the resource utilization rate of the job scheduling system.

Description

Technical Field

[0001] This application relates to the field of computer equipment technology, and in particular to a method, apparatus, device, medium and program product for determining backfill scheduling parameters. Background Technology

[0002] SLURM (Simple Linux Utility for Resource Management) is a highly scalable and fault-tolerant cluster manager and job scheduling system for large compute node clusters, widely used in supercomputers and computing clusters worldwide. The SLURM scheduling system includes a backfill scheduler. The backfill scheduler considers each running job, sorts the queued jobs in descending order of priority, and determines the start time of each job. If a job under consideration can start immediately without affecting the expected start time of any higher-priority jobs, then this job will be executed directly by the backfill scheduler to efficiently schedule jobs submitted by each compute node.

[0003] In practical use, the SLURM scheduling system provides numerous parameters that users can adjust to limit and manage the scheduling system. Similarly, the backfill scheduler within the SLURM system also has many parameters that can be set for backfill scheduling time, queue depth, and scheduling intervals. Adjusting these parameters allows for effective limitation and management of the scheduling system. Currently, there are many large-scale HPC (High Performance Computing) clusters in China, with varying cluster sizes and user numbers. Users can only set a relatively reasonable value for the backfill scheduling parameters based on cluster size, job load, and other information. However, the large number of parameters makes it impossible to achieve an optimal effect, leading to poor scheduling performance and low resource utilization.

[0004] Therefore, how to optimize the parameters of the backfill scheduler so that the SLURM scheduling system can achieve the best scheduling effect during job scheduling and thus improve resource utilization has become an urgent technical problem to be solved. Summary of the Invention

[0005] Therefore, it is necessary to provide a method, apparatus, equipment, medium, and program product for determining backfill scheduling parameters that can improve scheduling effectiveness and increase resource utilization, in order to address the above-mentioned technical problems.

[0006] Firstly, this application provides a method for determining backfill scheduling parameters, the method comprising:

[0007] Obtain the output logs of the job scheduling system during the previous backfill scheduling process;

[0008] The backfill status of the current backfill scheduling process is determined based on the output logs.

[0009] Adjust the current backfilling scheduling parameters according to the scheduling strategy corresponding to the backfilling status.

[0010] This embodiment realizes the status monitoring of the job scheduling system during the backfilling scheduling process. It can obtain the backfilling status in real time, and then realize the dynamic optimization of the current backfilling scheduling parameters based on the backfilling status, so as to achieve the best scheduling effect of backfilling and improve the resource utilization of the job scheduling system.

[0011] In one embodiment, if the backfilling state is a busy state, then adjusting the current backfilling scheduling parameters according to the scheduling policy corresponding to the backfilling state includes:

[0012] The reasons for job exit in the previous backfill scheduling process are detected, and the current backfill scheduling parameters are adjusted according to the reasons for job exit.

[0013] In one embodiment, adjusting the current backfill scheduling parameters based on the job exit reason includes:

[0014] Extract the set queue depth, backfill scheduling timeout status, and job scheduling status of each job from the output log of the previous backfill scheduling process.

[0015] If the reason for the job exit is that the set queue depth is less than the current actual queue depth, and the job scheduling status of each job indicates that some jobs have not been scheduled, then the values ​​of the job-related parameters in the current backfill scheduling parameters are increased.

[0016] If the reason for the job exit is that the backfill scheduling timeout status is backfill scheduling timeout exit, and the job scheduling status of each job indicates that some jobs have not been scheduled, then the backfill scheduling time in the current backfill scheduling parameters is increased.

[0017] This embodiment optimizes the backfill scheduling parameters by analyzing the reasons for job exit, which can reduce the probability of invalid job scheduling in the job scheduling system during the backfill scheduling process, thereby improving the scheduling efficiency of the job scheduling system and achieving the best scheduling effect.

[0018] In one embodiment, if the backfilling state is an idle state, then adjusting the current backfilling scheduling parameters according to the scheduling policy corresponding to the backfilling state includes:

[0019] Reduce the backfill scheduling time and / or set the queue depth in the current backfill scheduling parameters.

[0020] The method described in this embodiment can avoid the phenomenon that the backfill scheduler occupies the lock and there are no full backfill jobs in the queue when the backfill state is idle, which would affect the operation of other threads. This optimizes the operation of the backfill scheduler and makes it achieve an optimal state for the backfill scheduler.

[0021] In one embodiment, the method further includes:

[0022] If the number of jobs in the current backfill scheduling parameters is less than a preset threshold, or if no jobs meet the backfill requirements, then the backfill scheduling time interval in the current backfill scheduling parameters is increased.

[0023] The method described in this embodiment enables dynamic optimization of backfill scheduling parameters when the number of jobs is small or the jobs do not meet the backfill requirements, so that the backfill scheduler can effectively save scheduling resources when backfill scheduling is not required.

[0024] Secondly, this application also provides a job scheduling method. The method includes:

[0025] Retrieve jobs submitted by each job node in the job scheduling system;

[0026] The backfill scheduling parameters are adjusted according to the method for determining backfill scheduling parameters described in the first aspect above, to obtain the adjusted backfill scheduling parameters.

[0027] The backfill scheduling is performed on the jobs submitted by each job node according to the adjusted backfill scheduling parameters.

[0028] The job scheduling method provided in the above embodiments obtains the current backfill status of the backfill scheduling process by detecting the log output during the backfill scheduling process. It then determines whether the backfill status is optimal. If not, the backfill status is divided into two categories: a busy state and an idle state. In the busy state, the backfill scheduler takes too long to process each job and cannot traverse jobs deeper in the job queue, resulting in some jobs not being scheduled. Therefore, adjusting the backfill scheduling parameters based on this state can improve job scheduling efficiency. The idle state occurs when there are few jobs in the job queue, each backfill scheduling operation occupies a lock, and there are no jobs in the queue that are fully backfilled, causing the lock to be occupied and affecting the operation of other threads. Therefore, adjusting the backfill scheduling parameters based on this state can adjust the operation of the backfill scheduler to achieve an optimal state. When the management node uses the optimal backfill scheduling parameters for job backfill scheduling, it can further improve the job scheduling efficiency and resource utilization of the job scheduling system.

[0029] Thirdly, this application also provides a device for determining backfill scheduling parameters. The device includes:

[0030] The acquisition module is used to acquire the output logs of the job scheduling system during the backfilling scheduling process;

[0031] The determination module is used to determine the backfill status of the backfill scheduling process based on the output log;

[0032] The adjustment module is used to adjust the backfilling scheduling parameters according to the scheduling strategy corresponding to the backfilling status.

[0033] Fourthly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the methods described in the first or second aspect above.

[0034] Fifthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the methods described in the first or second aspect above.

[0035] Sixthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the methods described in the first or second aspect above. Attached Figure Description

[0036] Figure 1 This is a schematic diagram of the structure of a job scheduling system in one embodiment;

[0037] Figure 2 This is a schematic diagram of the backfill scheduling process in one embodiment;

[0038] Figure 3 This is a flowchart illustrating a method for determining backfill scheduling parameters in one embodiment;

[0039] Figure 4 This is a flowchart illustrating how backfill scheduling parameters are adjusted in one embodiment;

[0040] Figure 5 This is a flowchart illustrating the method for determining backfill scheduling parameters in another embodiment;

[0041] Figure 6 This is a schematic diagram of the optimized process in one embodiment;

[0042] Figure 7 This is a flowchart illustrating a job scheduling method in one embodiment;

[0043] Figure 8 This is a schematic diagram of the job execution flow in one embodiment;

[0044] Figure 9 This is a structural block diagram of a device for determining backfill scheduling parameters in one embodiment;

[0045] Figure 10 This is a structural block diagram of a device for determining backfill scheduling parameters in one embodiment;

[0046] Figure 11 This is a structural block diagram of a job scheduling device in one embodiment;

[0047] Figure 12 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0049] The method for determining backfill scheduling parameters provided in this application embodiment can be applied to, for example, Figure 1 The job scheduling system shown includes multiple computing nodes 102, a management node 104, and multiple user terminals 106. The management node 104 is connected to each user terminal 106 and each computing node 102. Each user terminal 106 submits its job application to the management node 104. The management node 104 schedules corresponding resources according to the job application's requirements, performing job scheduling and sending execution instructions to each computing node 102. Each computing node 102, upon receiving the execution instructions, starts running the job application and returns the calculation results to the user terminal 106 via the management node 104 after the scheduled job is completed. The user terminals 106 can be, but are not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle devices. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. The computing nodes 102 can be implemented using independent servers or a server cluster. The management node 104 can also be implemented using independent servers or a server cluster.

[0050] In existing SLURM scheduling systems, the SLURM scheduler maintains a queue of pending jobs and manages the overall resource utilization of these jobs. It manages available computing nodes in a shared or non-shared manner for users to execute jobs. The SLURM scheduler allocates resources appropriately to the task queue and monitors jobs until their completion. The SLURM scheduler provides three key functions: First, it assigns users exclusive and / or non-exclusive access to resources over a period of time so they can execute jobs. Second, it provides a framework for starting, executing, and monitoring jobs on a set of allocated nodes. Finally, it arbitrates resource contention by managing the queue of pending jobs.

[0051] The existing SLURM scheduling system's SLURM scheduler mainly consists of a master scheduler and a backfill scheduler. The backfill scheduler considers currently running jobs and then sorts the queued jobs in descending order of priority, determining the start time for each job. If a low-priority job can start immediately without affecting the expected start time of any higher-priority jobs, then this job will be executed directly by the backfill scheduler. The effects of priority scheduling and backfill scheduling are as follows: Figure 2 As shown in the figure, the horizontal axis represents time and the vertical axis represents resource quantity. It can be seen that the backfill scheduler advances jobs a and b without affecting the expected start time of the preceding jobs.

[0052] Typically, in the actual use of SLURM scheduling systems in large computing clusters, the number of jobs submitted by users varies at different times. When there is a surge in user-submitted jobs, if the scheduling time set by the backfill scheduler is short or the scheduling queue depth is small, many jobs will be unable to be scheduled, resulting in a "none" status. Even if idle resources meet the job's requirements, they cannot be provided to the job. In this case, administrators need to manually adjust the relevant backfill scheduling parameters. If administrators fail to notice this in time, jobs may remain "none" for an extended period, leading to unscheduled jobs, significant resource waste, and a poor user experience. To address these issues, this application provides a method for determining backfill scheduling parameters. This method enables the scheduling system to dynamically optimize backfill scheduling parameters during the backfill scheduling process, thereby achieving the best scheduling effect and improving resource utilization. The following embodiments specifically illustrate this method for determining backfill scheduling parameters.

[0053] In one embodiment, such as Figure 3 As shown, a method for determining backfill scheduling parameters is provided, which is then applied to... Figure 1 Taking the management node in the example, the following steps are included:

[0054] S101, Obtain the output logs of the job scheduling system during the previous backfill scheduling process.

[0055] The output logs may include parameters related to the backfilling scheduling process or the backfilling scheduler, such as the number of scheduled jobs, scheduling time, and reason for backfilling schedule exit. The job scheduling system can be, for example... Figure 1 The job scheduling system mentioned above includes a management node responsible for backfilling and scheduling jobs submitted by each user terminal.

[0056] In this embodiment, during each round of backfill scheduling of jobs submitted by user terminals, the management node records parameters related to the backfill scheduling process, such as the number of scheduled jobs, scheduling time, reason for exiting backfill scheduling, and exit status, in the output log. Therefore, when the management node performs the current round of backfill scheduling for jobs submitted by user terminals, it can check the output logs of previous rounds of backfill scheduling through polling. It should be noted that a backfill scheduler can be installed on the management node, and this backfill scheduler can be started during backfill scheduling to perform backfill scheduling and check the output logs of the backfill scheduler.

[0057] S102, determine the backfill status of the current backfill scheduling process based on the output log.

[0058] The backfilling status includes three states: optimal, busy, and idle. The optimal state indicates that the management node can efficiently handle jobs in the job queue during backfilling scheduling, ensuring both efficiency and quality. The busy state indicates that the management node takes too long to process each job and cannot traverse deeper jobs in the queue, resulting in some jobs not being scheduled. The idle state indicates that there are few jobs in the job queue, and each backfilling scheduling operation occupies a lock without a full queue of jobs, causing lock occupancy during backfilling to affect the operation of other threads.

[0059] In this embodiment, when the management node detects the output log of the previous backfill scheduling process, it can further extract parameters related to the backfill scheduling process from the output log, such as the number of scheduled jobs, the scheduling time, and the reason for backfill scheduling exit. Combined with the various parameters set in the current backfill scheduling parameters, such as the backfill scheduling time and the backfill scheduling time interval, as well as the schedulable resources of the management node and each connected computing node, the backfill status of the current backfill scheduling process can be determined.

[0060] S103, adjust the current backfilling scheduling parameters according to the scheduling strategy corresponding to the backfilling status.

[0061] Different backfill states correspond to different scheduling strategies. The scheduling strategy is used to adjust the current backfill scheduling parameters, that is, to optimize the current backfill scheduling parameters. Backfill scheduling parameters include many parameters, among which those with a significant impact on the backfill scheduling process are shown in Table 1. Users can control the backfill scheduling process in the scheduling system by modifying the backfill scheduling parameters, that is, control the backfill scheduler. The backfill scheduling parameters are mainly divided into two categories: one part modifies the timing of various functions of the backfill scheduler, and the other part modifies the number of jobs accessed in the backfill scheduler's queue.

[0062] Table 1

[0063]

[0064]

[0065] In this embodiment, the management node can pre-determine and store the correspondence between different backfill states and different scheduling strategies. When the management node determines the backfill state of the current backfill scheduling process based on the aforementioned steps, it can further analyze the impact of each backfill scheduling parameter on the backfill scheduling process and bind the backfill scheduling parameters and backfill states. This allows the management node to adjust the current backfill scheduling parameters according to the scheduling strategy corresponding to the backfill state, thereby optimizing the current backfill scheduling parameters. Specifically, when the backfill state is busy, the management node can determine the scheduling strategy corresponding to the busy state based on the correspondence between the backfill state and the scheduling strategy, and adjust the current backfill scheduling parameters accordingly. When the backfill state is idle, the management node can determine the scheduling strategy corresponding to the idle state based on the correspondence between the backfill state and the scheduling strategy, and adjust the current backfill scheduling parameters accordingly.

[0066] The method for determining backfill scheduling parameters described in the above embodiments obtains the output logs of the job scheduling system during the previous backfill scheduling process, determines the backfill status of the backfill scheduling process based on the output logs, and adjusts the current backfill scheduling parameters according to the scheduling strategy corresponding to the backfill status. This method enables status monitoring of the job scheduling system during the backfill scheduling process, allowing real-time acquisition of the backfill status. This, in turn, enables dynamic optimization of the current backfill scheduling parameters based on the backfill status, maximizing the scheduling effect of backfill scheduling and thereby improving the resource utilization rate of the job scheduling system.

[0067] In the method described in the above embodiment, the backfill status of the current backfill scheduling process detected by the management node can be divided into optimal status, busy status and idle status. When the backfill status is busy, this application provides a method for optimizing the backfill scheduling parameters under this backfill status, namely, the above-mentioned S103 "adjusting the current backfill scheduling parameters according to the scheduling strategy corresponding to the backfill status", including: detecting the job exit reason of the previous backfill scheduling process and adjusting the current backfill scheduling parameters according to the job exit reason.

[0068] Specifically, when the management node determines that the current backfilling scheduling process is in a busy state based on the aforementioned steps, it can further extract the job exit reasons from the output logs of previous backfilling scheduling processes. By analyzing these exit reasons, the corresponding scheduling policy can be obtained, and the current backfilling scheduling parameters can be adjusted according to the policy. It should be noted that different job exit reasons can correspond to different scheduling policies, and this correspondence can be predetermined and stored by the management node.

[0069] Furthermore, specific methods for adjusting backfill scheduling parameters based on the reasons for job exit during the backfill scheduling process are provided, such as... Figure 4 As shown, this method includes:

[0070] S201, extract the set queue depth, backfill scheduling timeout status, and job scheduling status from the output log of the previous backfill scheduling process.

[0071] The backfill scheduling timeout status includes either a backfill scheduling exit due to timeout or a backfill scheduling exit without timeout. The job scheduling status includes whether the job is scheduled or not. Specifically, when the management node needs to adjust the current backfill scheduling parameters based on the job exit reasons during the backfill scheduling process, it can further extract job-related parameters such as the set queue depth, backfill scheduling timeout status, and job scheduling status of each job from the output logs of the previous backfill scheduling process. These job-related parameters can then be used as a reference to adjust the current backfill scheduling parameters.

[0072] S202, if the reason for job exit is that the set queue depth is less than the current actual queue depth, then increase the value of the job-related parameters in the current backfill scheduling parameters.

[0073] When a job exits because the set queue depth is less than the actual current queue depth, and the job scheduling status indicates that some jobs have not been scheduled, it means the set queue depth is too small. The values ​​of job-related parameters in the current backfill scheduling parameters need to be appropriately increased. For example, the queue depth, number of partitions, number of users, and start time should be adjusted to ensure that all jobs in the queue can be effectively scheduled, thus making full use of resources. The values ​​of job-related parameters also include those in Table 1, such as the maximum number of jobs that can be started per partition and the maximum number of jobs that can be started per user.

[0074] S203, if the reason for job exit is that the backfill scheduling timeout status is backfill scheduling timeout exit, and the job scheduling status of each job indicates that some jobs have not been scheduled, then increase the backfill scheduling time in the current backfill scheduling parameters.

[0075] When a job exits due to a timeout in the backfill scheduling process, and the job scheduling status indicates that some jobs have not been scheduled, it suggests that the backfill scheduling time is too short and can be appropriately increased. This embodiment optimizes the backfill scheduling parameters by analyzing the reasons for job exits, which can reduce the probability of invalid job scheduling during the backfill scheduling process, thereby improving the scheduling efficiency of the job scheduling system and achieving the best scheduling effect.

[0076] In one embodiment, when the backfilling state is idle, a method for optimizing the backfilling scheduling parameters in the backfilling state is also provided, namely, the above-mentioned S103 "adjusting the current backfilling scheduling parameters according to the scheduling strategy corresponding to the backfilling state", including: reducing the backfilling scheduling time and / or setting the queue depth in the current backfilling scheduling parameters.

[0077] Specifically, when the management node determines that the current backfilling status of the backfilling scheduling process is idle based on the aforementioned steps, it indicates that the number of jobs in the job queue is low. The backfilling scheduling time in the current backfilling scheduling parameters can be appropriately reduced to match the number of jobs in the job queue, thus avoiding waste of backfilling scheduling time. Optionally, the set queue depth in the current backfilling scheduling parameters can be appropriately reduced to match the depth of the job queue, thus avoiding waste of queue resources. Optionally, both the backfilling scheduling time and the set queue depth in the current backfilling scheduling parameters can be reduced simultaneously, thereby avoiding waste of both backfilling scheduling time and queue resources. The method described in this embodiment can avoid the phenomenon where backfilling lock occupancy affects the operation of other threads when the backfilling status is idle and there are no fully backfilled jobs in the queue, thereby optimizing the operation of the backfilling scheduler and achieving an optimal state for the backfilling scheduler.

[0078] Furthermore, the above-mentioned S103 "adjusting the current backfill scheduling parameters according to the scheduling strategy corresponding to the backfill status" also includes: if the number of jobs in the current backfill scheduling parameters is less than the preset number threshold, or if no job meets the backfill requirements, then the time interval of backfill scheduling in the current backfill scheduling parameters is increased.

[0079] Specifically, when the management node determines that the current backfilling status of the backfilling scheduling process is idle based on the aforementioned steps, it can further determine whether the number of jobs in the current backfilling scheduling parameters is less than a preset threshold. If the number of jobs in the current backfilling scheduling parameters is less than the preset threshold, it indicates that the number of jobs waiting to be scheduled in the job queue is small. In this case, the number of backfilling scheduling operations can be reduced, and the time interval of backfilling scheduling in the current backfilling scheduling parameters can be appropriately increased. Optionally, when the management node determines that the current backfilling status of the backfilling scheduling process is idle based on the aforementioned steps, it can further determine whether the jobs in the queue meet the backfilling requirements. If no jobs meet the backfilling requirements, it indicates that backfilling scheduling is not required, or the number of backfilling scheduling operations can be reduced. In this case, the time interval of backfilling scheduling in the current backfilling scheduling parameters can be appropriately increased. The method described in this embodiment realizes dynamic optimization of backfilling scheduling parameters when the number of jobs is small or the jobs do not meet the backfilling requirements, enabling the backfilling scheduler to effectively save scheduling resources when backfilling scheduling is not required.

[0080] In summary, based on all the above embodiments, a method for determining backfill scheduling parameters is also provided, such as... Figure 5 As shown, the method includes:

[0081] S301, retrieve the output logs of the job scheduling system during the previous backfill scheduling process.

[0082] S302, determine the backfill status of the current backfill scheduling process based on the output log.

[0083] S303 extracts the set queue depth, backfill scheduling timeout status, and job scheduling status from the output log, and determines the reason for job exit during the backfill scheduling process.

[0084] S304, if the backfilling status is busy, determine the reason for the job exit in the backfilling scheduling process. If the reason for the job exit is that the set queue depth is less than the current actual queue depth, increase the value of the job-related parameters in the current backfilling scheduling parameters. If the reason for the job exit is that the backfilling scheduling timeout status is backfilling scheduling timeout exit, and the job scheduling status of each job indicates that some jobs have not been scheduled, increase the backfilling scheduling time in the current backfilling scheduling parameters.

[0085] S305, if the backfilling status is idle, then reduce the backfilling scheduling time and / or set the queue depth in the current backfilling scheduling parameters, and increase the backfilling scheduling time interval in the current backfilling scheduling parameters if the number of jobs in the current backfilling scheduling parameters is less than the preset number threshold, or if no jobs meet the backfilling requirements.

[0086] The above Figures 2-5 The method described in this embodiment is a method for adjusting backfill scheduling parameters in the management node, which optimizes the backfill scheduling parameters. See also... Figure 6 The diagram illustrates the optimized process. During the entire scheduling process, the backfill scheduler executes a cyclical step: real-time monitoring of logs, determining the backfill status based on log output, and adjusting backfill scheduling parameters according to the backfill status. During this process, the backfill scheduler can extract relevant parameters from the log output, such as backfill lock release, backfill time interval, and backfill scheduling parameters, and determine the backfill status based on these parameters. This is achieved through... Figure 6 The illustrated iterative optimization process achieves dynamic optimization of the backfill scheduling parameters. Specifically, it will... Figures 2-6 The method described in the embodiments is applied in, for example, Figure 1 The job scheduling system shown can improve the efficiency of job scheduling and increase resource utilization during the scheduling process. The following... Figure 7 The embodiments illustrate an application of Figure 1 The job scheduling method for the application environment shown.

[0087] In one embodiment, such as Figure 7 As shown, a job scheduling method is provided, which can be applied to... Figure 1 Taking the management node in the example, the following steps are included:

[0088] S401: Obtain jobs submitted by each user terminal in the job scheduling system.

[0089] Today, applications in fields vital to national welfare and people's livelihoods, such as weather forecasting, satellite image processing, biomedicine, and industrial simulation, all rely heavily on high-performance computing. However, every high-performance cluster depends on a scheduling system. A job scheduling system (such as a SLURM scheduling system) rationally allocates and arranges resources. Specifically, management nodes can employ methods such as... Figure 8The job execution process allocates or manages jobs. In the job scheduling system, after a user terminal submits a job, the master scheduler or backfill scheduler on the management node of the job scheduling system will apply for or allocate corresponding resources according to the job's needs to ensure the normal execution of the job. The management node will send the execution instructions to the computing nodes that have applied for them. At the same time, each computing node has a corresponding daemon process. After receiving the execution instructions from the management node, the computing node starts running the program. During the execution, the computing node collects the job's running status and resource usage on the node. Until the job ends, the computing node notifies the management node, and the scheduling system reclaims the computing node's resources.

[0090] S402, according to Figures 2-7 The method for determining backfill scheduling parameters described in the embodiment adjusts the backfill scheduling parameters to obtain the adjusted backfill scheduling parameters.

[0091] according to Figures 2-7 The method for determining backfill scheduling parameters described in the embodiment adjusts the backfill scheduling parameters, thereby optimizing the backfill scheduling parameters so that the optimized backfill scheduling parameters can match the current scheduling process.

[0092] S403 performs backfill scheduling on the jobs submitted by each user terminal according to the adjusted backfill scheduling parameters.

[0093] By using the adjusted backfill scheduling parameters to backfill and schedule the jobs submitted by each user terminal, the job scheduling system can achieve the best scheduling effect during the backfill scheduling process.

[0094] The job scheduling method provided in the above embodiments obtains the current backfill status of the backfill scheduling process by detecting the log output during the backfill scheduling process. It then determines whether the backfill status is optimal. If not, the backfill status is divided into two categories: a busy state and an idle state. In the busy state, the backfill scheduler takes too long to process each job and cannot traverse jobs deeper in the job queue, resulting in some jobs not being scheduled. Therefore, adjusting the backfill scheduling parameters based on this state can improve job scheduling efficiency. The idle state occurs when there are few jobs in the job queue, each backfill scheduling operation occupies a lock, and there are no jobs in the queue that are fully backfilled, causing the lock to be occupied and affecting the operation of other threads. Therefore, adjusting the backfill scheduling parameters based on this state can adjust the operation of the backfill scheduler to achieve an optimal state. When the management node uses the optimal backfill scheduling parameters for job backfill scheduling, it can further improve the job scheduling efficiency and resource utilization of the job scheduling system.

[0095] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0096] Based on the same inventive concept, this application also provides a device for determining backfill scheduling parameters to implement the method for determining backfill scheduling parameters described above. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more embodiments of the device for determining backfill scheduling parameters provided below can be found in the limitations of the method for determining backfill scheduling parameters described above, and will not be repeated here.

[0097] In one embodiment, such as Figure 9 As shown, a device for determining backfill scheduling parameters is provided, comprising:

[0098] Module 10 is used to acquire the output logs of the job scheduling system during the backfill scheduling process;

[0099] The determination module 11 is used to determine the backfill status of the backfill scheduling process based on the output log;

[0100] The adjustment module 12 is used to adjust the backfilling scheduling parameters according to the scheduling strategy corresponding to the backfilling status.

[0101] In one embodiment, if the backfilling status is busy, the adjustment module 12 is specifically used to detect the job exit reason of the previous backfilling scheduling process and adjust the current backfilling scheduling parameters according to the job exit reason.

[0102] In one embodiment, the aforementioned adjustment module 12, such as Figure 10 As shown, it includes:

[0103] Extraction unit 120 is used to extract the set queue depth, the timeout status of backfill scheduling, and the job scheduling status of each job from the output log of the previous backfill scheduling process.

[0104] The first adjustment unit 121 is used to increase the value of the job-related parameters in the current backfill scheduling parameters when the reason for the job exit is that the set queue depth is less than the current actual queue depth, and the job scheduling status of each job indicates that some jobs have not been scheduled.

[0105] The second adjustment unit 122 is used to increase the backfill scheduling time in the current backfill scheduling parameters when the reason for the job exit is that the backfill scheduling timeout status is backfill scheduling timeout exit, and the job scheduling status of each job indicates that some jobs have not been scheduled.

[0106] In one embodiment, if the backfilling state is an idle state, the adjustment module 12 is specifically used to reduce the backfilling scheduling time and / or set the queue depth in the current backfilling scheduling parameters.

[0107] In one embodiment, if the number of jobs in the current backfill scheduling parameters is less than a preset threshold, or if no jobs meet the backfill requirements, the adjustment module 12 is specifically used to increase the backfill scheduling time interval in the current backfill scheduling parameters.

[0108] Based on the same inventive concept, this application also provides a job scheduling apparatus for implementing the job scheduling method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more job scheduling apparatus embodiments provided below can be found in the limitations of the job scheduling method described above, and will not be repeated here.

[0109] In one embodiment, such as Figure 11 As shown, a job scheduling device is provided, comprising:

[0110] Module 20 is used to acquire jobs submitted by each user terminal in the job scheduling system;

[0111] Adjustment module 21 is used to determine the backfill status of the backfill scheduling process based on the output log;

[0112] The scheduling module 22 is used to adjust the backfilling scheduling parameters according to the scheduling strategy corresponding to the backfilling status.

[0113] The aforementioned backfilling scheduling parameter determination device or the various modules in the aforementioned job scheduling device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0114] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 12 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores output log data. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements a method for determining backfill scheduling parameters.

[0115] Those skilled in the art will understand that Figure 12 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0116] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0117] Obtain the output logs of the job scheduling system during the previous backfill scheduling process;

[0118] The backfill status of the current backfill scheduling process is determined based on the output logs.

[0119] Adjust the current backfilling scheduling parameters according to the scheduling strategy corresponding to the backfilling status.

[0120] The computer device provided in the above embodiments has a similar implementation principle and technical effect to the above method embodiments, and will not be described again here.

[0121] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0122] Obtain the output logs of the job scheduling system during the previous backfill scheduling process;

[0123] The backfill status of the current backfill scheduling process is determined based on the output logs.

[0124] Adjust the current backfilling scheduling parameters according to the scheduling strategy corresponding to the backfilling status.

[0125] The computer-readable storage medium provided in the above embodiments has a similar implementation principle and technical effect to the above method embodiments, and will not be described again here.

[0126] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0127] Obtain the output logs of the job scheduling system during the previous backfill scheduling process;

[0128] The backfill status of the current backfill scheduling process is determined based on the output logs.

[0129] Adjust the current backfilling scheduling parameters according to the scheduling strategy corresponding to the backfilling status.

[0130] The computer program product provided in the above embodiments has a similar implementation principle and technical effect to the above method embodiments, and will not be described again here.

[0131] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0132] Retrieve jobs submitted by each user terminal in the job scheduling system;

[0133] The backfilling scheduling parameters are adjusted according to the above method for determining backfilling scheduling parameters to obtain the adjusted backfilling scheduling parameters.

[0134] The backfill scheduling is performed on the jobs submitted by each user terminal according to the adjusted backfill scheduling parameters.

[0135] The computer device provided in the above embodiments has a similar implementation principle and technical effect to the above method embodiments, and will not be described again here.

[0136] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0137] Retrieve jobs submitted by each user terminal in the job scheduling system;

[0138] The backfilling scheduling parameters are adjusted according to the above method for determining backfilling scheduling parameters to obtain the adjusted backfilling scheduling parameters.

[0139] The backfill scheduling is performed on the jobs submitted by each user terminal according to the adjusted backfill scheduling parameters.

[0140] The computer-readable storage medium provided in the above embodiments has a similar implementation principle and technical effect to the above method embodiments, and will not be described again here.

[0141] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0142] Retrieve jobs submitted by each user terminal in the job scheduling system;

[0143] The backfilling scheduling parameters are adjusted according to the above method for determining backfilling scheduling parameters to obtain the adjusted backfilling scheduling parameters.

[0144] The backfill scheduling is performed on the jobs submitted by each user terminal according to the adjusted backfill scheduling parameters.

[0145] The computer program product provided in the above embodiments has a similar implementation principle and technical effect to the above method embodiments, and will not be described again here.

[0146] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0147] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0148] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for determining backfill scheduling parameters, characterized in that, The method includes: Obtain the output logs of the job scheduling system in the previous backfill scheduling process, and extract the parameters related to the backfill scheduling process, the current backfill scheduling parameters, and the schedulable resources of the management node and each connected computing node from the output logs to determine the backfill status of the current backfill scheduling process. When the backfilling state is in the optimal state, the backfilling scheduling parameters corresponding to the optimal state are used to schedule the jobs submitted by each user terminal for backfilling. When the backfilling status is busy, the set queue depth, backfilling scheduling timeout status, and job scheduling status of each job are extracted from the output log, and the reason for job exit during the backfilling scheduling process is determined. If the reason for job exit is that the set queue depth is less than the current actual queue depth, and the job scheduling status of each job indicates that some jobs have not been scheduled, then at least one of the following parameters in the current backfilling scheduling parameters is increased: set queue depth, number of partitions, number of users, start time, maximum number of jobs started per partition, and maximum number of jobs started per user. If the reason for job exit is that the backfilling scheduling timeout status is backfilling scheduling timeout exit, and the job scheduling status of each job indicates that some jobs have not been scheduled, then the backfilling scheduling time in the current backfilling scheduling parameters is increased. Backfilling scheduling is performed on the jobs submitted by each user terminal according to the adjusted backfilling scheduling parameters. When the backfilling status is idle, the backfilling scheduling time and / or the set queue depth in the current backfilling scheduling parameters are reduced. When the number of jobs in the current backfilling scheduling parameters is less than a preset threshold or no jobs meet the backfilling requirements, the backfilling scheduling time interval in the current backfilling scheduling parameters is increased. Backfilling scheduling is performed on the jobs submitted by each user terminal according to the adjusted backfilling scheduling parameters.

2. A job scheduling method, characterized in that, The method includes: Retrieve jobs submitted by each user terminal in the job scheduling system; The backfill scheduling parameters are adjusted according to the method for determining backfill scheduling parameters as described in claim 1 to obtain the adjusted backfill scheduling parameters; The backfill scheduling is performed on the jobs submitted by each user terminal according to the adjusted backfill scheduling parameters.

3. A device for determining backfill scheduling parameters, used to implement the method for determining backfill scheduling parameters as described in claim 1, characterized in that, The device includes: The acquisition module is used to acquire the output logs of the job scheduling system during the backfilling scheduling process; The determination module is used to determine the backfill status of the backfill scheduling process based on the output log; The adjustment module is used to adjust the current backfilling scheduling parameters according to the scheduling strategy corresponding to the backfilling status.

4. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method of claim 1 or 2.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method of claim 1 or 2.

6. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method of claim 1 or 2.

Images