Job scheduling method and device, computer device, and storage medium

Abstract

The application relates to a job scheduling method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: determining job information of a to-be-scheduled job and current resource gap information of a job queue; in the case that the job information of the to-be-scheduled job and the current resource gap information do not match, taking resource idle information as a constraint item of job resource demand information, taking resource idle duration information as a constraint item of job duration information, adjusting the job demand information and the job duration information according to a preset job adjustment strategy, and obtaining backfill job information of the to-be-scheduled job. In the case that the backfill job information satisfies backfill conditions, the to-be-scheduled job is scheduled to a resource gap corresponding to the current resource gap information based on the backfill job information. The method can improve the utilization rate of a high-performance computing cluster.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of high-performance computing, and in particular to a job scheduling method and device, computer equipment, a storage medium and a computer program product. BACKGROUND

[0002] In the field of high-performance computing, each job submitted by a user is run in order of priority from high to low, that is, job scheduling is performed in a first-in first-out manner. In order to improve efficiency, a backfill scheduling algorithm is proposed. Under the premise of job scheduling in a first-in first-out manner, the start time of each job is determined. If a newly received job can start immediately without affecting the expected start time of any higher priority job, the job will be executed directly based on the backfill scheduling algorithm to efficiently schedule jobs submitted by users.

[0003] However, when the system performs backfill scheduling, it schedules jobs according to the amount of resources and the duration of time applied for by the user when submitting the job, while considering the current availability of system resources. However, users cannot accurately provide the amount of resources and the duration of time required by the job, and often apply for more resources and time than the job itself requires, resulting in jobs that could have been backfilled remaining in a queue, reducing the utilization of the high-performance computing cluster. SUMMARY

[0004] Therefore, it is necessary to provide a job scheduling method, device, computer equipment, computer readable storage medium and computer program product that can improve the utilization of a high-performance computing cluster to solve the above technical problems.

[0005] In a first aspect, the present application provides a job scheduling method. The method comprises:

[0006] determining job information of a to-be-scheduled job and current resource gap information of a job queue; the job information comprises job resource requirement information and job duration information of the to-be-scheduled job; the current resource gap information comprises resource idle information and resource idle duration information of the job queue;

[0007] in a case where the job information of the to-be-scheduled job and the current resource gap information do not match, using the resource idle information as a constraint term of the job resource requirement information, using the resource idle duration information as a constraint term of the job duration information, adjusting the job requirement information and the job duration information according to a preset job adjustment strategy, to obtain backfill job information of the to-be-scheduled job;

[0008] In a case where the backfill operation information meets a backfill condition, the to-be-scheduled operation is scheduled to a resource gap corresponding to the current resource gap information based on the backfill operation information.

[0009] By the operation scheduling method, the low-priority operation is backfilled as much as possible by jumping the operation information of the low-priority operation without affecting the running of the high-priority operation, so that the low-priority operation can be run as soon as possible, and the utilization rate of the high-performance computing cluster is improved.

[0010] In one of the embodiments, in a case where the backfill operation information meets a backfill condition, the to-be-scheduled operation is scheduled to a resource gap corresponding to the current resource gap information based on the backfill operation resource information and the backfill operation duration information, including:

[0011] In a case where it is determined that the backfill operation information matches the current resource gap information and confirmation information of the user for the backfill operation information is received, the to-be-scheduled operation is scheduled to a resource gap corresponding to the current resource gap information based on the backfill operation resource information and the backfill operation duration information.

[0012] In this embodiment, the opinion of the user is consulted before the to-be-scheduled operation is adjusted and backfilled, so as to avoid problems unexpected by the user in the running process.

[0013] In one of the embodiments, the resource idle information is taken as a constraint term of the operation resource requirement information, the resource idle duration information is taken as a constraint term of the operation duration information, the operation requirement information and the operation duration information are adjusted according to a preset operation adjustment strategy, and backfill operation information of the to-be-scheduled operation is obtained, including:

[0014] In a case where it is determined that the operation resource requirement information is higher than the resource idle information and the operation duration information is lower than the resource idle duration information, the operation resource requirement information is adjusted downward according to a preset downward adjustment requirement to obtain adjusted downward resource requirement information.

[0015] In a case where it is determined that the adjusted downward resource requirement information is not higher than the resource idle information, task information of the to-be-scheduled operation and the adjusted downward resource requirement information are input into a preset operation duration estimation model to obtain operation estimated duration information of the operation under the adjusted downward resource requirement information.

[0016] In a case where the operation estimated duration information is lower than the resource idle duration information, the adjusted downward resource requirement information is taken as backfill operation resource information of the to-be-scheduled operation, and the operation estimated duration information is taken as backfill operation duration information of the operation.

[0017] In this embodiment, for the to-be-scheduled job whose job resource requirement information does not meet the resource idle information of the current resource gap, the job resource requirement information is adjusted downward so that the to-be-scheduled job can be backfilled into the current resource gap.

[0018] In one of the embodiments, the resource idle information is taken as a constraint term of the job resource requirement information, the resource idle duration information is taken as a constraint term of the job duration information, the job requirement information and the job duration information are adjusted according to a preset job adjustment strategy, and backfill job information of the to-be-scheduled job is obtained, including:

[0019] In a case where it is determined that the job resource requirement information is lower than the resource idle information and the job duration information is higher than the resource idle duration information, the job resource requirement information is adjusted upward according to a preset upward adjustment requirement to obtain upward adjusted resource requirement information.

[0020] In a case where it is determined that the upward adjusted resource requirement information is not higher than the resource idle information, task information of the job and the upward adjusted resource requirement information are input into a preset job duration estimation model to obtain job estimated duration information of the job under the condition of the upward adjusted resource requirement information.

[0021] In a case where the job estimated duration information is lower than the resource idle duration information, the upward adjusted resource requirement information is taken as backfill job resource information of the to-be-scheduled job, and the job estimated duration information is taken as backfill job duration information of the job.

[0022] In this embodiment, for the to-be-scheduled job whose job duration information does not meet the resource idle duration information of the current resource gap, the job resource requirement information is adjusted upward so that the to-be-scheduled job can be backfilled into the current resource gap.

[0023] In one of the embodiments, the method further includes:

[0024] In a case where a job submitted by a user is received, user job information set by the user is determined; the user job information includes user job resource requirement information and user job duration information.

[0025] The user job resource requirement information included in the user job information and task information of the job are input into a preset job duration estimation model to obtain job estimated duration information of the job under the condition of the user job resource requirement information.

[0026] The job is taken as a to-be-scheduled job, the user job resource requirement information is taken as job requirement resource information, and the job estimated duration information is taken as job duration information.

[0027] In this embodiment, the job duration information of the job is determined by using the job duration estimation model, so as to reduce resource waste caused by too long job duration set by the user subjectively or job running failure caused by too short resource duration set.

[0028] In one of the embodiments, the method further comprises:

[0029] When the to-be-scheduled job reaches the duration corresponding to the backfill scheduling job duration information, if the to-be-scheduled job is not running completely, it is judged whether the user job duration information and the next job estimated starting time meet the delay condition;

[0030] In the case that the user job duration information and the next job estimated starting time meet the delay condition, the to-be-scheduled job is continued to run according to the preset delay strategy.

[0031] In the case that the user job duration information and the next job estimated starting time do not meet the delay condition, the to-be-scheduled job is ended.

[0032] In this embodiment, for the job that is not running completely, the running duration of the job is prolonged as much as possible without affecting the running of the next job, so as to reduce the proportion of the job that is killed and reduce the job that is killed at the time not expected by the user.

[0033] In one of the embodiments, in the case that the user job duration information, the backfill job duration information and the next job estimated starting time meet the delay condition, the to-be-scheduled job is continued to run according to the preset delay strategy, which comprises:

[0034] In the case that the user job duration information, the backfill job duration information and the next job estimated starting time meet the delay condition, the user expected running end time of the to-be-scheduled job is determined according to the starting time of the to-be-scheduled job and the user job duration information.

[0035] In the case that the duration from the current time to the user expected running end time meets the minimum duration threshold and the duration from the current time to the next job estimated starting time meets the minimum duration threshold, the to-be-scheduled job is continued to run according to the minimum duration threshold.

[0036] In this embodiment, the to-be-scheduled job is delayed according to the minimum duration threshold, and whether it meets the user expectation and whether it will affect the running of the next job are considered.

[0037] In a second aspect, the present application provides a job scheduling device. The device comprises:

[0038] a determining module configured to determine job information of a to-be-scheduled job and current resource gap information of a job queue; the job information comprises job resource requirement information and job duration information of the to-be-scheduled job; the current resource gap information comprises resource idle information and resource idle duration information of the job queue;

[0039] an adjusting module configured to, in a case where the job information of the to-be-scheduled job does not match the current resource gap information, take the resource idle information as a constraint term of the job resource requirement information, take the resource idle duration information as a constraint term of the job duration information, and adjust the job requirement information and the job duration information according to a preset job adjustment strategy, to obtain backfill job information of the to-be-scheduled job;

[0040] a scheduling module configured to, in a case where the backfill job information satisfies a backfill condition, schedule the to-be-scheduled job to a resource gap corresponding to the current resource gap information based on the backfill job information.

[0041] In one of the embodiments, the scheduling module is specifically configured to:

[0042] in a case where it is determined that the backfill job information matches the current resource gap information and confirmation information of the backfill job information is received from a user, schedule the to-be-scheduled job to the resource gap corresponding to the current resource gap information based on the backfill job resource information and the backfill job duration information.

[0043] In one of the embodiments, the adjusting module specifically comprises:

[0044] a first input unit configured to, in a case where it is determined that the job resource requirement information is higher than the resource idle information and the job duration information is lower than the resource idle duration information, input task information of the to-be-scheduled job and the job resource requirement information into a preset job duration estimation model to obtain job estimated duration information of the to-be-scheduled job under the job resource requirement information;

[0045] a first input unit configured to, in a case where it is determined that the job resource requirement information is higher than the resource idle information and the job duration information is lower than the resource idle duration information, input task information of the to-be-scheduled job and the job resource requirement information into a preset job duration estimation model to obtain job estimated duration information of the to-be-scheduled job under the job resource requirement information;

[0046] The first scheduling unit is configured to, in a case where the job estimated duration information is lower than the resource idle duration information, take the down-regulated resource requirement information as backfill job resource information of the job to be scheduled, and take the job estimated duration information as backfill job duration information of the job.

[0047] In one of the embodiments, the adjusting module specifically includes:

[0048] The up-regulating unit is configured to, in a case where the job resource requirement information is lower than the resource idle information and the job duration information is higher than the resource idle duration information, up-regulate the job resource requirement information according to a preset up-regulation requirement to obtain up-regulated resource requirement information.

[0049] The second input unit is configured to, in a case where the up-regulated resource requirement information is not higher than the resource idle information, input task information of the job and the up-regulated resource requirement information into a preset job duration estimation model to obtain job estimated duration information of the job in the case of the up-regulated resource requirement information.

[0050] The second scheduling unit is configured to, in a case where the job estimated duration information is lower than the resource idle duration information, take the up-regulated resource requirement information as backfill job resource information of the job to be scheduled, and take the job estimated duration information as backfill job duration information of the job.

[0051] In one of the embodiments, the device further includes:

[0052] The receiving module is configured to, in a case where a job submitted by a user is received, determine user job information set by the user; the user job information includes user job resource requirement information and user job duration information.

[0053] The input module is configured to input the user job resource requirement information included in the user job information and task information of the job into a preset job duration estimation model to obtain job estimated duration information of the job in the case of the user job resource requirement information.

[0054] The job to be scheduled determining module is configured to take the job as a job to be scheduled, take the user job resource requirement information as job requirement resource information, and take the job estimated duration information as job duration information.

[0055] In one of the embodiments, the device further includes:

[0056] determining, by a judging module, whether the user job duration information and the next job estimated starting time meet a delay condition, when the to-be-scheduled job is running and reaches a duration corresponding to the backfill scheduling job duration information, and the to-be-scheduled job is not running to completion;

[0057] a first running module, configured to continue running the to-be-scheduled job according to a preset delay strategy, when the user job duration information and the next job estimated starting time meet the delay condition;

[0058] a second running module, configured to end running the to-be-scheduled job, when the user job duration information and the next job estimated starting time do not meet the delay condition.

[0059] In one embodiment, the first running module specifically includes:

[0060] a running end time and space determination unit, configured to determine a user expected running end time of the to-be-scheduled job according to a starting time of the to-be-scheduled job and the user job duration information, when the user job duration information, the backfill job duration information, and the next job estimated starting time information meet the delay condition;

[0061] a running unit, configured to continue running the to-be-scheduled job according to a minimum duration threshold, when a duration from a current time to the user expected running end time meets the minimum duration threshold, and a duration from the current time to the next job estimated starting time meets the minimum duration threshold.

[0062] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0063] determining job information of a to-be-scheduled job and current resource gap information of a job queue; the job information includes job resource demand information and job duration information of the to-be-scheduled job; the current resource gap information includes resource idle information and resource idle duration information of the job queue;

[0064] when the job information of the to-be-scheduled job and the current resource gap information do not match, using the resource idle information as a constraint term of the job resource demand information, using the resource idle duration information as a constraint term of the job duration information, adjusting the job demand information and the job duration information according to a preset job adjustment strategy, to obtain backfill job information of the to-be-scheduled job;

[0065] In a case where the backfill operation information meets a backfill condition, the to-be-scheduled operation is scheduled to a resource gap corresponding to the current resource gap information based on the backfill operation information.

[0066] In a fourth aspect, the present application further provides a computer readable storage medium. The computer readable storage medium has a computer program stored thereon, and the computer program, when executed by a processor, implements the following steps:

[0067] determining operation information of a to-be-scheduled operation and current resource gap information of an operation queue; the operation information includes operation resource requirement information and operation duration information of the to-be-scheduled operation; the current resource gap information includes resource idle information and resource idle duration information of the operation queue;

[0068] in a case where the operation information of the to-be-scheduled operation and the current resource gap information do not match, adjusting the operation requirement information and the operation duration information according to a preset operation adjustment strategy, to obtain backfill operation information of the to-be-scheduled operation, by taking the resource idle information as a constraint term of the operation resource requirement information and taking the resource idle duration information as a constraint term of the operation duration information;

[0069] in a case where the backfill operation information meets a backfill condition, scheduling the to-be-scheduled operation to a resource gap corresponding to the current resource gap information based on the backfill operation information.

[0070] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and the computer program, when executed by a processor, implements the following steps:

[0071] determining operation information of a to-be-scheduled operation and current resource gap information of an operation queue; the operation information includes operation resource requirement information and operation duration information of the to-be-scheduled operation; the current resource gap information includes resource idle information and resource idle duration information of the operation queue;

[0072] in a case where the operation information of the to-be-scheduled operation and the current resource gap information do not match, adjusting the operation requirement information and the operation duration information according to a preset operation adjustment strategy, to obtain backfill operation information of the to-be-scheduled operation, by taking the resource idle information as a constraint term of the operation resource requirement information and taking the resource idle duration information as a constraint term of the operation duration information;

[0073] in a case where the backfill operation information meets a backfill condition, scheduling the to-be-scheduled operation to a resource gap corresponding to the current resource gap information based on the backfill operation information.

[0074] The job scheduling method, device, computer device, storage medium and computer program product described above, under the premise of not affecting the running of high-priority jobs, by jumping the job information of low-priority jobs, the low-priority jobs are backfilled as much as possible, on the one hand, the low-priority jobs can be run as quickly as possible, on the other hand, the utilization rate of the high-performance computing cluster is improved. BRIEF DESCRIPTION OF DRAWINGS

[0075] Figure 1 An application environment diagram of the job scheduling method in one embodiment;

[0076] Figure 2 A flowchart of the job scheduling method in one embodiment;

[0077] Figure 3 A structure diagram of a job queue in one embodiment;

[0078] Figure 4 A schematic diagram of a job adjustment process in one embodiment;

[0079] Figure 5 A schematic diagram of another job adjustment process in one embodiment;

[0080] Figure 6 A flowchart of the job scheduling method in another embodiment;

[0081] Figure 7 A structure block diagram of the job scheduling device in one embodiment;

[0082] Figure 8 An internal structure diagram of the computer device in one embodiment. DETAILED DESCRIPTION

[0083] In order to make the purpose, technical scheme and advantages of the present application clearer, further detailed description of the present application will be given below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0084] In the related art, when the system performs backfill scheduling, the system schedules the jobs according to the resource amount and time length applied by the user when submitting the jobs, and considers the current available system resources. However, the user cannot accurately give the resource amount and time length required by the jobs, and often applies more resource amount and time length than required by the jobs, which causes the jobs that can be backfilled to be in a queuing state all the time, and reduces the utilization rate of the high-performance computing cluster.

[0085] Based on this, the application provides a job scheduling method, determines job information of a to-be-scheduled job and current resource gap information of a job queue, in a case where the job information of the to-be-scheduled job and the current resource gap information do not match, takes resource idle information as a constraint term of job resource demand information and takes resource idle duration information as a constraint term of job duration information, adjusts the job demand information and the job duration information according to a preset job adjustment strategy, and obtains backfill job information of the to-be-scheduled job. In a case where the backfill job information satisfies a backfill condition, the to-be-scheduled job is scheduled to a resource gap corresponding to the current resource gap information based on the backfill job information.

[0086] Through the job scheduling method, the job information of a low-priority job is jumped, the low-priority job is backfilled as much as possible, on the premise that the running of a high-priority job is not affected, on the one hand, the low-priority job can be run as soon as possible, and on the other hand, the utilization rate of a high-performance computing cluster is improved.

[0087] The job scheduling method provided in the embodiments of the application can be applied to a job scheduling system as shown in Figure 1 The job scheduling system includes a plurality of terminals 102, a job scheduler 104, and a high-performance computing cluster composed of a plurality of computing nodes 106. Each terminal 102 communicates with the job scheduler 104 through a network, and the job scheduler 104 communicates with each computing node through a network. Each user terminal 102 submits job information and related settings to the job scheduler 104, the job scheduler 104 schedules the jobs submitted by each terminal 102, and distributes the jobs to corresponding computing nodes 106, and the computing nodes 106 run the jobs distributed by the job scheduler 104. The terminal 102 can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, Internet of Things devices, and portable wearable devices. The Internet of Things device can be a smart speaker, a smart television, a smart air conditioner, a smart vehicle device, etc. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, etc. The computing node 104 can be implemented by an independent server or a server cluster composed of a plurality of servers.

[0088] In one embodiment, as shown in Figure 2 A job scheduling method is provided, and the job scheduling method is applied to a job scheduler in Figure 1 for example, and includes the following steps:

[0089] Step 201, determining job information of a to-be-scheduled job and current resource gap information of a job queue.

[0090] Job information includes the resource requirements and duration of the scheduled job. For example, the resource requirements might include the CPU, GPU, or other computing resources required for the job, while the duration refers to the duration of the job under that resource requirement. Current resource gap information includes the idle resources and idle duration of the job queue, including the number of idle compute nodes within the resource gap and the idle duration of each idle compute node within the resource gap, based on the job priority.

[0091] like Figure 3 As shown in the figure, the number of resources in a high-performance computing cluster is fixed. The vertical axis represents the number of resources, and the horizontal axis represents the duration. Each rectangle represents a job, with the vertical length indicating the number of resources required for the job and the horizontal length indicating the duration of the job. Jobs are arranged in order of priority, forming a job queue based on the resources occupied at each time. The current resource gap refers to the number of resources that will be idle during the execution of the next job, as well as the idle time. For example, if the high-performance computing cluster has 10 compute nodes and the next job requires 8 compute nodes to run for one day, there will be a resource gap between two compute nodes running for one day.

[0092] Specifically, the job scheduler receives jobs submitted by multiple users, one of which is not yet running. After determining the job to be scheduled, it obtains the job resource requirement information and job duration information for the job to be scheduled, as well as other necessary information such as the job name and job directory. The job scheduler traverses the current job queue and, based on the job information of the next job to be executed and the resource information of the high-performance computing cluster, determines the resource availability information and resource idle duration information of the current resource gap information.

[0093] Step 203: When the job information of the job to be scheduled does not match the current resource gap information, the resource idle information is used as the constraint item of the job resource demand information, and the resource idle duration information is used as the constraint item of the job duration information. According to the preset job adjustment strategy, the job demand information and the job duration information are adjusted to obtain the backfill job information of the job to be scheduled.

[0094] Specifically, after the job scheduler determines the job information of the to-be-scheduled job and the current resource gap information, the job demand information of the job information is compared with the resource idle information of the current resource gap information, and the job duration information of the job information is compared with the resource idle duration information of the current resource gap information. If the job demand information of the job information is greater than the resource idle information of the current resource gap information, or the job duration information of the job information is greater than the resource idle duration information of the current resource gap information, it indicates that the job information of the to-be-scheduled job does not match the current resource gap information, and the to-be-scheduled job cannot be scheduled to the current resource gap according to the job information. At this time, the job scheduler takes the resource idle information as a constraint item of the job resource demand information, takes the resource idle duration information as a constraint item of the job duration information, adjusts the job demand information and the job duration information according to a preset job adjustment strategy, so that the job demand information is not greater than the resource idle information and the job duration information is not greater than the job idle duration information, and obtains the backfill job information of the to-be-scheduled job.

[0095] The backfill scheduling job information includes backfill job resource information and backfill job duration information. The backfill job resource information is the job resource information obtained after adjustment according to the preset job adjustment strategy, and the backfill job duration information is the job duration information obtained after adjustment according to the preset job adjustment strategy.

[0096] Step 205: In the case that the backfill job information meets the backfill condition, the to-be-scheduled job is scheduled to the resource gap corresponding to the current resource gap information based on the backfill job information.

[0097] Specifically, after the job scheduler obtains the backfill job information of the to-be-scheduled job in step 103, it is judged whether the backfill information of the to-be-scheduled job meets the backfill condition, for example, whether the user has confirmed the backfill job information, or whether the backfill job information matches the current resource gap information, etc. In the case that the backfill information of the to-be-scheduled job meets the backfill condition, it indicates that the to-be-scheduled job can be backfilled, and the to-be-scheduled job is scheduled to the resource gap corresponding to the current resource gap information based on the backfill job information. In the case that the backfill information of the to-be-scheduled job does not meet the backfill condition, it indicates that the to-be-scheduled job cannot be backfilled, and the to-be-scheduled job cannot be backfilled at present, and continues to wait to be run or wait for the next resource gap in the job queue.

[0098] Through the job scheduling method provided in the present application, on the premise of not affecting the running of high-priority jobs, the job information of low-priority jobs is jumped to backfill the low-priority jobs as much as possible, which on the one hand makes the low-priority jobs run as quickly as possible, and on the other hand improves the utilization rate of the high-performance computing cluster.

[0099] In one of the embodiments, the step 205 specifically includes:

[0100] In a case where it is determined that the backfill job information matches the current resource gap information and the confirmation information of the user for the backfill job information is received, the to-be-scheduled job is scheduled to the resource gap corresponding to the current resource gap information based on the backfill job resource information and the backfill job duration information.

[0101] Specifically, the job scheduler adjusts the job information of the to-be-scheduled job to obtain the backfill job information, matches the backfill job information with the current resource information, compares the backfill job requirement information with the resource idle information, compares the backfill job duration information with the resource idle duration information, and if the backfill job information matches the current resource information, it indicates that the to-be-scheduled job information can be backfilled to the current resource gap according to the backfill job information, and further requests the user to confirm whether the backfill job information is confirmed, and if the confirmation information of the user for the backfill job information is received, it indicates that the user agrees to backfill according to the backfill job information, and the to-be-scheduled job is scheduled to the resource gap corresponding to the current resource gap information based on the backfill job resource information and the backfill job duration information. If the job scheduler determines that the backfill job information does not match the current resource information, it indicates that the to-be-scheduled job information cannot be backfilled to the current resource gap according to the backfill job information, and the to-be-scheduled job continues to wait to be run in the job queue or waits for the next resource gap. If the job scheduler receives the non-confirmation information of the user for the backfill job information, i.e., the user does not agree to backfill according to the backfill job information, the to-be-scheduled job continues to wait to be run in the job queue or waits for the next resource gap.

[0102] In this embodiment, the user's opinion is solicited before the to-be-scheduled job is adjusted and backfilled, so as to avoid problems that are unexpected by the user during running.

[0103] In one of the embodiments, the step 203 specifically includes:

[0104] Step A1, in a case where it is determined that the job resource requirement information is higher than the resource idle information and the job duration information is lower than the resource idle duration information, the job resource requirement information is adjusted downward according to a preset downward adjustment requirement to obtain adjusted resource requirement information.

[0105] Specifically, when matching the job information of the job to be scheduled and the current resource gap information, the job scheduler compares the job demand information of the job information with the resource idle information of the current resource gap information, and compares the job duration information of the job information with the resource idle duration information of the current resource gap information. If it is determined that the job resource demand information is higher than the resource idle information, and the job duration information is lower than the resource idle duration information, it indicates that the resource quantity of the current resource gap does not meet the resource demand of the job to be scheduled, but the idle duration of the current resource gap meets the running duration of the job to be scheduled. The job resource demand information is adjusted downward according to the preset downward adjustment requirement to obtain adjusted resource demand information. For example, the job resource demand information is adjusted downward by 20%. The resource demand information is originally 5 computing nodes, and the adjusted resource demand information obtained after adjustment is 4 computing nodes.

[0106] Step A2, in the case where it is determined that the adjusted resource demand information is not higher than the resource idle information, inputting the task information of the job to be scheduled and the adjusted resource demand information into a preset job duration estimation model to obtain job estimated duration information of the job to be scheduled under the adjusted resource demand information.

[0107] Specifically, after the job scheduler adjusts the job resource demand information according to the preset downward adjustment requirement to obtain the adjusted resource demand information, it is determined whether the adjusted resource demand information is not higher than the resource idle information. If the adjusted resource demand information is higher than the resource idle information, it indicates that the current resource gap cannot meet the running of the job to be scheduled under the adjusted resource demand information, and the adjustment amplitude of the resource demand information of the job to be scheduled is too large. For example, the adjustment ratio is greater than 30%, which may cause the job to be scheduled to run incorrectly. At this time, the job to be scheduled cannot be backfilled, and the job to be scheduled continues to wait to be run or waits for the next resource gap in the job queue. If the job scheduler determines that the adjusted resource demand information is not higher than the resource idle information, it indicates that the resource gap information meets the adjusted resource demand information. Inputting the task information of the job to be scheduled and the adjusted resource demand information into a preset job duration estimation model to obtain job estimated duration information of the job to be scheduled under the adjusted resource demand information.

[0108] The job duration estimation model can refer to the description in the related art, which will not be described in detail here.

[0109] Step A3, in the case where the job estimated duration information is lower than the resource idle duration information, the adjusted resource demand information is used as the backfill job resource information of the job, and the job estimated duration information is used as the backfill job duration information of the job.

[0110] Specifically, the job scheduler inputs the task information and the downward adjustment resource demand information of the job to be scheduled into the preset job duration estimation model. After obtaining the job estimated duration information of the job to be scheduled under the condition of the downward adjustment resource demand information, it determines whether the job estimated duration information is lower than the resource idle duration information. If the job estimated duration information is not lower than the resource idle duration information, it means that the current resource gap cannot enable the job to be scheduled to run under the downward adjustment resource demand information. The job to be scheduled continues to wait in the job queue to be run or wait for the next resource gap. If the job scheduler determines that the job estimated duration information is lower than the resource idle duration information, it means that the current resource gap can enable the job to be scheduled to run under the downward adjustment resource demand information. The downward adjustment resource demand information is used as the backfill job resource information of the job, and the job estimated duration information is used as the backfill job duration information of the job, waiting to be backfilled.

[0111] like Figure 4 As shown, the gray box is the job to be scheduled, the dotted box is the current resource gap, the left figure is a schematic diagram of the job information before the job to be scheduled is adjusted, the job resource demand information is higher than the resource idle information, and the job duration information is lower than the resource idle duration information. After adjusting the job information to be scheduled, the job information schematic diagram before and after the adjustment of the job to be scheduled is obtained as shown in the middle figure, and then the adjusted job to be scheduled is backfilled into the current resource gap, and the job queue schematic diagram on the right is obtained.

[0112] In this embodiment, for a to-be-scheduled job whose resource idle information in the current resource gap does not satisfy the job resource requirement information, the job resource requirement information is adjusted downward so that the to-be-scheduled job can be backfilled into the current resource gap.

[0113] In one embodiment, the step 205 specifically includes:

[0114] Step B1: When it is determined that the job resource demand information is lower than the resource idle information and the job duration information is higher than the resource idle duration information, the job resource demand information is increased according to a preset increase requirement to obtain increased resource demand information.

[0115] Specifically, when matching the job information of the job to be scheduled and the current resource gap information, the job scheduling device compares the job demand information of the job information with the resource idle information of the current resource gap information, and compares the job duration information of the job information with the resource idle duration information of the current resource gap information. If it is determined that the job resource demand information is lower than the resource idle information, and the job duration information is higher than the resource idle duration information, it indicates that the resource quantity of the current resource gap satisfies the resource demand of the job to be scheduled, but the idle duration of the current resource gap does not satisfy the running duration of the job to be scheduled. The job resource demand information is adjusted according to the preset adjustment requirement to obtain adjusted resource demand information. For example, the job resource demand information is adjusted by 20%, and the original resource demand information is 5 computing nodes, and the adjusted resource demand information is 6 computing nodes.

[0116] Step B2, in the case that it is determined that the adjusted resource demand information is not higher than the resource idle information, the task information of the job and the adjusted resource demand information are input into a preset job duration estimation model to obtain job estimated duration information of the job to be scheduled under the adjusted resource demand information.

[0117] Specifically, after the job scheduling device adjusts the job resource demand information according to the preset adjustment requirement to obtain the adjusted resource demand information, it is determined whether the adjusted resource demand information is not higher than the resource idle information. If the adjusted resource demand information is higher than the resource idle information, it indicates that the current resource gap cannot satisfy the running of the job to be scheduled under the adjusted resource demand information, and the adjustment range of the resource demand information of the job to be scheduled is too large, for example, the adjustment ratio is greater than 30%, which will cause the running error of the job to be scheduled. At this time, the scheduled job cannot be backfilled, and the job to be scheduled continues to wait to be run or waits for the next resource gap in the job queue. If the job scheduling device determines that the adjusted resource demand information is not higher than the resource idle information, it indicates that the resource gap information satisfies the adjusted resource demand information. The task information of the job to be scheduled and the adjusted resource demand information are input into a preset job duration estimation model to obtain job estimated duration information of the job to be scheduled under the adjusted resource demand information.

[0118] The job duration estimation model can refer to the description in the related art, and will not be described in detail here.

[0119] Step B3, in the case that the job estimated duration information is lower than the resource idle duration information, the adjusted resource demand information is used as the backfill job resource information of the job, and the job estimated duration information is used as the backfill job duration information of the job to be scheduled.

[0120] Specifically, the job scheduler inputs the task information of the job to be scheduled and the adjusted resource requirement information into a preset job duration estimation model, obtains job estimated duration information of the job to be scheduled under the adjusted resource requirement information, and determines whether the job estimated duration information is lower than the resource idle duration information. If the job estimated duration information is not lower than the resource idle duration information, it indicates that the current resource gap cannot enable the job to be scheduled to run under the adjusted resource requirement information, and the job to be scheduled continues to wait to be run in the job queue or waits for the next resource gap. If the job scheduler determines that the job estimated duration information is lower than the resource idle duration information, it indicates that the current resource gap can enable the job to be scheduled to run under the adjusted resource requirement information, and the adjusted resource requirement information is used as the backfill job resource information of the job, the job estimated duration information is used as the backfill job duration information of the job, and the job is waited to be backfilled.

[0121] As shown in Figure 5 the gray box is a job to be scheduled, the dashed box is a current resource gap, the left figure is a job information diagram before adjustment of the job to be scheduled, the job resource requirement information is lower than the resource idle information, and the job duration information is higher than the resource idle duration information. After adjustment of the job information of the job to be scheduled, the job information diagram before and after adjustment of the job to be scheduled is obtained as shown in the middle figure, and then the adjusted job to be scheduled is backfilled to the current resource gap to obtain the job queue diagram as shown in the right figure.

[0122] In this embodiment, for the job to be scheduled whose resource idle duration information of the current resource gap does not meet the job duration information, the job resource requirement information is adjusted to enable the job to be scheduled to be backfilled to the current resource gap.

[0123] In one of the embodiments, the above method further includes:

[0124] Step 207, in the case where the job submitted by the user is received, user job information set by the user is determined.

[0125] The user job information includes user job resource requirement information and user job duration information.

[0126] Specifically, when the user submits the job, the user generally sets the user job information, including user resource requirement information expected by the user and user job duration information expected by the user. In the case where the job submitted by the user is received, the job scheduler determines the user job information set by the user, including the user job resource requirement information and the user job duration information.

[0127] Step 209, the user job resource requirement information included in the user job information and the task information of the job are input into a preset job duration estimation model to obtain job estimated duration information of the job under the user job resource requirement information.

[0128] Specifically, after receiving the job submitted by the user and determining the user job information set by the user, the job scheduler inputs the user job resource requirement information included in the user job information and the task information of the job into the preset job duration estimation model to obtain the job estimated duration information of the job under the condition of the user job resource requirement information.

[0129] Step 211: regarding the job as a to-be-scheduled job, regarding the user job resource requirement information as job requirement resource information, and regarding the job estimated duration information as job duration information.

[0130] Specifically, after determining the job estimated duration information of the job under the condition of the user job resource requirement information, the job scheduler regards the job as a to-be-scheduled job, regards the user job resource requirement information as job requirement resource information, and regards the job estimated duration information as job duration information, so as to wait for a suitable gap to be backfilled or to be executed.

[0131] In one embodiment, if the user job duration information is lower than the job estimated duration information, the job scheduler can prompt the user with prompt information that the current set job duration information is too short, so as to make the user confirm whether to modify the user job duration information.

[0132] In this embodiment, the job duration information of the job is determined by using the job duration estimation model, so as to reduce the resource waste caused by the too long job duration set by the user subjectively or the job running failure caused by the too short resource duration set by the user.

[0133] In one of the embodiments, the above method further includes:

[0134] Step 213: when the running to-be-scheduled job reaches the duration corresponding to the backfilled scheduling job duration information, if the to-be-scheduled job is not running completed, it is determined whether the user job duration information and the next job estimated starting time meet the delay condition.

[0135] Specifically, after the job scheduler sends the to-be-scheduled job to each computing node, it can monitor the running status of each job in real time. When it is determined that the running to-be-scheduled job reaches the duration corresponding to the backfilled scheduling job duration information, but the to-be-scheduled job is not running completed, it is determined whether the user job duration information and the next job estimated starting time meet the delay condition, i.e., whether the running duration of the to-be-scheduled job has reached the user job duration set by the user and whether the continuous running of the to-be-scheduled job will hinder the running of the next job.

[0136] Step 215: in the case that the user job duration information and the next job estimated starting time meet the delay condition, the to-be-scheduled job is continuously run according to the preset delay strategy.

[0137] Specifically, if the job scheduler determines that the running time of the to-be-scheduled job has not reached the user job time set by the user and the continuous running of the to-be-scheduled job will not hinder the running of the next job, it is indicated that the to-be-scheduled job can be delayed, and the to-be-scheduled job is continued to run according to a preset delay strategy, for example, the to-be-scheduled job is continued to run for five minutes.

[0138] Step 217, in the case where the user job time information and the next job estimated starting time do not meet the delay condition, ending the running of the to-be-scheduled job.

[0139] Specifically, if the job scheduler determines that the running time of the to-be-scheduled job has reached the user job time set by the user or the continuous running of the to-be-scheduled job will hinder the running of the next job, ending the running of the to-be-scheduled job, that is, killing the to-be-scheduled job.

[0140] In one embodiment, if the to-be-scheduled job is ended, the job scheduler re-adds the to-be-scheduled job to the job queue to wait to be scheduled and run again.

[0141] In this embodiment, for a job that has not been run to completion, the running time of the job is as long as possible under the premise of not affecting the running of the next job, on the one hand, reducing the proportion of jobs that are killed before being completed, and on the other hand, reducing the killing of jobs at a time that is not expected by the user.

[0142] In one of the embodiments, the step 215 specifically includes:

[0143] Step 215A, in the case where the user job time information, the backfill job time information, and the next job estimated starting time information meet the delay condition, determining a user expected running end time of the to-be-scheduled job according to a starting running time of the to-be-scheduled job and the user job time information.

[0144] Specifically, if the job scheduler determines that the running time of the to-be-scheduled job has not reached the user job time set by the user and the continuous running of the to-be-scheduled job will not hinder the running of the next job, it is indicated that the to-be-scheduled job can be delayed, and the user expected running end time of the to-be-scheduled job is determined according to the starting running time of the to-be-scheduled job and the user job time information, for example, the starting running time of the to-be-scheduled job is 11:51 on February 21, and the user job time information is 24 hours, so the user expected running end time of the to-be-scheduled job is 11:51 on February 22.

[0145] Step 215B, in the case where the time length from the current time to the user expected running end time meets the minimum time threshold and the time length from the current time to the next job estimated starting time meets the minimum time threshold, continuing to run the to-be-scheduled job according to the minimum time threshold.

[0146] The minimum duration threshold is set empirically and can be five minutes, ten minutes, etc. or set according to a certain proportion of the user job duration information, for example, the minimum duration threshold is one thousandth of the user job duration.

[0147] Specifically, after the job scheduler determines the user expected end time of the to-be-scheduled job, it determines whether the duration from the current time to the user expected end time meets the minimum duration threshold. If the duration from the current time to the user expected end time is negative or the duration from the current time to the user expected end time is less than the minimum duration threshold, it means that the current end meets the user's expectation, and the to-be-scheduled job is ended. If the duration from the current time to the user expected end time is negative or the duration from the current time to the user expected end time is greater than the minimum duration threshold, it means that the current end does not meet the user's expectation, and further determines whether the duration from the current time to the next job estimated start time meets the minimum duration threshold, i.e., whether continuing to run the to-be-scheduled job will affect the running of the next job. If the duration from the current time to the next job estimated start time does not meet the minimum duration threshold, the to-be-scheduled job is ended. If the job scheduler determines that the duration from the current time to the next job estimated start time meets the minimum duration threshold, the to-be-scheduled job is continued according to the minimum duration threshold.

[0148] In this embodiment, the to-be-scheduled job is delayed according to the minimum duration threshold, and whether it meets the user's expectation and whether it will affect the running of the next job are considered.

[0149] Next, as shown in Figure 6 , a specific embodiment provided by the present application is described in detail.

[0150] Scheduling process:

[0151] Step 601, after the user submits the job to the job scheduler, the job scheduler queries whether the user has submitted and run the same or similar job in the recent period, such as querying whether the user has submitted and run the same or similar job in the past week or month. If the job record of the user submitting and running the same or similar job in the recent period is queried, step 603 is executed; if the job record of the user submitting and running the same or similar job in the recent period is not queried, step 602 is executed.

[0152] Step 602, input the job information submitted by the user into the job duration estimation model to obtain the job estimated duration, take the job submitted by the user as the to-be-scheduled job, and take the job estimated duration as the job duration information.

[0153] Step 603, take the actual running duration of the same or similar job as the job duration information of this job.

[0154] Step 604, determine whether the high-performance computing cluster is currently idle. If it is determined that the high-performance computing cluster is currently idle, step 605 is performed; if it is determined that the high-performance computing cluster is not currently idle, the job is treated as a job to be scheduled, and step 604 is performed.

[0155] Step 605, run the job to be scheduled.

[0156] Step 606, determine whether the current resource gap information matches the job information of the job to be scheduled. If they match, step 607 is performed. If they do not match, step 608 is performed.

[0157] Step 607, backfill the job to be scheduled to the current resource gap based on the job information.

[0158] Step 608, adjust the job information of the job to be scheduled according to a preset strategy to obtain backfill scheduling job information.

[0159] Step 609, send the backfill job information to the user for confirmation. If confirmation information from the user is received, step 611 is performed; if rejection information from the user is received, step 610 is performed.

[0160] Step 610, the job to be scheduled continues to wait to be run or scheduled.

[0161] Step 611, backfill the job to be scheduled to the current resource gap based on the backfill job information.

[0162] Running process:

[0163] Step 612, determine whether the running duration of the job to be scheduled that has not been run to completion has reached the duration corresponding to the backfill job duration information. If it has reached the duration corresponding to the backfill job duration information, step 613 is performed; if it has not reached the duration corresponding to the backfill job duration information, step 612 is continued.

[0164] Step 613, determine whether the job to be scheduled meets the delay condition. If it meets the delay condition, step 615 is performed; if it does not meet the delay condition, step 614 is performed.

[0165] Step 614, end the running of the job to be scheduled.

[0166] Step 615, appropriately delay according to the delay strategy and continue to run the job to be scheduled.

[0167] It should be understood that although each step in the flowchart involved in the embodiments described above is shown in sequence according to the arrow, these steps are not necessarily executed in the order indicated by the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other orders. Moreover, at least part of the steps in the flowchart involved in the embodiments described above can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be alternately executed with at least part of other steps or steps or stages in other steps.

[0168] Based on the same inventive concept, the embodiments of the present application also provide a job scheduling device for implementing the above-mentioned job scheduling method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more job scheduling device embodiments provided below can refer to the limitations of the job scheduling method described above, which will not be repeated here.

[0169] In one embodiment, as shown in Figure 7 A job scheduling device is provided, comprising:

[0170] A determination module 701 is configured to determine job information of a to-be-scheduled job and current resource gap information of a job queue; the job information comprises job resource requirement information and job duration information of the to-be-scheduled job; and the current resource gap information comprises resource idle information and resource idle duration information of the job queue.

[0171] An adjustment module 703 is configured to, in a case where the job information of the to-be-scheduled job and the current resource gap information do not match, take the resource idle information as a constraint term of the job resource requirement information, take the resource idle duration information as a constraint term of the job duration information, and adjust the job requirement information and the job duration information according to a preset job adjustment strategy, to obtain backfill job information of the to-be-scheduled job.

[0172] A scheduling module 705 is configured to, in a case where the backfill job information satisfies a backfill condition, schedule the to-be-scheduled job to a resource gap corresponding to the current resource gap information based on the backfill job information.

[0173] In one embodiment, the scheduling module 705 is specifically configured to:

[0174] In a case where it is determined that the backfill job information matches the current resource gap information and confirmation information of the user for the backfill job information is received, the to-be-scheduled job is scheduled to a resource gap corresponding to the current resource gap information based on the backfill job resource information and the backfill job duration information.

[0175] In one of the embodiments, the adjusting module 703 specifically includes:

[0176] The up-adjusting unit is configured to, in a case where it is determined that the job resource requirement information is higher than the resource idle information and the job duration information is lower than the resource idle duration information, down-adjust the job resource requirement information according to a preset down-adjusting requirement to obtain down-adjusted resource requirement information.

[0177] The first input unit is configured to, in a case where it is determined that the down-adjusted resource requirement information is not higher than the resource idle information, input task information of the to-be-scheduled job and the down-adjusted resource requirement information into a preset job duration estimation model to obtain job estimated duration information of the job in the case of the down-adjusted resource requirement information.

[0178] The first scheduling unit is configured to, in a case where the job estimated duration information is lower than the resource idle duration information, take the down-adjusted resource requirement information as backfill job resource information of the to-be-scheduled job and take the job estimated duration information as backfill job duration information of the job.

[0179] In one of the embodiments, the adjusting module 705 specifically includes:

[0180] The up-adjusting unit is configured to, in a case where it is determined that the job resource requirement information is lower than the resource idle information and the job duration information is higher than the resource idle duration information, up-adjust the job resource requirement information according to a preset up-adjusting requirement to obtain up-adjusted resource requirement information.

[0181] The second input unit is configured to, in a case where it is determined that the up-adjusted resource requirement information is not higher than the resource idle information, input task information of the job and the up-adjusted resource requirement information into a preset job duration estimation model to obtain job estimated duration information of the job in the case of the up-adjusted resource requirement information.

[0182] The second scheduling unit is configured to, in a case where the job estimated duration information is lower than the resource idle duration information, take the up-adjusted resource requirement information as backfill job resource information of the to-be-scheduled job and take the job estimated duration information as backfill job duration information of the job.

[0183] In one of the embodiments, the apparatus further includes:

[0184] receive a user job information set by the user, wherein the user job information comprises user job resource requirement information and user job duration information;

[0185] input the user job resource requirement information comprised in the user job information and task information of the job into a preset job duration estimation model to obtain job estimation duration information of the job under the user job resource requirement information;

[0186] determine the job as a to-be-scheduled job, determine the user job resource requirement information as job requirement resource information, and determine the job estimation duration information as job duration information.

[0187] In one of the embodiments, the apparatus further comprises:

[0188] determine whether the user job duration information and the next job estimation start time meet a delay condition when the running of the to-be-scheduled job reaches the duration corresponding to the backfill job duration information;

[0189] continue running the to-be-scheduled job according to a preset delay strategy when the user job duration information and the next job estimation start time meet the delay condition;

[0190] end the running of the to-be-scheduled job when the user job duration information and the next job estimation start time do not meet the delay condition.

[0191] In one of the embodiments, the first running module specifically comprises:

[0192] determine a user expected running end time of the to-be-scheduled job according to the user job duration information and a start running time of the to-be-scheduled job when the user job duration information, the backfill job duration information, and the next job estimation start time information meet the delay condition;

[0193] continue running the to-be-scheduled job according to a minimum duration threshold when a duration from a current time to the user expected running end time meets the minimum duration threshold and a duration from the current time to the next job estimation start time meets the minimum duration threshold.

[0194] The modules in the job scheduling apparatus can be implemented by software, hardware, or a combination thereof. The modules can be embedded in or independent of a processor in a computer device in hardware form, or stored in a memory in the computer device in software form, so as to be invoked and executed by the processor to perform operations corresponding to the modules.

[0195] In an embodiment, a computer device, which can be a terminal, has an internal structure as shown in Figure 8 The computer device includes a processor, a memory, and a communication interface connected by a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is configured to perform wired or wireless communication with an external terminal. The wireless communication can be achieved by WIFI, mobile cellular network, NFC (Near Field Communication), or other technologies. The computer program is executed by the processor to implement a job scheduling method.

[0196] Those skilled in the art can understand that Figure 8 The structure shown in the above embodiment is only a block diagram of part of the structure related to the scheme of the present application, and does not limit the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0197] In an embodiment, a computer device is also provided, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0198] In an embodiment, a computer readable storage medium is provided, which stores a computer program. The computer program is executed by a processor to implement the steps in the above method embodiments.

[0199] In an embodiment, a computer program product is provided, which includes a computer program. The computer program is executed by a processor to implement the steps in the above method embodiments.

[0200] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties.

[0201] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric memory (Ferroelectric Random Access Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0202] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist contradictory, it should be considered as the scope of the present application.

[0203] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent of the present application. It should be noted that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of protection of the present application. Therefore, the scope of protection of the present application should be subject to the appended claims.

Claims

1. A job scheduling method, characterized in that: The method comprises: Determine job information of a job to be scheduled and current resource gap information of a job queue; the job information includes job resource requirement information and job duration information of the job to be scheduled; the current resource gap information includes resource idle information and resource idle duration information of the job queue; In a case where the job information of the job to be scheduled does not match the current resource gap information, adjusting the resource requirement information of the job to be scheduled based on matching the current resource gap information, and adjusting the job duration information based on the adjusted resource requirement information, and determining backfill job information of the job to be scheduled based on the adjusted resource requirement information and the adjusted job duration information; the backfill job information includes backfill job resource information and backfill job duration information; In a case where the backfill job information satisfies a backfill condition, the to-be-scheduled job is scheduled to the resource gap corresponding to the current resource gap information based on the backfill job information.

2. The method according to claim 1, characterized in that The step of scheduling the to-be-scheduled job to the resource gap corresponding to the current resource gap information based on the backfill job information when the backfill job information satisfies the backfill condition includes: When it is determined that the backfill job information matches the current resource gap information and confirmation information regarding the backfill job information is received from the user, the to-be-scheduled job is scheduled to the resource gap corresponding to the current resource gap information based on the backfill job resource information and the backfill job duration information.

3. The method according to claim 1, characterized in that The step of adjusting the resource requirement information of the job to be scheduled based on matching the current resource gap information, adjusting the job duration information based on the adjusted resource requirement information, and determining the backfill job information of the job to be scheduled based on the adjusted resource requirement information and the adjusted job duration information includes: When it is determined that the job resource demand information is higher than the resource idle information and the job duration information is lower than the resource idle duration information, the job resource demand information is lowered according to a preset lowering requirement to obtain lowered resource demand information; When it is determined that the lowered resource demand information is not higher than the resource idle information, the task information of the to-be-scheduled job and the lowered resource demand information are input into a preset job duration estimation model to obtain the job estimated duration information of the job under the condition of the lowered resource demand information; When the job estimated duration information is lower than the resource idle duration information, the downward resource demand information is used as the backfill job resource information of the to-be-scheduled job, and the job estimated duration information is used as the backfill job duration information of the job.

4. The method according to claim 1, wherein The step of adjusting the resource requirement information of the job to be scheduled based on matching the current resource gap information, adjusting the job duration information based on the adjusted resource requirement information, and determining the backfill job information of the job to be scheduled based on the adjusted resource requirement information and the adjusted job duration information includes: When it is determined that the job resource demand information is lower than the resource idle information and the job duration information is higher than the resource idle duration information, the job resource demand information is increased according to a preset increase requirement to obtain increased resource demand information; When it is determined that the increased resource demand information is not higher than the resource availability information, the task information of the job and the increased resource demand information are input into a preset job duration estimation model to obtain the job estimated duration information of the job under the condition of the increased resource demand information; When the job estimated duration information is lower than the resource idle duration information, the upward resource demand information is used as the backfill job resource information of the job to be scheduled, and the job estimated duration information is used as the backfill job duration information of the job.

5. The method according to claim 1, wherein The method further comprises: Upon receiving a job submitted by a user, determining user job information set by the user; the user job information includes user job resource requirement information and user job duration information; Inputting the user job resource requirement information and the task information of the job included in the user job information into a preset job duration estimation model to obtain the job estimated duration information of the job under the user job resource requirement information; The job is used as a job to be scheduled, the user job resource requirement information is used as job resource requirement information, and the job estimated duration information is used as job duration information.

6. The method according to claim 5, characterized in that The method further comprises: When the scheduled job reaches the duration corresponding to the backfill job duration information, if the scheduled job has not been completed, determine whether the user job duration information and the next job estimated start time meet the delay condition; If the user job duration information and the estimated start time of the next job meet the delay conditions, continue to run the scheduled job according to the preset delay strategy; When the user job duration information and the estimated start time of the next job do not meet the delay condition, the running of the to-be-scheduled job is terminated.

7. The method according to claim 6, characterized in that When the user job duration information and the estimated start time of the next job meet the delay condition, continuing to run the to-be-scheduled job according to the preset delay strategy includes: If the user job duration information, the backfill job duration information, and the next job estimated start time information meet the delay condition, determine the user expected end time of the job to be scheduled based on the start time of the job to be scheduled and the user job duration information; If the duration from the current moment to the user's expected end time of the operation meets the minimum duration threshold and the duration from the current moment to the estimated start time of the next job meets the minimum duration threshold, the scheduled job continues to run according to the minimum duration threshold.

8. A job scheduling device, characterized in that: The device comprises: A determination module, configured to determine job information of a job to be scheduled and current resource gap information of a job queue; the job information includes job resource requirement information and job duration information of the job to be scheduled; the current resource gap information includes resource idle information and resource idle duration information of the job queue; an adjustment module, configured to adjust the resource requirement information of the job to be scheduled based on matching the current resource gap information, adjust the job duration information based on the adjusted resource requirement information, and determine backfill job information of the job to be scheduled based on the adjusted resource requirement information and the adjusted job duration information; the backfill job information includes backfill job resource information and backfill job duration information; The scheduling module is configured to schedule the to-be-scheduled job to the resource gap corresponding to the current resource gap information based on the backfill job information if the backfill job information satisfies a backfill condition.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

11. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images