Job scheduling method, scheduling device and computer storage medium

Abstract

The embodiment of the invention discloses a job scheduling method, a scheduling device and a computer storage medium, and belongs to the technical field of distributed computing. In the embodiment of the invention, a scheduling device can aggregate IO information of a plurality of jobs operated by a plurality of computing nodes in a computing system into group IO information by taking a group to which the jobs belong as granularity, and for any group, a scheduling strategy of the jobs of the group is determined according to the group IO information of the group. On one hand, the situation that other jobs cannot run normally due to the fact that some jobs occupy a large number of IO resources can be avoided, and therefore the performance of a computing system is improved; and on the other hand, IO access overload of the storage system in the computing system can be prevented, so that the stability of the computing system is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of distributed computing technology, and in particular to a job scheduling method, a scheduling device, and a computer storage medium. Background Art

[0002] In a computing system that provides high-performance computing (HPC) services, the computing system includes a scheduling device and multiple computing nodes. The scheduling device is used to schedule jobs on the computing nodes. For example, after a user submits a job, the scheduling device can assign the job to an appropriate computing node according to a specific scheduling policy, so that the job runs on the assigned computing node. For another example, during the execution of a job, the scheduling device can also control the computing node to suspend the job. Summary of the Invention

[0003] The embodiments of the present application provide a job scheduling method, a scheduling device, and a computer storage medium that can improve the performance of a computing system. The technical solution is as follows:

[0004] In a first aspect, a job scheduling method is provided. The method is applied to a scheduling device in a computing system, the computing system further comprising multiple computing nodes, each of which runs multiple jobs. In the method, the scheduling device obtains input / output (IO) information of multiple jobs from the multiple computing nodes; aggregates the IO information of the multiple jobs at the group level according to the groups to which the multiple jobs belong to obtain group IO information; and determines a scheduling policy for the jobs of a first group in the group based on the group IO information of the first group.

[0005] In an embodiment of the present application, the scheduling device aggregates the IO information of multiple jobs running on multiple computing nodes in the computing system into group IO information, using the group to which the jobs belong as the granularity. For any group, the scheduling policy for the group's jobs is determined based on the group IO information. This can prevent certain jobs from occupying a large amount of IO resources, preventing other jobs from running normally. It can also prevent IO access overload on the storage system in the computing system, thereby improving the stability of the computing system.

[0006] Furthermore, in an embodiment of the present application, in order to refine the granularity of job scheduling, the scheduling device may aggregate the IO information of multiple jobs according to the groups to which the jobs belong, so as to facilitate subsequent scheduling of the jobs of any group based on the group IO information of the group, that is, to realize group-level job scheduling.

[0007] Based on the method provided in the first aspect, in one possible implementation, the implementation process of determining the scheduling strategy for the jobs of the first group based on the group IO information of the first group in the group can be: comparing the group IO information of the first group with the first set of thresholds, and determining the scheduling strategy for the jobs to be scheduled in the first group based on the comparison results.

[0008] In the embodiment of the present application, the scheduling policy of the to-be-scheduled jobs in the first group can be adjusted by using the pre-configured first set of thresholds and the statistically obtained group IO information of the first group to avoid IO overload of the computing system.

[0009] Based on the method provided in the first aspect, in one possible implementation, the first set of thresholds includes a first threshold and a second threshold, and the first threshold is greater than the second threshold. In this scenario, the implementation process of comparing the group IO information of the first group with the first set of thresholds and determining the scheduling policy for the jobs to be scheduled in the first group based on the comparison results can be as follows: comparing the group IO information of the first group with the first threshold and the second threshold; if the group IO information of the first group is greater than or equal to the first threshold, stopping the scheduling of the jobs to be scheduled in the first group; if the group IO information of the first group is less than the first threshold and greater than or equal to the second threshold, maintaining the scheduling policy for the jobs to be scheduled in the first group; if the group IO information of the first group is less than the first threshold and greater than or equal to the second threshold, resuming the scheduling of the jobs to be scheduled in the first group.

[0010] By stopping scheduling pending jobs, more jobs can be prevented from occupying IO resources in the computing system.

[0011] Based on the method provided in the first aspect, in one possible implementation, the implementation process of determining the scheduling strategy of the job of the first group based on the group IO information of the first group in the group can be: comparing the group IO information of the first group with the second set of thresholds, and determining the scheduling strategy of the job being executed in the first group based on the comparison result.

[0012] In the embodiment of the present application, the scheduling policy of the jobs being executed in the first group can be adjusted by using the pre-configured second set of thresholds and the statistically obtained group IO information of the first group to avoid IO overload of the computing system.

[0013] Based on the method provided in the first aspect, in one possible implementation, the second set of thresholds includes a third threshold and a fourth threshold, and the third threshold is greater than the fourth threshold. In this scenario, the implementation process of comparing the group IO information of the first group with the second set of thresholds and determining the scheduling policy for the jobs currently being executed in the first group based on the comparison results can be as follows: comparing the group IO information of the first group with the third threshold and the fourth threshold; if the group IO information of the first group is greater than or equal to the third threshold, suspending the jobs currently being executed in the first group, or limiting the flow of the jobs currently being executed in the first group; if the group IO information of the first group is less than the third threshold and greater than or equal to the second threshold, maintaining the scheduling policy for the jobs currently being executed in the first group; if the group IO information of the first group is less than the third threshold, selecting the previously suspended jobs of the first group and resuming their execution, or canceling the flow limiting of the jobs currently being executed in the first group.

[0014] By suspending or limiting the number of jobs being executed, you can reduce the IO resources occupied by the jobs being executed.

[0015] Based on the method provided in the first aspect, in a possible implementation manner, the group is a queue, a user, a department, or a cluster.

[0016] In an embodiment of the present application, the IO information of multiple jobs running on multiple computing nodes can be aggregated at the user granularity, department granularity, queue granularity, or cluster granularity to implement scheduling of jobs at different levels based on the IO information running on each computing node.

[0017] In a second aspect, a scheduling device is provided, which has the function of implementing the job scheduling method described in the first aspect. The scheduling device includes at least one module, which is used to implement the job scheduling method described in the first aspect.

[0018] In a third aspect, a scheduling device is provided. The scheduling device includes a processor and a memory, wherein the memory is configured to store a program that supports the scheduling device in executing the job scheduling method provided in the first aspect, and to store data used to implement the job scheduling method provided in the first aspect. The processor is configured to execute the program stored in the memory.

[0019] In a fourth aspect, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is run on the computer, the computer executes the job scheduling method described in the first aspect.

[0020] In a fifth aspect, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute the job scheduling method described in the first aspect.

[0021] The technical effects obtained by the corresponding technical means in the above-mentioned second to fifth aspects are similar and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a schematic diagram of the architecture of an HPC service provided by an embodiment of the present application;

[0023] Figure 2 This is a schematic diagram of the architecture of a computing system provided in an embodiment of the present application;

[0024] Figure 3 This is a schematic diagram of the architecture of another computing system provided in an embodiment of the present application;

[0025] Figure 4 This is a flow chart of a job scheduling method provided by an embodiment of the present application;

[0026] Figure 5 is a schematic diagram of a threshold value provided in an embodiment of the present application;

[0027] Figure 6 This embodiment of the present application provides a Figure 3 The functional diagram of the IO perception module shown;

[0028] Figure 7 This is a flow chart of another job scheduling method provided by an embodiment of the present application;

[0029] Figure 8 This is a schematic diagram of the structure of a scheduling device provided in an embodiment of the present application;

[0030] Figure 9 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] In order to make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0032] Before explaining the embodiments of the present application in detail, the terms and application scenarios involved in the embodiments of the present application are first explained.

[0033] High-performance computing (HPC) is used to calculate and solve complex and cutting-edge scientific problems using cutting-edge computer technologies. It is widely used in computing large-scale scientific problems and processing massive amounts of data, such as in weather forecasting, biopharmaceuticals, nuclear explosion simulation, automotive simulation, and chip design and manufacturing.

[0034] HPC cluster: A computer cluster that provides HPC services, also known as a supercomputer.

[0035] HPC scheduler: It is used to manage various resources within the HPC cluster, receive jobs submitted by users, assign jobs to the computing nodes of the HPC cluster for execution through certain scheduling strategies, and monitor and manage the life cycle of jobs.

[0036] Job: A collection of program instances required to complete a specific computing task. Depending on whether the job has started running, the job status can be divided into PENDING and RUNNING. In this embodiment of the application, jobs in the PENDING state are referred to as pending jobs, and jobs in the RUNNING state are referred to as executing jobs.

[0037] Process: A computer program's activity on a set of data. A process is the basic unit of resource allocation in the system.

[0038] Input / output (IO): The process of data input and output between computing nodes and storage systems or other peripheral devices.

[0039] I / O overload (IO overuse): The job accesses the storage system too frequently during operation, consuming excessive I / O resources. If the job frequently accesses I / O, causing I / O information to exceed the storage system's upper limit, this may cause storage system instability.

[0040] Management node (master): A computer used to receive new jobs through the HPC scheduler, manage the cluster, and issue jobs.

[0041] Compute node (agent): A computer that receives jobs from the HPC scheduler and runs / monitors the jobs.

[0042] Distributed file system (DFS): A complete hierarchical file system formed by combining several different logical disk partitions or volumes. The physical storage resources managed by DFS are not necessarily directly connected to the computing nodes, but are connected to the computing nodes through a computer network. The DFS involved in the embodiments of the present application can be a distributed file system such as General Parallel File System (GPFS), Lustre (an open source distributed parallel file system), and FusionStorage (a distributed storage system that can be scaled out on a large scale).

[0043] Figure 1 This is a schematic diagram of the architecture of an HPC service provided by an embodiment of the present application. Figure 1 As shown, HPC services include basic software and hardware platforms. The basic software of HPC services consists of three layers: the operating system (OS) at the bottom, the distributed file system in the middle, and the HPC scheduler at the top. The hardware platform includes computing resources for providing computing power, network resources for network communication, and storage resources for storing data. Examples of computing resources include X86 and graphics processing units (GPUs). Examples of network resources include communication resources based on wireless bandwidth (Infiniband, IB) and remote direct memory access over converged Ethernet (RoCE). Examples of storage resources include high-performance storage.

[0044] For ease of management, computing resources can be divided into multiple compute nodes. The distributed file system ensures that all compute nodes have access to storage resources. The HPC scheduler is responsible for job management, resource management, and cluster management. For example, it dispatches jobs to appropriate compute nodes for execution, which then run the jobs as one or more processes.

[0045] During job execution, the job's processes access storage resources through the distributed file system to read or write application data. This access consumes I / O resources, generating I / O statistics (hereinafter referred to as IO information). The distributed file system can monitor and collect IO information for each process.

[0046] Because storage resources have an upper limit on their I / O access capacity, if certain jobs consume a large amount of these resources, other jobs will not have sufficient resources, causing them to fail or be delayed. Furthermore, if the I / O resources used by all jobs in an HPC cluster exceed the upper limit of the storage system, this can cause the storage system to crash. The storage system can be understood as the entire storage system within the computing system.

[0047] Based on this, an embodiment of the present application provides a job scheduling method in which a scheduling device aggregates the IO information of multiple jobs running on multiple computing nodes in a computing system into group IO information, using the group to which the jobs belong as the granularity. For any group, the scheduling policy for the group's jobs is determined based on the group IO information. This can prevent certain jobs from occupying a large amount of IO resources, preventing other jobs from being unable to run normally. It can also prevent IO access overload on the storage system in the computing system, thereby improving the stability of the computing system.

[0048] The computing system provided in the embodiment of the present application is explained below.

[0049] Figure 2 This is a schematic diagram of the architecture of a computing system provided by an embodiment of the present application. Figure 2 As shown, the computing system 00 includes a scheduling device 10, multiple computing nodes 20, a distributed file system 30 and a storage system 40, wherein each computing node 20 communicates with the scheduling device 10, and each computing node also communicates with the distributed file system 30, and the distributed file system 30 communicates with the storage system 40.

[0050] For example, the computing system 00 is used to provide Figure 1 The HPC service shown, that is, the computing system 00 can be an HPC cluster. In this scenario, the scheduling device 10 is Figure 1 The HPC scheduler in the storage system 40 is Figure 1 A system composed of storage resources in.

[0051] The scheduling device 10 is used to schedule user-submitted jobs to computing nodes 20 according to the scheduling policy. For ease of explanation, the computing node 20 that receives the job is referred to as the primary computing node 20 for the job. After receiving the job, the primary computing node 20 starts the job. After the job starts, one or more processes corresponding to the job are run. These one or more processes may include processes in the primary computing node 20 or processes in computing nodes 20 other than the primary computing node 20. In other words, these one or more processes can be distributed across different computing nodes 20.

[0052] During operation, each process accesses the storage system 40 through the distributed file system 30 to read data from or write data to the storage system 40. Therefore, the distributed file system 30 can count the IO information of each process. In the embodiment of the present application, the IO information is exemplified by the number of IO operations per second (IOPS) or the amount of IO bits per second (BPS). For ease of distinction, the IO information of each process is referred to as process-level IO information.

[0053] For any job, the job-level IO information of the job can be aggregated based on the process-level IO information of one or more processes corresponding to the job. The job-level IO information of the job can be, for example, the sum of the process-level IO information of one or more processes, or the product of the sum and a preset weight, which is not limited in the embodiments of the present application. Aggregating the job-level IO information of the job can be implemented by a distributed file system, or by the main computing node 20 of the job, or by the scheduling device 10, which is not limited in the embodiments of the present application.

[0054] In the embodiment of the present application, the scheduling device 10 is further configured to adjust the scheduling policy for pending and / or currently executing jobs in the computing system based on the IO information of the jobs currently executing on each computing node 20. This can prevent certain jobs from occupying a large amount of IO resources, preventing other jobs from running normally. Furthermore, it can prevent IO access overload on the storage system in the computing system, thereby improving the stability of the computing system. The specific implementation method will be described in detail in the subsequent method embodiments and will not be elaborated here.

[0055] in addition, Figure 2 The scheduling device 10 and the multiple computing nodes 20 managed by the scheduling device 10 can be collectively referred to as a cluster. Figure 2 The description is based on a single cluster. Alternatively, the computing system 00 may include multiple clusters, each of which includes a scheduling device and multiple computing nodes managed by the scheduling device. Different clusters may share the same distributed file system and storage system. Alternatively, each cluster may independently configure a corresponding distributed file system and storage system.

[0056] Figure 3 This is a schematic diagram of another computing system architecture provided by an embodiment of the present application. Figure 3 As shown, the computing system includes a scheduling device 10 , a first computing node, a second computing node and a distributed file system 30 .

[0057] The scheduling device 10 includes a general scheduling module 101, a job management module 102, and an IO perception module 103. The first computing node includes a first agent service module, and the second computing node includes a second agent service module.

[0058] like Figure 3 As shown, the job management module 102 can request resources, such as computing resources, for user-submitted jobs, thereby allocating computing nodes to the jobs through the general scheduling module 101. For example, the general scheduling module 101 schedules job A to the first proxy service module according to the scheduling policy. The first proxy service module initiates job A, that is, runs one or more processes corresponding to job A, thereby enabling the first computing node to run job A. The general scheduling module 101 also schedules job B to the second proxy service module according to the scheduling policy. The second proxy service module initiates job B, that is, runs one or more processes corresponding to job B, thereby enabling the second computing node to run job B.

[0059] During the execution of jobs A and B, the distributed file system 30 periodically writes (dumps) the statistical job-level IO information of job A into the first IO file and writes the statistical job-level IO information of job B into the second IO file, that is, periodically writes IO information.

[0060] The first proxy service module periodically reads the job-level IO information of job A in the first IO file, that is, periodically reads the IO information, and reports the job-level IO information of job A to the action management module 102, that is, reports the IO information. After receiving the job-level IO information of job A, the job management module 102 updates the currently stored job-level IO information of job A, that is, updates the IO information, and reports the updated job-level IO information to the IO perception module 103. In addition, the second proxy service module periodically reads the job-level IO information of job B in the second IO file, and reports the job-level IO information of job B to the job management module 102. After receiving the job-level IO information of job B, the job management module 102 updates the currently stored job-level IO information of job B, and reports the updated job-level IO information to the IO perception module 103.

[0061] The IO awareness module 103 can adjust the scheduling policy for pending jobs based on the job-level IO information of Job A and Job B, such as pausing the scheduling of certain pending jobs. It can also adjust the scheduling policy for currently executing jobs based on the job-level IO information of Job A and Job B, such as limiting the IO access of Job A and / or suspending Job B.

[0062] Limiting the IO access of job A means limiting the subsequent IO information accessed by the process corresponding to job A to the storage system. Suspending job B means pausing the execution of one or more processes corresponding to job B.

[0063] In addition, if Figure 3 As shown, the IO perception module 103 can also issue an IO overload alert via at least one of email, SMS, WeChat, etc. when detecting an overload in the job-level IO information of a job. For example, an email may be sent to the user who submitted the job to notify the user of the current IO overload of the job. IO overload refers to when the job-level IO information of the job exceeds a pre-configured threshold.

[0064] in addition, Figure 3 Each module in the embodiment is a software module. When applying the embodiment of the present application, more or fewer modules can be set according to needs, and they will not be illustrated one by one here.

[0065] Figure 4 This is a flow chart of a job scheduling method provided by an embodiment of the present application. The method is applied to Figure 2 and Figure 3 The scheduling device shown in FIG. Figure 4 As shown, the method includes the following steps.

[0066] Step 401: The scheduling device obtains IO information of multiple running jobs from multiple computing nodes.

[0067] The scheduling device may receive IO collection information from multiple computing nodes, where the IO collection information includes IO information of jobs running on corresponding computing nodes, so that the scheduling device can obtain IO information of multiple jobs running on multiple computing nodes.

[0068] In some embodiments, for any computing node in the computing system, the IO collection information reported by the computing node to the scheduling device includes at least one job identifier and job-level IO information corresponding to the at least one job identifier. The at least one job identifier is used to identify the job being executed on the computing node itself.

[0069] For example, the computing node may periodically obtain job-level IO information for each job being executed on the computing node from the distributed file system. For another example, the computing node may periodically obtain process-level IO information for each process of each job being executed on the computing node from the distributed file system. For any job being executed, the computing node may obtain the job-level IO information of the job being executed based on the process-level IO information of the job being executed.

[0070] Optionally, in other embodiments, for any computing node in the computing system, the IO collection information reported by the computing node includes at least one job identifier and one or more process-level IO information corresponding to the at least one job identifier. The at least one job identifier is used to identify the job being executed on the computing node itself.

[0071] For example, the computing node may periodically obtain process-level IO information of each process of each job being executed on the computing node from the distributed file system.

[0072] In this scenario, the scheduling device can obtain the job-level IO information corresponding to each job identifier based on one or more process-level IO information corresponding to the job identifier, that is, obtain the job-level IO information of the executing job indicated by the job identifier.

[0073] After obtaining the IO information of the multiple jobs running on the multiple computing nodes, the scheduling device can adjust the scheduling policy of the jobs to be scheduled and / or the jobs currently being executed in the computing system based on the IO information of the multiple jobs running on the multiple computing nodes. For specific implementation methods, refer to steps 402 and 403 below.

[0074] Step 402: The scheduling device aggregates the IO information of the multiple jobs running on the multiple computing nodes according to the groups to which the multiple jobs belong, with the group as the granularity, to obtain group IO information.

[0075] In an embodiment of the present application, in order to refine the granularity of job scheduling, the scheduling device may aggregate IO information of multiple jobs according to the groups to which the jobs belong, so as to facilitate subsequent scheduling of jobs of any group based on the group IO information of the group.

[0076] The group to which a job belongs can be understood as a label for classifying the job according to specified rules.

[0077] For example, if jobs are categorized by the user who submitted them, each group to which multiple jobs belong can be a user or a class of users. In this scenario, the scheduling device can implement user-level job scheduling based on the IO information of jobs running on each computing node through the method provided in the embodiments of the present application.

[0078] For example, if the first group is a target user, the scheduling device may adjust the scheduling policy of the jobs to be scheduled and / or jobs currently being executed that match the target user based on the group IO information corresponding to the target user. The jobs to be scheduled and / or jobs currently being executed that match the target user may be understood as the jobs to be scheduled and / or jobs currently being executed that are submitted by the target user.

[0079] For example, if jobs are categorized by the department to which the user submitting the job belongs, each of the groups to which multiple jobs belong can be a department or a category of departments. In this scenario, the scheduling device can implement department-level job scheduling based on the IO information of jobs running on each computing node through the method provided in the embodiments of the present application.

[0080] For example, if the first group is a target department, the scheduling device can adjust the scheduling policy of the pending jobs and / or currently executing jobs that match the target department based on the group IO information corresponding to the target department. The pending jobs and / or currently executing jobs that match the target department can be understood as: pending jobs and / or currently executing jobs submitted by users under the target department.

[0081] For example, if jobs are categorized according to the queues in which they are queued in the scheduling device, each group to which multiple jobs belong can be a queue or a type of queue. In this scenario, the scheduling device can implement queue-level job scheduling based on the IO information of jobs running on each computing node through the method provided in the embodiments of the present application.

[0082] For example, if the first group is a target queue, the scheduling device may adjust the scheduling policy of the jobs to be scheduled and / or jobs being executed that match the target queue based on the group IO information corresponding to the target queue. The jobs to be scheduled and / or jobs being executed that match the target queue may be understood as the jobs to be scheduled and / or jobs being executed in the target queue.

[0083] For example, if jobs are classified according to the cluster in which the scheduling device is located, each group to which multiple jobs belong can be a cluster or a type of cluster. In this scenario, through the method provided in the embodiments of the present application, the scheduling device can implement cluster-level job scheduling based on the IO information of jobs running on each computing node.

[0084] For example, if the first group is the target cluster, the scheduling device can adjust the scheduling policy of the jobs to be scheduled and / or jobs being executed that match the target cluster based on the group IO information corresponding to the target cluster. The jobs to be scheduled and / or jobs being executed that match the target cluster can be understood as the jobs to be scheduled and / or jobs being executed in the target cluster.

[0085] That is, in an embodiment of the present application, the scheduling device can aggregate the IO information of multiple jobs running on multiple computing nodes at user granularity, department granularity, queue granularity, or cluster granularity, so as to realize scheduling of jobs at different levels based on the IO information of jobs running on each computing node.

[0086] It should be noted that the above-mentioned users, departments, queues, or clusters are used as examples to illustrate the group to which the job belongs. The embodiment of the present application does not limit the group to which the job belongs.

[0087] Furthermore, in some embodiments, when the IO information of multiple jobs running on multiple computing nodes includes job-level IO information of the jobs currently executing on each computing node, step 402 may be implemented by: for any group, determining the jobs matching the group among the jobs currently executing on each computing node, summing the job-level IO information of the determined jobs, and obtaining the obtained value as the group IO information corresponding to the group. Alternatively, the job-level IO information of the determined jobs may be summed, and the summed result may be multiplied by a preset weight, and the obtained value may be the group IO information corresponding to the group.

[0088] In an embodiment of the present application, the scheduling device may further include a second interface, so that operation and maintenance personnel can view the aggregated group IO information through the second interface, and then maintain the computing system based on the group IO information.

[0089] Based on this, in some embodiments, the scheduling device receives a group IO information display instruction through the second interface; and the scheduling device displays at least one group IO information corresponding to at least one group.

[0090] For example, the scheduling device provides a human-computer interaction interface, and the operation and maintenance personnel can trigger the group IO information display instruction through the human-computer interaction interface, so that the scheduling device displays at least one group IO information corresponding to at least one group on the human-computer interaction interface. No detailed description is given here.

[0091] Step 403: The scheduling device determines a scheduling strategy for the jobs of the first group according to the group IO information of the first group in the group.

[0092] The first group is any one of the collection of groups to which multiple jobs running on multiple computing nodes belong.

[0093] In some embodiments, step 403 may be implemented by: obtaining a target threshold corresponding to the first group; and adjusting a scheduling strategy for jobs to be scheduled and / or jobs being executed in the first group based on the target threshold and group IO information of the first group.

[0094] The target threshold corresponding to the first group is pre-configured for the first group, and the thresholds for different groups can be the same. Optionally, the thresholds for different groups can also be independent and different values, so that jobs in different groups can be personalized and scheduled, further refining the granularity of job scheduling.

[0095] For example, when the IO information of multiple jobs is aggregated at the user granularity according to the users to whom the jobs belong, a corresponding threshold is independently configured for each user.

[0096] In some embodiments, the scheduling device may further include a first interface. In this scenario, the scheduling device may obtain the target threshold configured by the operation and maintenance personnel through the first interface. That is, in embodiments of the present application, the scheduling device is configured with the first interface, and the operation and maintenance personnel can use the first interface to configure the corresponding threshold for each group on the scheduling device.

[0097] Optionally, the threshold corresponding to each group can also be calculated by the scheduling device using a preset algorithm. For example, the threshold corresponding to each group can be determined based on the total I / O bandwidth of the computing system and the job weight of each group. The job weight can indicate the importance of the job in the corresponding group. This will not be described in detail here.

[0098] Since the jobs in the first group can be jobs to be scheduled or jobs currently being executed, step 403 is described below in two different scenarios. It should be noted that scenario 1 and scenario 2 can be performed independently or simultaneously. Furthermore, the following description uses the first group as an example; the following description can also be applied to other groups within the group to which multiple jobs belong.

[0099] Case 1: Based on the first set of thresholds and the group IO information of the first group, the scheduling policy of the to-be-scheduled jobs in the first group is adjusted.

[0100] In the first scenario, in order to facilitate adjustment of the scheduling policy of the job to be scheduled, a first set of thresholds is set for the first group, that is, the target threshold includes the first set of thresholds.

[0101] In this scenario, step 403 may be implemented by comparing the group IO information of the first group with the first set of thresholds, and determining the scheduling policy for the to-be-scheduled jobs in the first group according to the comparison result.

[0102] In the embodiment of the present application, the scheduling policy of the to-be-scheduled jobs in the first group can be adjusted by using the pre-configured first set of thresholds and the statistically obtained group IO information of the first group to avoid IO overload of the computing system.

[0103] In some embodiments, the first set of thresholds includes a first threshold and a second threshold. The second threshold may be smaller than the first threshold to increase flexibility. Alternatively, the first threshold and the second threshold may be the same, which is not limited in this embodiment of the present application.

[0104] In this scenario, step 403 can be implemented by comparing the group IO information of the first group with the first threshold and the second threshold; if the group IO information of the first group is greater than or equal to the first threshold, stopping the scheduling of the to-be-scheduled jobs of the first cluster; if the group IO information of the first group is less than the first threshold and greater than or equal to the second threshold, maintaining the scheduling policy of the to-be-scheduled jobs of the first cluster; if the group IO information is less than the second threshold, resuming the scheduling of the to-be-scheduled jobs of the first cluster.

[0105] By stopping scheduling pending jobs, more jobs can be prevented from occupying IO resources in the computing system.

[0106] For example, if the group IO information of the first group exceeds a first threshold, the scheduling policy determined by the scheduling device includes stopping scheduling of target pending jobs, where the target pending jobs include one or more pending jobs in the first group. Accordingly, if the group IO information of the first group falls below a second threshold, the scheduling policy determined by the scheduling device includes resuming scheduling of target pending jobs that were stopped prior to the current time.

[0107] Among them, stopping the scheduling of the target job to be scheduled can be understood as: the scheduling device stops allocating computing nodes to the target job to be scheduled. For example, the scheduling device is configured with a scheduling queue, and the scheduling queue includes the jobs currently to be scheduled. The scheduling device selects a job to be scheduled from the scheduling queue according to a certain strategy and allocates computing nodes to the job to be scheduled to complete the scheduling of the job to be scheduled. In this scenario, when the scheduling device determines to stop scheduling the target job to be scheduled, the target job to be scheduled is removed from the scheduling queue. When the scheduling device determines to resume scheduling the target job to be scheduled, the target job to be scheduled is moved into the scheduling queue.

[0108] For example, for Figure 3 In the system shown, when the IO perception module determines that a job to be scheduled needs to be stopped, the scheduling policy for stopping the job to be scheduled can be sent to the general scheduling module, which then stops the job to be scheduled. When the IO perception module determines that a job to be scheduled needs to be resumed, the scheduling policy for resuming the job to be scheduled can be sent to the general scheduling module, which then resumes the job to be scheduled.

[0109] In some embodiments, after determining the jobs to be scheduled belonging to the first group, the scheduling device may select an IO-intensive job from the determined jobs to be scheduled, and determine the selected job as the target job to be scheduled.

[0110] When a user submits a job to the scheduling device, they can indicate whether the job is IO-intensive. For example, when submitting a job, the user can add the "ioIntensive" tag to the submitted job. If "ioIntensive" = true, it indicates that the job is IO-intensive. For example, the user can use the command "dsub --tag "ioIntensive = true" user_cmd" to indicate that the job is IO-intensive.

[0111] Optionally, the scheduling device may also determine whether a job is an IO-intensive job. For example, the scheduling device may determine whether the job is an IO-intensive job based on historical IO information when the job was executed, or based on historical IO information when similar jobs were executed. This will not be explained in detail here.

[0112] In some other embodiments, after the scheduling device determines the jobs to be scheduled that belong to the first group, the scheduling device may also use all the jobs to be scheduled in the first group as target jobs to be scheduled.

[0113] The following is an example of aggregating the IO information of multiple jobs based on user granularity, with the first group being the target user.

[0114] In the scheduling device, a critical value A is configured as the second threshold value and a critical value B is configured as the first threshold value for the target user, wherein the critical value A is smaller than the critical value B.

[0115] When the group IO information of the target user exceeds a critical value B, the scheduling of IO-intensive jobs among the jobs to be scheduled submitted by the target user may be stopped, or the scheduling of all jobs to be scheduled submitted by the target user may be stopped.

[0116] For example, when the target user's group IO information exceeds the critical value B, the IO perception module generates a scheduling policy of io.overuse.pendingJobSchedPolicy. When the value of io.overuse.pendingJobSchedPolicy is holdAllJobs, the scheduling of all pending jobs submitted by the target user is stopped. When the value of io.overuse.pendingJobSchedPolicy is holdIOIntensiveJobs, the scheduling of IO-intensive jobs among the pending jobs submitted by the target user is stopped.

[0117] Accordingly, when the group IO information of the target user is lower than the critical value A, the job that was stopped before the current time can be resumed.

[0118] In addition, in case 1, when the group is a department, a queue, or a cluster, the implementation method of the group being a user can also be referred to, and will not be repeated here.

[0119] Case 2: Based on the target threshold and the group IO information of the first group, the scheduling device determines the scheduling policy for the jobs being executed in the first group.

[0120] Accordingly, any computing node in the computing system can receive the scheduling policy of the job being executed from the scheduling device; and control the job being executed based on the scheduling policy.

[0121] The scheduling device may carry the scheduling policy of the job being executed in a job control message and send the job control message to the computing node. The embodiment of the present application does not limit the format of the job control message.

[0122] In the second scenario, in order to facilitate the scheduling device to adjust the scheduling policy of the job being executed, a second set of thresholds is configured for the first group, that is, the target threshold includes the second set of thresholds.

[0123] In this scenario, step 403 may be implemented by comparing the group IO information of the first group with the second set of thresholds, and determining a scheduling policy for the jobs being executed in the first group according to the comparison result.

[0124] In the embodiment of the present application, the scheduling policy of the jobs being executed in the first group can be adjusted by using the pre-configured second set of thresholds and the statistically obtained group IO information of the first group to avoid IO overload of the computing system.

[0125] In some embodiments, the second set of thresholds includes a third threshold and a fourth threshold. The fourth threshold may be smaller than the third threshold to increase flexibility. Alternatively, the fourth threshold and the third threshold may be the same, which is not limited in this embodiment of the present application.

[0126] It should be noted that the third and fourth thresholds set for the first group are independent of the first and second thresholds set for the first group. In other words, the third and fourth thresholds have no relationship with the first and second thresholds. In applying the embodiments of the present application, operations and maintenance personnel can configure the first and second thresholds for the jobs to be scheduled in the first group, and configure the third and fourth thresholds for the jobs currently being executed in the first group, as needed. This will not be described in detail here.

[0127] In this scenario, step 403 can be implemented by comparing the group IO information of the first group with the third threshold and the fourth threshold; if the group IO information of the first group is greater than or equal to the third threshold, suspending the job being executed, or limiting the flow of the job being executed; if the group IO information of the first group is less than the third threshold and greater than or equal to the second threshold, maintaining the scheduling policy of the job being executed of the first cluster; if the group IO information of the first group is less than the third threshold, selecting the job that was previously suspended of the first cluster and resuming its execution, or canceling the flow limiting of the job being executed.

[0128] For example, if the group IO information of the first group exceeds a third threshold, the scheduling policy determined by the scheduling device includes suspending the target job being executed, or limiting IO access to the target job being executed, where the target job being executed includes one or more of the jobs being executed in the first group. If the group IO information of the first group is below a fourth threshold, the scheduling policy determined by the scheduling device includes resuming the target job being executed that was suspended before the current time, or removing the throttling applied to the target job being executed before the current time.

[0129] Suspending the target job can be understood as controlling the computing node running the target job to stop executing the process corresponding to the target job. Based on this, in some embodiments, after determining that the scheduling policy includes suspending the target job, the scheduling device can also send the scheduling policy to a target computing node among the multiple computing nodes, so that the target computing node suspends the target job. The target computing node is the computing node running the target job.

[0130] For example, for Figure 3 In the illustrated system, when the IO perception module determines that a target job currently being executed needs to be suspended, the scheduling policy for suspending the target job can be sent to the target computing node, and the proxy service module of the target computing node stops executing the process corresponding to the target job currently being executed, thereby suspending the target job currently being executed. When the IO perception module determines that a target job currently being executed that was suspended before the current time needs to be resumed, the scheduling policy for resuming the target job currently being executed that was suspended before the current time can be sent to the target computing node, and the proxy service module of the target computing node re-executes the process corresponding to the target job currently being executed.

[0131] Additionally, limiting the IO access of the target job being executed can be understood as controlling the IO access rate of the computing node running the target job to the storage system. Based on this, in some embodiments, after determining that the scheduling policy includes limiting the IO access of the target job being executed, the scheduling device may further send the scheduling policy to a target computing node among the multiple computing nodes, so that the target computing node limits the IO access of the target job being executed. The target computing node is the computing node running the target job being executed.

[0132] Based on the above content, it can be seen that when the target computing node runs the target job being executed, the process corresponding to the target job being executed accesses the storage system through the distributed file system. Therefore, the IO access of the target job being executed can be limited through the distributed file system.

[0133] Based on this, in some embodiments, when the target computing node receives a scheduling policy from a scheduling device, and the scheduling policy includes limiting the IO access of the job being executed by the target, it can send a limiting notification message to the distributed file system through the IO limiting interface. The limiting notification message carries the identifier of the job being executed by the target, so that the distributed file system limits the IO access of the job being executed by the target.

[0134] That is, in the embodiment of the present application, an IO current limiting interface is configured on the target computing node to notify the distributed file system to limit the IO access of the target job being executed through the IO current limiting interface.

[0135] The embodiment of the present application does not describe in detail how the distributed file system limits the IO access of the target job being executed. When applying the embodiment of the present application, the IO access of the target job being executed can be limited by any method.

[0136] In some embodiments, after determining the currently executing jobs belonging to the first group, the scheduling device may select a specific job from the currently executing jobs in the first group and determine the selected job as the target currently executing job.

[0137] For example, the scheduling device may select the currently executing job with the largest job-level IO information from the currently executing jobs in the first group as the target currently executing job, or select the currently executing job with the shortest running time as the target currently executing job, or select the currently executing job with the lowest priority as the target currently executing job. Examples are not given here one by one.

[0138] In other embodiments, after determining the currently executing jobs belonging to the first group, the scheduling device may also use all the currently executing jobs in the first group as target currently executing jobs.

[0139] The following is an example of aggregating the IO information of multiple jobs based on user granularity, with the first group being the target user.

[0140] In the scheduling device, a critical value C is configured as the fourth threshold value and a critical value D is configured as the third threshold value for the target user, wherein the critical value C is smaller than the critical value D.

[0141] When the target user's group IO information exceeds a critical value D, the target job being executed among the jobs submitted by the target user can be suspended, or the IO access to the target job being executed can be limited.

[0142] For example, when the group IO information of the target user exceeds the critical value D, the IO perception module generates a scheduling policy of io.overuse.runningJobSelectPolicy. When the value of io.overuse.runningJobSelectPolicy is most, the executing job with the largest job-level IO information among all jobs initiated by the target user is suspended, or the IO access to the executing job with the largest job-level IO information is limited. When the value of io.overuse.pendingJobSchedPolicy is recent, the executing job with the shortest running time among all jobs initiated by the target user is suspended, or the IO access to the executing job with the shortest running time is limited. When the value of io.overuse.pendingJobSchedPolicy is minor, the executing job with the lowest priority among all jobs initiated by the target user is suspended, or the IO access to the executing job with the lowest priority is limited.

[0143] Correspondingly, when the group IO information corresponding to the target user is lower than the critical value C, the target job being executed can be resumed or the flow restriction on the target job being executed can be released.

[0144] In addition, in the second case, when the group is a department, a queue, or a cluster, the implementation method of the group being a user can also be referred to, and will not be repeated here.

[0145] To further understand the embodiments of the present application, the method provided in the embodiments of the present application is illustrated below by aggregating the IO information of multiple jobs at the user granularity.

[0146] Figure 5This is a schematic diagram of a threshold value provided in an embodiment of the present application. Figure 5 As shown, for each user who submits a job to the computing system, the operation and maintenance personnel can configure Figure 5 The four thresholds shown are a first threshold, a second threshold, a third threshold and a fourth threshold.

[0147] like Figure 5 As shown, for any user, when the user's group IO information is lower than the second threshold, the jobs submitted by the user to be scheduled are scheduled normally. When the user's group IO information exceeds the first threshold, the scheduling of the jobs submitted by the user to be scheduled is suspended, such as suspending the scheduling of IO-intensive jobs among the jobs submitted by the user to be scheduled. When the user's group IO information is lower than the fourth threshold, the jobs submitted by the user are run normally. When the user's group IO information exceeds the third threshold, the jobs being executed among the jobs submitted by the user are limited or suspended.

[0148] for Figure 3 When initializing the computing system, the administrator can enable the job-level IO monitoring function to enable the computing system to achieve Figure 3 The following functions are shown:

[0149] 1. The distributed file system periodically collects job-level IO information of each running job and writes the job-level IO information of each running job to the corresponding IO file, that is, periodically writes IO information;

[0150] 2. The proxy service module in each computing node periodically reads and parses the IO file to obtain the job-level IO information of the job being executed on itself, that is, periodically reads the IO information. The job-level IO information of the job being executed is reported to the job management module in the scheduling device, that is, reports the IO information;

[0151] 3. When the job management module receives the job-level IO information of any job, it updates the job-level IO information of the job, that is, updates the IO information, and sends the updated IO information to the IO perception module.

[0152] In addition, compute nodes also provide an I / O flow control interface to support job-level, process group-level, and single-process-level I / O flow control for the target job being executed. Job-level I / O flow control can be understood as flow control of the overall I / O information of the target job being executed. Process group-level I / O flow control can be understood as flow control of the total I / O information of a group of processes in the target job being executed. Single-process-level I / O flow control can be understood as flow control of the I / O information of a single process in the target job being executed.

[0153] Figure 6 This embodiment of the present application provides a Figure 3 The functional diagram of the IO sensing module is shown in FIG. Figure 6 As shown, the IO perception module includes the following functions.

[0154] (1) The IO perception module adjusts the scheduling policy of the job to be scheduled, which illustratively includes the following steps:

[0155] 1. After the IO perception module receives new job-level IO information of a job being executed, it automatically updates the job-level IO information of the job recorded internally.

[0156] 2. The IO perception module periodically calculates and updates the group IO information of all users.

[0157] 3. The IO perception module periodically detects the group IO information of all users.

[0158] For any user, such as user U, if user U's group IO information exceeds the first threshold: suspend all job scheduling of user U (io.overuse.pendingJobSchedPolicy = holdAllJobs), or suspend user U's IO-intensive job scheduling (io.overuse.pendingJobSchedPolicy = holdIOIntensiveJobs); if user U's group IO information is lower than the second threshold: resume user U's job scheduling.

[0159] 4. The IO perception module generates an "IO perception scheduling decision (also known as scheduling policy)" based on the detection results, and notifies the scheduling policy to the general scheduling module, thereby affecting job scheduling.

[0160] (2) The IO awareness module adjusts the scheduling policy of the job being executed, which illustratively includes the following steps:

[0161] 5. The IO perception module periodically monitors the group IO information of all users. If user U's group IO information exceeds the third threshold, the module selects the most severely IO-overloaded job among user U's currently executing jobs, such as job X, and suspends or limits it. If user U's group IO information is below the fourth threshold, the module resumes a previously suspended job or releases the current limit on a job, such as job Y.

[0162] (3) The proxy service module controls the job being executed according to the scheduling policy, which illustratively includes the following steps:

[0163] 6. After receiving the job control message carrying the scheduling policy, the proxy service module makes the following judgment:

[0164] If the actionType of the job control message is suspend, the specified job is suspended through the cgroup subsystem (a mechanism for managing processes in groups). If the actionType of the job control message is traffic limit, the distributed file system is notified through the IO traffic limit interface on the compute node to implement IO traffic limit on the executing job. If the actionType of the job control message is resume, the cgroup subsystem is used to resume the specified job. If the actionType of the job control message is traffic limit release, the distributed file system is notified through the IO traffic limit interface on the compute node to release IO traffic limit on the job.

[0165] in addition, Figure 3 The computing system shown can also perform the following functions.

[0166] 7. User-submitted jobs support setting functions. For example, setting ioIntensive=true indicates that the job is an IO-intensive job; for example: dsub--tag "ioIntensive=true" user_cmd.

[0167] 8. The general scheduling module can decide whether to schedule the job to be scheduled based on whether the job to be scheduled is an IO-intensive job and the scheduling policy issued by the IO perception module.

[0168] 9. The scheduling device provides command line tools and API interfaces to support users to query aggregated IO information.

[0169] 10. The scheduling device also supports IO overload alarm function: when IO overload is detected in a certain job being executed, an alarm will be issued via email, SMS or WeChat.

[0170] In addition, the above is an example of aggregating jobs according to the group to which they belong. Optionally, in an embodiment of the present application, when the scheduling device receives IO information reported by each computing node, it may not aggregate the IO information according to the group to which the job belongs, but may control the job through other means. For example, it may directly filter out the executing job with the largest IO information from the IO information, and then control the execution of the executing job to suspend it or limit the IO access to the executing job, or stop scheduling other executing jobs that belong to the same user as the executing job, etc., and the examples will not be explained one by one here.

[0171] In summary, in the embodiments of the present application, the scheduling device can aggregate the IO information of multiple jobs running on multiple computing nodes in the computing system into group IO information, using the group to which the jobs belong as the granularity. For any group, the scheduling policy for the jobs in that group is determined based on the group IO information. This can prevent certain jobs from occupying a large amount of IO resources, preventing other jobs from running normally. It can also prevent IO access overload on the storage system in the computing system, thereby improving the stability of the computing system.

[0172] In addition, an embodiment of the present application also provides another job scheduling method. Figure 7 This is another flow chart of a job scheduling method provided by an embodiment of the present application. Figure 7 As shown, the method includes the following steps.

[0173] Step 701: The computing node sends IO information of the job being executed to the scheduling device.

[0174] Step 702: The computing node receives a scheduling policy from a scheduling device, where the scheduling policy includes suspending or limiting the current of the job being executed.

[0175] Step 703: Based on the scheduling policy, the computing node schedules the job being executed.

[0176] Figure 7 The implementation of the embodiment shown can refer to Figure 4 The embodiments shown will not be described in detail here.

[0177] Figure 8 This is a schematic diagram of the structure of a scheduling device provided by an embodiment of the present application. The scheduling device is applied to a computing system, which also includes multiple computing nodes, and multiple jobs are running in the computing nodes. Figure 8 As shown, the device 800 includes the following modules.

[0178] The acquisition module 801 is used to obtain the input and output IO information of multiple jobs from multiple computing nodes; for specific implementation, refer to Figure 4 Step 401 in the embodiment.

[0179] Aggregation module 802 is used to aggregate the IO information of multiple jobs at the group level to obtain group IO information according to the groups to which the multiple jobs belong. Figure 4 Step 402 in the embodiment.

[0180] The determination module 803 is used to determine the scheduling strategy of the first group's job according to the group IO information of the first group in the group. Figure 4 Step 403 in the embodiment.

[0181] Optionally, the determination module is configured to compare the group IO information of the first group with a first set of thresholds, and determine a scheduling policy for the to-be-scheduled jobs in the first group according to the comparison result.

[0182] Optionally, the first set of thresholds includes a first threshold and a second threshold, the first threshold being greater than the second threshold. The determination module is configured to: compare the group IO information of the first group with the first threshold and the second threshold; if the group IO information of the first group is greater than or equal to the first threshold, stop scheduling the jobs to be scheduled in the first group; if the group IO information of the first group is less than the first threshold and greater than or equal to the second threshold, maintain the scheduling policy for the jobs to be scheduled in the first group; and if the group IO information is less than the second threshold, resume scheduling the jobs to be scheduled in the first group.

[0183] Optionally, the determination module is configured to compare the group IO information of the first group with the second set of thresholds, and determine a scheduling policy for the jobs being executed in the first group according to the comparison result.

[0184] Optionally, the second set of thresholds includes a third threshold and a fourth threshold, and the third threshold is greater than the fourth threshold. The determination module is configured to: compare the group IO information of the first group with the third threshold and the fourth threshold; if the group IO information of the first group is greater than or equal to the third threshold, suspend the currently executing jobs in the first group, or perform flow control on the currently executing jobs in the first group; if the group IO information of the first group is less than the third threshold and greater than or equal to the second threshold, maintain the scheduling policy for the currently executing jobs in the first group; if the group IO information of the first group is less than the third threshold, select the previously suspended jobs of the first group and resume their execution, or cancel the flow control on the currently executing jobs in the first group.

[0185] Optionally, the group is a queue, user, department, or cluster.

[0186] In summary, in the embodiments of the present application, the scheduling device can aggregate the IO information of multiple jobs running on multiple computing nodes in the computing system into group IO information, using the group to which the jobs belong as the granularity. For any group, the scheduling policy for the jobs in that group is determined based on the group IO information. This can prevent certain jobs from occupying a large amount of IO resources, preventing other jobs from running normally. It can also prevent IO access overload on the storage system in the computing system, thereby improving the stability of the computing system.

[0187] It should be noted that the scheduling device provided in the above embodiment only uses the division of the above functional modules as an example to illustrate job scheduling. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the scheduling device provided in the above embodiment and the job scheduling method embodiment are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0188] Figure 9 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. The scheduling device or computing node in the aforementioned embodiment can be Figure 9 See the computer device shown. Figure 9 The computer device includes at least one processor 901 , a communication bus 902 , a memory 903 and at least one communication interface 904 .

[0189] The processor 901 may be a general-purpose central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present application.

[0190] The communication bus 902 may include a pathway for transmitting information between the aforementioned components.

[0191] The memory 903 may be a read-only memory (ROM) or other static storage device that can store static information and instructions, a random access memory (RAM) or other dynamic storage device that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, an optical disc storage (including a compact disc, a laser disc, an optical disc, a digital versatile disc, a Blu-ray disc, etc.), a magnetic disk or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 903 may be independent and connected to the processor 901 via the communication bus 902. The memory 903 may also be integrated with the processor 901.

[0192] Memory 903 is used to store program code for executing the solution of the present application, and is controlled by processor 901 for execution. Processor 901 is used to execute the program code stored in memory 903. The program code may include one or more software modules. The scheduling device or computing node in the aforementioned embodiment can determine data for developing an application through processor 901 and one or more software modules in the program code in memory 903.

[0193] The communication interface 904 uses any transceiver or other device for communicating with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area network (WLAN), etc.

[0194] In a specific implementation, as an embodiment, a computer device may include multiple processors, such as Figure 9 901 and processor 905 are shown in FIG. Each of these processors can be a single-CPU processor or a multi-CPU processor. A processor herein can refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0195] In a specific implementation, as an embodiment, the computer device may further include an output device 906 and an input device 907. The output device 906 communicates with the processor 901 and can display information in a variety of ways. For example, the output device 906 can be a liquid crystal display (LCD), a light emitting diode (LED) display device, a cathode ray tube (CRT) display device, or a projector. The input device 907 communicates with the processor 901 and can receive user input in a variety of ways. For example, the input device 907 can be a mouse, a keyboard, a touch screen device, or a sensor device.

[0196] The above-mentioned computer device can be a general-purpose computer device or a special-purpose computer device. The embodiment of the present application does not limit the type of computer device.

[0197] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a digital versatile disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).

[0198] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.

[0199] The above content is not intended to limit the embodiments of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of the embodiments of the present application.

Claims

1. A job scheduling method, characterized in that: The method is applied to a scheduling device in a computing system, wherein the computing system further comprises a plurality of computing nodes, wherein a plurality of jobs are running in the computing nodes; the method comprises: Obtaining input and output IO information of the plurality of jobs from the plurality of computing nodes; aggregating the IO information of the plurality of jobs according to the groups to which the plurality of jobs belong, with the group as a granularity, to obtain group IO information; A scheduling strategy for jobs of a first group among the groups is determined according to the group IO information of the first group.

2. The method according to claim 1, wherein The determining, according to the group IO information of the first group among the groups, a scheduling strategy for the jobs of the first group includes: The group IO information of the first group is compared with a first set of thresholds, and a scheduling policy for the to-be-scheduled jobs in the first group is determined according to the comparison result.

3. The method according to claim 2, wherein The first set of thresholds includes a first threshold and a second threshold, the first threshold being greater than the second threshold; The comparing the group IO information of the first group with the first set of thresholds and determining the scheduling strategy for the to-be-scheduled jobs in the first group according to the comparison result includes: comparing the group IO information of the first group with the first threshold and the second threshold; If the group IO information of the first group is greater than or equal to the first threshold, stopping the scheduling of the to-be-scheduled jobs of the first group; If the group IO information of the first group is less than the first threshold and greater than or equal to the second threshold, maintaining the scheduling policy of the to-be-scheduled jobs of the first group; If the group IO information is less than the second threshold, the scheduling of the to-be-scheduled jobs of the first group is resumed.

4. The method according to claim 1, wherein The determining, according to the group IO information of the first group among the groups, a scheduling strategy for the jobs of the first group includes: The group IO information of the first group is compared with a second set of thresholds, and a scheduling policy for the jobs being executed in the first group is determined according to the comparison result.

5. The method according to claim 4, wherein The second set of thresholds includes a third threshold and a fourth threshold, and the third threshold is greater than the fourth threshold; The comparing the group IO information of the first group with the second set of thresholds and determining a scheduling strategy for the jobs being executed in the first group according to the comparison result includes: comparing the group IO information of the first group with the third threshold and the fourth threshold; If the group IO information of the first group is greater than or equal to the third threshold, suspending the jobs being executed in the first group, or limiting the flow of the jobs being executed in the first group; If the group IO information of the first group is less than the third threshold and greater than or equal to the second threshold, maintaining the scheduling policy of the jobs being executed in the first group; If the group IO information of the first group is less than the third threshold, the previously suspended jobs of the first group are selected and resumed, or the flow limiting of the jobs being executed in the first group is canceled.

6. The method according to any one of claims 1 to 5, wherein: The group is a queue, user, department or cluster.

7. A scheduling device, characterized in that: The scheduling device is applied to a computing system, wherein the computing system further comprises a plurality of computing nodes, wherein a plurality of jobs are run on the computing nodes; the device comprises: An acquisition module, configured to acquire input and output IO information of the plurality of jobs from the plurality of computing nodes; an aggregation module, configured to aggregate the IO information of the plurality of jobs according to the groups to which the plurality of jobs belong, with the group as a granularity, to obtain group IO information; The determining module is configured to determine a scheduling strategy for jobs of a first group among the groups according to the group IO information of the first group.

8. The scheduling device according to claim 7, characterized in that: The determining module is used for: The group IO information of the first group is compared with a first set of thresholds, and a scheduling policy for the to-be-scheduled jobs in the first group is determined according to the comparison result.

9. The scheduling device according to claim 8, characterized in that: The first set of thresholds includes a first threshold and a second threshold, the first threshold being greater than the second threshold; The determining module is used for: comparing the group IO information of the first group with the first threshold and the second threshold; If the group IO information of the first group is greater than or equal to the first threshold, stopping the scheduling of the to-be-scheduled jobs of the first group; If the group IO information of the first group is less than the first threshold and greater than or equal to the second threshold, maintaining the scheduling policy of the to-be-scheduled jobs of the first group; If the group IO information is less than the second threshold, the scheduling of the to-be-scheduled jobs of the first group is resumed.

10. The scheduling device according to claim 7, wherein: The determining module is used for: The group IO information of the first group is compared with a second set of thresholds, and a scheduling policy for the jobs being executed in the first group is determined according to the comparison result.

11. The scheduling device according to claim 10, wherein: The second set of thresholds includes a third threshold and a fourth threshold, and the third threshold is greater than the fourth threshold; The determining module is used for: comparing the group IO information of the first group with the third threshold and the fourth threshold; If the group IO information of the first group is greater than or equal to the third threshold, suspending the jobs being executed in the first group, or limiting the flow of the jobs being executed in the first group; If the group IO information of the first group is less than the third threshold and greater than or equal to the second threshold, maintaining the scheduling policy of the jobs being executed in the first group; If the group IO information of the first group is less than the third threshold, the previously suspended jobs of the first group are selected and resumed, or the flow limiting of the jobs being executed in the first group is canceled.

12. The scheduling device according to any one of claims 7 to 11, characterized in that: The group is a queue, user, department or cluster.

13. A computer-readable storage medium, characterized in that The method comprises computer program instructions, and when the computer program instructions are executed by a scheduling device, the scheduling device executes the method according to any one of claims 1 to 6.

Images