Resource scheduling method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses a resource scheduling method and device, electronic equipment and a storage medium. The method comprises the steps that it is determined that idle resources do not meet the operation requirement of a current job; the idle resource is the idle resource of the target heterogeneous partition indicated by the scheduling request, and the scheduling request is generated according to the current job; according to scheduling parameters carried in the scheduling request, calculating a first node number threshold value of a first type of resources required for executing the current job; the first type of resources are any type of resources required for executing the current job; comparing the first node number threshold value with the node number of the first type of resources in the idle resources, and determining a resource reservation strategy of the first type of resources; the reserved resource indicated by the resource reservation strategy is used for executing the current job. According to the embodiment of the invention, the resource utilization rate in a heterogeneous partition scene can be improved, the job queuing time is shortened, and the user experience is improved.

Description

Technical Field

[0001] This application relates to the technical field of resource scheduling, and in particular, to a resource scheduling method, apparatus, electronic device, and storage medium. Background Art

[0002] The birth of the supercomputing Internet brings applications and computing power together, which can be used to solve the computing needs of computing-intensive and massive data processing services. For example, for services such as computational simulation or image processing, it can shorten the computing time and improve the computing accuracy. However, when submitting business calculations to a high-performance computing cluster for operation, resource conflicts may occur between different services, and it is very important to reasonably control and allocate heterogeneous resources.

[0003] In the related art, the SLURM system can support the scheduling of heterogeneous resource partitions, but there are significant defects in the mechanism. For example, to ensure the priority use of resources for high-priority jobs, when a high-priority dual-way job queues due to insufficient resources, the main scheduler will reserve all idle nodes in the partition. Even single-way nodes that do not meet the requirements of high-priority dual-way jobs will be reserved and will not be scheduled to low-priority single-way jobs. This mechanism introduces serious job queuing problems and resource waste problems. Although backfilling scheduling can handle this scenario, the scheduling time interval set for backfilling scheduling is generally long, so it is difficult for jobs applying for different resource types to be quickly scheduled. Summary of the Invention

[0004] Embodiments of this application provide a resource scheduling method, apparatus, electronic device, and storage medium to improve resource utilization rate in heterogeneous partition scenarios, reduce job queuing time, and improve user experience.

[0005] In a first aspect, an embodiment of this application provides a resource scheduling method, including:

[0006] Determine that the idle resources do not meet the running requirements of the current job; where the idle resources are the idle resources of the target heterogeneous partition indicated by the scheduling request, and the scheduling request is generated according to the current job;

[0007] Calculate a first node quantity threshold of a first type of resource required to execute the current job according to the scheduling parameters carried in the scheduling request; where the first type of resource is any one of the various resources required to execute the current job;

[0008] Compare the first node quantity threshold with the number of nodes of the first type of resource in the idle resources to determine a resource reservation policy for the first type of resource; where the reserved resources indicated by the resource reservation policy are used to execute the current job.

[0009] In the embodiments of the present application, when the idle resources do not meet the running requirements of the current job, the node quantity thresholds corresponding to various types of resources required for executing the current job can be calculated according to the scheduling parameters carried in the scheduling request. Then, for each type of resource, the corresponding node quantity threshold is compared with the number of nodes of this type of resource in the idle resources to determine the resource reservation strategy for this type of resource. In this way, the resource reservation strategies for various types of resources can be determined respectively. In this manner, instead of pulling the entire target heterogeneous partition into the blacklist for reservation, it is refined to the node level, the resources are classified, and only the nodes related to the job are reserved, and the resources of the remaining nodes can continue to be tried to schedule subsequent other jobs. This improves the resource utilization rate in the heterogeneous resource partition scenario, reduces the job queuing time, and improves the user experience.

[0010] In an alternative embodiment, comparing the first node quantity threshold with the number of nodes of the first type of resource in the idle resources to determine the resource reservation strategy for the first type of resource includes:

[0011] If the number of nodes of the first type of resource in the idle resources is less than the first node quantity threshold, then all of the first type of resource in the idle resources are reserved;

[0012] If the number of nodes of the first type of resource in the idle resources is greater than or equal to the first node quantity threshold, then the first type of resource in the idle resources is reserved according to the first node quantity threshold.

[0013] In the above embodiment, taking the first type of resource as an example, after determining the first node quantity threshold of the first type of resource required for executing the current job, the reservation strategy can be determined in different cases according to the size relationship between the number of nodes of the first type of resource in the idle resources and the first node quantity threshold. For the case where the number of nodes of the first type of resource in the idle resources is less than the first node quantity threshold, it indicates that the first type of resource in the current idle resources cannot meet the demand of the current job for the first type of resource, and other nodes need to release the first type of resource. At this time, all of the first type of resource in the idle resources can be reserved; for the case where the number of nodes of the first type of resource in the idle resources is less than or equal to the first node quantity threshold, it indicates that the first type of resource in the current idle resources can meet the demand of the current job for the first type of resource. At this time, only the first node quantity threshold is required to reserve the first type of resource in the idle resources. In this way, compared with reserving the entire target heterogeneous partition, the resource utilization rate is improved.

[0014] In an alternative embodiment, the scheduling parameters include the requested number of nodes and the requested number of CPU cores; the first type of resource is CPU cores;

[0015] Calculating the first node quantity threshold of the first type of resource required for executing the current job according to the scheduling parameters carried in the scheduling request includes:

[0016] When determining the number of CPU cores that meet the request and the number of required nodes is not greater than the number of requested nodes according to the specifications of the CPU cores deployed on each node, the maximum number of nodes required to execute the current job is the first node quantity threshold.

[0017] In the above embodiments, the specifications of the CPU cores deployed on each node may be different. Therefore, when determining the first quantity threshold, the specifications of the CPU cores deployed on the nodes are considered, and the number of CPU cores that meet the request can ensure the operation of the current job; the number of required nodes cannot be greater than the number of requested nodes, ensuring the rationality of resource allocation and not affecting the operation of other jobs.

[0018] In an alternative embodiment, the specifications of the CPU cores include a first specification and a second specification;

[0019] When determining the number of CPU cores that meet the request and the number of required nodes is not greater than the number of requested nodes according to the specifications of the CPU cores, and the maximum number of nodes required to execute the current job is the first node quantity threshold, it includes:

[0020] Calculate respectively the number of nodes with CPU cores of the first specification and the number of nodes with CPU cores of the second specification required to execute the current job when the number of CPU cores that meet the request and the number of required nodes is not greater than the number of requested nodes;

[0021] Among them, the number of nodes with CPU cores of the first specification is the first node quantity threshold when the first type of resource is CPU cores of the first specification, and the number of nodes with CPU cores of the second specification is the first node quantity threshold when the first type of resource is CPU cores of the second specification.

[0022] In the above embodiments, for resources such as CPU cores, due to their different specifications, they can be regarded as two types of resources, CPU cores of the first specification and CPU cores of the second specification. In this way, when calculating the first node quantity threshold, the first node quantity threshold for CPU cores of the first specification and the first node quantity threshold for CPU cores of the second specification can be calculated respectively. This way, through a more refined division of resources such as CPU cores, the utilization rate of resources is further improved.

[0023] In an alternative embodiment, the scheduling parameter includes the target memory occupied by one CPU core, and the method further includes:

[0024] Determine the specification of the CPU cores deployed on the first node according to the relationship between the target memory and the memory of the first node; where the first node is any one node.

[0025] In the above embodiments, if the target memory occupied by a CPU core is not specified in the scheduling request, the specification of the CPU cores deployed on a node is its initial specification; if the target memory occupied by a CPU core is specified in the scheduling request, in such a case, the actual specification of the CPU cores deployed on the node can be determined according to the relationship between the target memory and the memory of the node. Such a setting comprehensively considers the influence relationship between the two in the case of considering resources including memory and CPU cores, and can improve the resource utilization rate as much as possible while meeting the requirements of CPU cores and memory requirements.

[0026] In an alternative embodiment, the scheduling parameters include the number of requested nodes, the types and quantities of requested heterogeneous cards; the first type of resource is the corresponding type of heterogeneous card.

[0027] Calculating a first node quantity threshold of the first type of resource required to execute the current job according to the scheduling parameters carried in the scheduling request includes:

[0028] According to the types and quantities of heterogeneous cards deployed on each node, determine that when the types and quantities of requested heterogeneous cards are satisfied and the number of required nodes is not greater than the number of requested nodes, the maximum number of nodes required to execute the current job is the first node quantity threshold.

[0029] In the above embodiments, in the case where the types and quantities of requested heterogeneous cards are included in the scheduling parameters, when calculating the first node quantity threshold, similar to the case where the resource is a CPU core, the types and quantities of heterogeneous cards deployed on the node are considered, and the requirements for the types and quantities of requested heterogeneous cards are satisfied, which can ensure the running of the current job; the number of required nodes cannot be greater than the number of requested nodes, ensuring the rationality of resource allocation and not affecting the running of other jobs.

[0030] In an alternative embodiment, after determining the resource reservation policy for the first type of resource, the method further includes:

[0031] Fuse the resource reservation policies of various types of resources to determine the reserved resources;

[0032] Execute the current job using the reserved resources.

[0033] In the above embodiments, when the requested resources are multiple types, when calculating the resource reservation policies for each type of resource, the resource reservation policies for various types can be fused, and then the reserved resources obtained by fusion are used to execute the current job. It is not only applicable to the case where the requested resource is a single type, but also applicable to the case where the requested resources are multiple types.

[0034] In a second aspect, an embodiment of the present application provides a resource scheduling device, including:

[0035] A data processing unit, configured to: determine that the idle resources do not meet the running requirements of the current job; wherein, the idle resources are the idle resources of the target heterogeneous partition indicated by the scheduling request, and the scheduling request is generated according to the current job.

[0036] A data calculation unit, configured to: calculate a first node quantity threshold of a first type of resources required to execute the current job according to the scheduling parameters carried in the scheduling request; wherein, the first type of resources is any one of various types of resources required to execute the current job.

[0037] The data processing unit is further configured to: compare the first node quantity threshold with the number of nodes of the first type of resources in the idle resources to determine a resource reservation policy for the first type of resources; wherein, the reserved resources indicated by the resource reservation policy are used to execute the current job.

[0038] In a third aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the steps of any of the above methods are implemented.

[0039] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the steps of any of the above methods are implemented.

[0040] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program. When the computer program is executed by a processor, the steps of any of the above methods are implemented. Description of the Drawings

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required to be used in the embodiments of the present application will be briefly introduced below. Obviously, the drawings introduced below are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0042] Figure 1 It is an architecture diagram of a scheduling system provided by an embodiment of the present application;

[0043] Figure 2 It is a schematic diagram of a resource scheduling process in the related art;

[0044] Figure 3 It is a flowchart of a resource scheduling method provided by an embodiment of the present application;

[0045] Figure 4 It is a schematic diagram of a resource scheduling process provided by an embodiment of the present application;

[0046] Figure 5 A flowchart of a complete resource scheduling method provided by an embodiment of the present application;

[0047] Figure 6 A schematic structural diagram of a resource scheduling device provided by an embodiment of the present application;

[0048] Figure 7 A schematic structural diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0049] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application.

[0050] For the convenience of understanding, the terms involved in the embodiments of the present application are explained below:

[0051] (1) Computing resources: Abbreviated as resources, which refer to various hardware and software resources that can be used to execute computing tasks in a computer system or network environment. In the embodiments of the present application, it mainly refers to hardware resources, which mainly include the following types:

[0052] Central Processing Unit (CPU), the core component of a computer, responsible for executing instructions and processing data. Its performance is usually determined by factors such as the number of cores, main frequency, and cache. For example, a processor of a certain manufacturer has 12 cores and a main frequency of up to 3.6 GHz, and can quickly process various complex computing tasks.

[0053] Memory, used to temporarily store running programs and data. Its capacity and speed have an important impact on the running efficiency of the computer. When the memory is insufficient, the computer may experience lag. For example, a computer equipped with 16GB, fourth-generation double data rate synchronous dynamic random access memory (DDR4), 3200MHz memory can run multiple applications simultaneously without significant performance degradation.

[0054] Storage devices, including hard disks (mechanical hard disks and solid-state drives), optical discs, USB flash drives, etc., used for long-term storage of data and programs. Solid-state drives have the advantages of fast read and write speeds and good shock resistance, and can significantly improve the system startup speed and software loading speed. For example, a solid-state drive of a certain manufacturer adopts the Non-Volatile Memory Express (NVMe) protocol, and the sequential read speed can reach more than 7000MB / s.

[0055] The Graphics Processing Unit (GPU) also plays an important role in fields such as graphics rendering, deep learning, and scientific computing. For example, NVIDIA's A100 Tensor Core GPU has powerful parallel computing capabilities and performs excellently in artificial intelligence model training.

[0056] Network devices, such as network cards and routers, are used for communication and data transmission between computers, and their transmission rate and stability affect the performance of network computing. For example, a 10 Gigabit Ethernet network card can provide a network bandwidth of up to 10 Gbps to meet the needs of high-speed data transmission.

[0057] (2) High Performance Computing (HPC) Scheduling System: An operating system for clusters, which is a key component in a high-performance computing environment and is used to manage and allocate computing node resources in the cluster. Its main task is to effectively manage thousands or even millions of processor cores, a large amount of memory, and storage resources, and efficiently execute large-scale computing tasks.

[0058] At the resource management level, it can uniformly manage and monitor various resources in the computing cluster, such as CPUs, memory, storage, and networks. It can understand the usage of resources in real time, including the idle and busy states of resources and the performance metrics of resources.

[0059] At the task scheduling level, it can reasonably allocate tasks to different computing nodes according to the requirements of the tasks and the availability of resources. Scheduling algorithms usually consider factors such as task priority, resource requirements, and node load to ensure that tasks can be completed efficiently.

[0060] At the job management level, it can support users in submitting jobs and manage the entire life cycle of jobs, including states such as job queuing, waiting, running, pausing, resuming, and terminating. Users can view the execution progress and status information of jobs through the interfaces provided by the scheduling system.

[0061] (3) Scheduling Strategy: A strategy for reasonably arranging various job requests to run on different types of computing nodes. Its essence is to solve the contradiction between the infinity of resource requests and the finiteness of resources. Common scheduling algorithms include: First-Come, First-Served, Shortest Job First, Multi-Factor Priority, Resource Reservation, Job Backfilling, Fair Sharing, Load Balancing, Preemption Strategy, etc.

[0062] (4) The Simple Linux Utility for Resource Management (SLURM) is an open-source and highly scalable cluster resource management and job scheduling system. It can be simply understood as a multi-machine resource and task management system and is widely used in large-scale computing environments such as supercomputers and cloud computing clusters. It mainly provides the following three key functions: Resource allocation means allocating computing resources to users within a specific time period with exclusive or non-exclusive access rights so that they can execute jobs. Simply put, it provides authorization and allocation of computing resources for user jobs; Job management means providing a framework for starting, executing, and monitoring job segments on nodes; Job scheduling means arbitrating resource contention by managing the queue of pending jobs, for example, adjusting the resource allocation order according to priorities or different scheduling policies.

[0063] Its main functions include resource management, job scheduling, job management, monitoring, and reporting, etc.

[0064] (5) Heterogeneous cards refer to computing cards that contain different types of processor cores, such as GPU-CPU heterogeneous cards, FPGA-CPU heterogeneous cards, ASIC-CPU heterogeneous cards, etc.

[0065] (6) Homogeneous nodes and heterogeneous nodes: A homogeneous node refers to a node without a deployed heterogeneous card, and a heterogeneous node refers to a node with a deployed heterogeneous card.

[0066] (7) Homogeneous partitions and heterogeneous partitions: A partition is a set of node resources represented by the scheduling system. Each node is deployed with resources such as CPUs, GPUs, memory, and hard disks. Placing the same type of resources in one partition is called a homogeneous partition, and when there are nodes with multiple resource types in a partition, it is called a heterogeneous partition. For example, if the resources deployed on multiple nodes within a partition are all 32-core CPUs, high-frequency, 64G memory, and heterogeneous nodes, then this partition is called a homogeneous partition. If any element of any two nodes is different, then this partition is called a heterogeneous partition.

[0067] The quantity of any element in the attached drawings is for illustration rather than limitation, and any naming is only for distinction without any restrictive meaning.

[0068] In recent years, the rapid development of artificial intelligence and large models has placed extremely high demands on computing power, making high-performance computing an important indicator for measuring technological and economic strength. For this reason, the Supercomputing Internet has emerged. It draws on the concept of the Internet to achieve the interconnection and sharing of computing resources, and through collaboration with application operation and development, it has commercial sustainability. The Supercomputing Internet not only provides users with efficient computing services, improving resource utilization efficiency, but also lays a foundation for establishing an independent and controllable technical system in the field of high-performance computing. The construction of the Supercomputing Internet will boost the development of high-performance computing and provide solid support for complex computing needs, cutting-edge scientific research, and industrial innovation.

[0069] The Supercomputing Internet brings together all applications and computing power, providing a solution for ultra-high floating-point computing capabilities, which can be used to meet the computing needs of computing-intensive and massive data processing services, such as scientific research, weather forecasting, computational simulation, military research, CAD / CAE, biopharmaceuticals, gene sequencing, image processing, etc., shortening the large amount of computing time required and improving computing accuracy. However, how can the calculations of the above services be submitted to the high-performance computing cluster for operation? How can conflicts in resources used between different services be avoided? How can the corresponding heterogeneous resources be reasonably controlled and allocated? At this time, a scheduling system is needed to integrate jobs and resources.

[0070] With the interconnection of various computing resources in the Supercomputing Internet, these computing resources are also divided into different types. According to the processor frequency, there are high main frequencies and multi-cores, etc. Generally, different queues (partitions) need to be divided according to different types. Each queue must ensure that there is only the same type of computing resources. During use, the same type of resources can also be divided into multiple queues according to user needs. The number of resources owned by each queue is different, and the users allowed to submit to each queue are also different, thus realizing the function of isolation and customization. When users submit tasks, they need to specify which queue to submit to for operation. The scheduling system will allocate resources according to the resources owned by the queue to ensure the operation of the job.

[0071] The complexity and diversity of computing tasks have promoted the evolution of different node specifications within the cluster. The drawbacks of the management form of dividing different partitions according to node specifications are gradually emerging, such as large-scale jobs cannot run, resource fragmentation, uneven load between partitions, limited support for heterogeneous jobs, high management difficulty, low resource utilization rate, etc., which cannot meet the requirements of job diversification and high cluster utilization rate. The demand for merging multiple different specifications of nodes into the same partition for management to reduce the number of partition divisions and improve resource utilization rate is increasing.

[0072] After nodes of multiple resource types are merged into one partition, it is a challenge for the scheduling policy to reasonably select appropriate nodes from the partition for job execution. SLURM can support the scheduling of heterogeneous resource partitions, but there are significant defects in its mechanism. Among them, to ensure the priority use of resources for high-priority jobs, when a high-priority dual-socket job queues due to insufficient resources, the main scheduler will reserve all idle nodes in the partition. Even single-socket nodes that do not meet the requirements of high-priority dual-socket jobs will be reserved and will not be scheduled to low-priority single-socket jobs. This mechanism introduces serious job queuing problems and resource waste problems. Backfill scheduling can handle this scenario, but the scheduling time interval set for backfill scheduling is generally long, so it is difficult for jobs applying for different resource types to be quickly scheduled.

[0073] In the actual application process, applications in fields such as weather forecasting, satellite image processing, biomedicine, and industrial simulation are inseparable from high-performance computing during calculations. However, every high-performance cluster is inseparable from a scheduling system. The scheduling system will reasonably allocate and arrange resources. After the user submits the application program through commands or scripts, the server will notify the scheduler when it receives the command. The scheduler will match the partition where the job is submitted with the requested resources. If the idle resources in the partition meet the job execution requirements, it will generate a corresponding resource list and notify the server. After receiving the resource list, the server will notify the computing nodes through the network. Each node also has a corresponding daemon process. After receiving the request command from the server, the computing node starts to run the program. During the running process, the computing node collects the running status of the job on the node and the resource usage situation until the job ends. The computing node notifies the management node, and the scheduling system reclaims the resources of the computing node and stores some information during the entire process from job submission to completion in the accounting database.

[0074] This application mainly aims at the situation where the idle resources in the partition do not meet the job execution requirements, and provides a resource scheduling method. When the idle resources do not meet the running requirements of the current job, this solution can calculate the node quantity thresholds corresponding to various resources required to execute the current job according to the scheduling parameters carried in the scheduling request. Then, for each type of resource, it can compare its quantity threshold with the number of nodes of this type of resource in the idle resources to determine the resource reservation strategy for this type of resource. By applying this method, it is not necessary to reserve the resources of the entire partition, thus not affecting the normal operation of other jobs that require other resources in the partition.

[0075] After introducing the design concept of the embodiments of this application, the following briefly introduces the application scenarios applicable to the technical solutions of the embodiments of this application. It should be noted that the following introduced application scenarios are only for explaining the embodiments of this application rather than limiting. In specific implementation, the technical solutions provided by the embodiments of this application can be flexibly applied according to actual needs.

[0076] ReferenceFigure 1 , which is an architecture diagram of a scheduling system provided by an embodiment of the present application. When applying this scheduling system, after a job is submitted to the cluster, each job will be sorted in descending order of priority. The scheduler will schedule jobs in sequence according to the sorted job list, compare the resources requested by each job with the idle resources in the submission partition. If the resources meet the requirements for job running, they will be reasonably allocated to the corresponding resources for running; if not, the job will continue to queue and wait for the running job to complete and release resources. Common scheduling policies include first-come-first-served, resource reservation, job backfilling, fair sharing, preemption strategy, etc.

[0077] In the SLURM scheduling system, there are mainly two types of job scheduling: main scheduling and backfill scheduling. Main scheduling is the core of the scheduling system, which is characterized by a short scheduling time interval. When events such as job submission, completion, and configuration changes occur, scheduling can be quickly performed. However, when a high-priority job is scheduled and cannot run due to insufficient resources in the partition, the scheduling system will blacklist the entire partition to ensure that the job can run first, and the remaining idle resources will be reserved for this job. For homogeneous partitions, since the resource types in the homogeneous partitions are the same, this operation has no problem. However, for heterogeneous partitions, such as single-way and dual-way partitions, when a job requesting dual-way resources cannot run, it should not affect the scheduling of single-way jobs. Figure 2 It is a schematic diagram of a resource scheduling process in the prior art. Among them, the number of nodes N requested by the submitted dual-way job is 4, and the number of CPU cores n requested is 240. However, there are insufficient dual-way nodes (64-core nodes) in the idle resources. At this time, the entire partition is blacklisted. However, if there is a single-way node job at this time (the number of nodes N requested is 1, and the number of CPU cores n requested is 24), then the operation of this single-way job will be affected at this time. However, in the actual application process, only dual-way nodes need to be reserved, which will neither affect the operation of dual-way jobs nor the operation of single-way jobs.

[0078] It should be noted that in this example, a single-way node usually refers to a 32-core node, and a dual-way node usually refers to a 64-core node; a single-way job usually refers to a job that requires a 32-core node, and a dual-way job usually refers to a job that requires a 64-core node. This is only an example and does not form a specific limitation.

[0079] Of course, the method provided by the embodiments of the present application is not limited to Figure 1 the application scenarios shown, and can also be used in other possible application scenarios, which are not limited by the embodiments of the present application. The functions that can be realized by each device in Figure 1 the application scenarios shown will be described together in the subsequent method embodiments, and will not be elaborated here too much.

[0080] To further illustrate the technical solutions provided by the embodiments of the present application, the following will be described in detail in conjunction with the accompanying drawings and specific implementation manners. Although the embodiments of the present application provide method operation steps as shown in the following embodiments or drawings, based on routine or non-creative labor, more or fewer operation steps may be included in the method. In steps where there is no necessary causal relationship logically, the execution order of these steps is not limited to the execution order provided by the embodiments of the present application.

[0081] The following will be described in conjunction with Figure 1 the application scenario shown below to illustrate the technical solutions provided by the embodiments of the present application.

[0082] Referring to Figure 3 , the embodiments of the present application provide a resource scheduling method, including the following steps:

[0083] S301: Determine that the idle resources do not meet the running requirements of the current job.

[0084] Among them, the idle resources are the idle resources of the target heterogeneous partition indicated by the scheduling request, and the scheduling request is generated according to the current job.

[0085] S302: Calculate the first node quantity threshold of the first type of resources required to execute the current job according to the scheduling parameters carried in the scheduling request.

[0086] Among them, the first type of resources is any one of various types of resources required to execute the current job;

[0087] S303: Compare the first node quantity threshold with the number of nodes of the first type of resources in the idle resources to determine the resource reservation strategy for the first type of resources.

[0088] Among them, the reserved resources indicated by the resource reservation strategy are used to execute the current job

[0089] In the embodiments of the present application, when the idle resources do not meet the running requirements of the current job, the node quantity thresholds corresponding to various types of resources required to execute the current job can be calculated according to the scheduling parameters carried in the scheduling request. Then, for each type of resources, the corresponding node quantity threshold is compared with the number of nodes of this type of resources in the idle resources to determine the resource reservation strategy for this type of resources. In this way, the resource reservation strategies for various types of resources can be determined respectively. In this way, instead of pulling the entire target heterogeneous partition into the blacklist for reservation, it is refined to the node level, the resources are classified, and only the nodes related to the job are reserved, and the resources of the remaining nodes can continue to be tried to schedule subsequent other jobs. This improves the resource utilization rate in the heterogeneous resource partition scenario, reduces the job queuing time, and improves the user experience.

[0090] Regarding S301, when a job (such as image processing) is detected, a scheduling request is generated. The parameters carried in the scheduling request include some or all of the requested partition identifier, the requested number of nodes, the requested number of CPU cores, the requested memory, the type and quantity of the requested heterogeneous cards.

[0091] Determine whether the idle resources of the target heterogeneous partition indicated by the scheduling request meet the running requirements of the current job. If they meet, the current job can be directly run. In the embodiments of the present application, it is mainly for the case where the idle resources do not meet the running requirements of the current job. Optionally, meeting the running requirements of the current job means that all types of resources meet the running requirements of the current job.

[0092] Regarding S302, for each type of resource, the corresponding node quantity threshold can be calculated. Taking any one type of resource (represented by the first type of resource) as an example for illustration. Optionally, according to the type of the first type of resource and the content in the scheduling parameters, the calculation process of the first node quantity threshold is described in different cases.

[0093] Optionally, according to the type of the first type of resource and the content in the scheduling parameters, the calculation process of the first node quantity threshold is described in different cases.

[0094] The first case: The first type of resource is CPU cores, and the scheduling parameters include the requested number of nodes and the requested number of CPU cores.

[0095] In this case, according to the specifications of the CPU cores deployed on each node, when the requested number of CPU cores is met and the required number of nodes is not greater than the requested number of nodes, the maximum number of nodes required to execute the current job is the first node quantity threshold.

[0096] Generally, the specifications of the CPU cores deployed on multiple nodes can be the same or different. For example, the specifications of the CPU cores deployed on some nodes are 32 cores, and the specifications of the CPU cores deployed on some nodes are 64 cores. Therefore, according to the specifications of the CPU cores, the corresponding first node quantity thresholds can be calculated respectively.

[0097] Exemplarily, the specifications of the CPU cores can include the first specification (such as a 32-core CPU) and the second specification (such as a 64-core CPU). When the requested number of CPU cores is met and the required number of nodes is not greater than the requested number of nodes, the number of nodes of the CPU cores of the first specification required to execute the current job and the number of nodes of the CPU cores of the second specification can be calculated respectively. The number of nodes of the CPU cores of the first specification here is the first node quantity threshold when the first type of resource is the CPU cores of the first specification, and the number of nodes of the CPU cores of the second specification is the first node quantity threshold when the first type of resource is the CPU cores of the second specification.

[0098] In a specific example, the number of requested nodes is 2, and the number of requested CPU cores is 96. In the first implementation method, it is achieved by two nodes each deployed with a 64-core CPU; in the second implementation method, it is achieved by one node deployed with a 32-core CPU and one node deployed with a 64-core CPU. It cannot be achieved by three nodes each deployed with a 32-core CPU, because the number of requested nodes is 2, and the number of nodes required in this implementation method is greater than 2, so this method is not optional. In this example, for the first implementation method, the number of nodes with 64-core CPUs can be 1 or 2, so the first quantity threshold corresponding to the CPU cores of this specification is 2; the number of nodes with 32-core CPUs is 1, so the first quantity threshold corresponding to the CPU cores of this specification is 1.

[0099] The second case: The scheduling parameter includes the target memory occupied by one CPU core.

[0100] In this case, during the scheduling process, the target memory occupied by one CPU core is specified, which will affect the specification of the CPU cores deployed on a node. Therefore, the specification of the CPU cores deployed on the first node can be determined according to the relationship between the target memory and the memory of the first node. Here, the first node is any node.

[0101] In a specific example, the idle resources include the resources of 10 nodes, each node has 32 cores. Among them, the memory of nodes 1 - 5 is 32G, and the memory of nodes 6 - 10 is 64G. If the target memory occupied by one CPU core is not limited, each node still has 32 cores. However, if the target memory occupied by one CPU core is 2G, then nodes 1 - 5 actually have 16 cores, and nodes 6 - 10 actually have 32 cores.

[0102] Therefore, in this case, the specification of the CPU cores actually deployed on each node can be determined by combining the influence of memory on cores, and thus the accuracy of the first quantity threshold calculated in the first case can be ensured.

[0103] The third case: The scheduling parameter includes the number of requested nodes, the type and quantity of requested heterogeneous cards; the first type of resources is heterogeneous cards of the corresponding type.

[0104] In this case, according to the type and quantity of heterogeneous cards deployed on each node, when the type and quantity of requested heterogeneous cards are satisfied and the required number of nodes is not greater than the number of requested nodes, the maximum number of nodes required to execute the current job is the first node quantity threshold.

[0105] For example, if the requested heterogeneous cards are 4 CPU-GPUs and the number of requested nodes is 3, then among the nodes with idle resources of this type of heterogeneous card, a node with two CPU-GPUs and a node with two CPU-GPUs can be found, so the first node quantity threshold is 3.

[0106] Regarding S303, compare the first node quantity threshold with the number of nodes of the first type of resources in the idle resources to determine the resource reservation policy for the first type of resources.

[0107] Optionally, according to the comparison result, it can be determined that there are mainly the following two situations:

[0108] The first situation: The number of nodes of the first type of resources in the idle resources is less than the first node quantity threshold, then all the first type of resources in the idle resources are reserved.

[0109] In this case, it indicates that other nodes need to release the first type of resources before the current job's demand for the first type of resources can be met. At this time, all the first type of resources in the idle resources are reserved.

[0110] The second situation: The number of nodes of the first type of resources in the idle resources is greater than or equal to the first node quantity threshold, then the first type of resources in the idle resources are reserved according to the first node quantity threshold.

[0111] In this case, it indicates that the first type of resources in the idle resources are sufficient for the current job. Therefore, it can be reserved according to the first node quantity threshold. However, at this time, other resources that may be used to execute the current job may still need to wait for other jobs to release.

[0112] In a specific example, the requested resources are three 64-core nodes and one 32-core node, but there are two idle 64-core nodes, then all are reserved, and there are many idle 32-core nodes, only one needs to be reserved.

[0113] In the above embodiments, for the resource reservation policy of each type of resource, in the actual application process, if the resources used to execute the current job are multiple, after determining the resource reservation policy for the first type of resources, the resource reservation policies of various types of resources can be integrated to determine the reserved resources; the reserved resources are used to execute the current job.

[0114] Exemplarily, the integration here can comprehensively consider memory, CPU cores, heterogeneous cards, etc., and ensure that the number of nodes for the overall application is less than or equal to the number of requested nodes, and follow the principle of the least number of nodes for the application.

[0115] In summary, in the embodiments of the present application, multiple optimization and improvement operations have been performed on the main scheduling process for heterogeneous partitions. The partition list mechanism is cancelled, and the node blacklist is refined at a fine-grained level to ensure that when high-priority jobs cannot run due to lack of resources, low-priority jobs can still continue to attempt scheduling. The algorithm is improved. By traversing the nodes from the most to the least in terms of the number of CPUs in a recursive manner, the maximum number of jobs that each type of resource can support is determined, quickly excluding the resource types that cannot support job running, narrowing the scope of resource scheduling, and providing data support for subsequent resource selection. Among them, the selection and reservation of resources are relatively important. Compare the actual situation of the cluster with the nodes of each type of resource required by the job. If the idle resources of the current resource type no longer meet the job running requirements, it means that there is a lack of resources and it is necessary to wait for the job to end and release the resources. Therefore, all idle resources need to be reserved. If the current resource type meets the job requirements, reservation is made according to the maximum number of the resource types required by the job. The selection mechanism for reserved nodes will select whether to apply in a centralized block or decentralized manner according to the configuration in the system. Finally, only the resources required by the job are retained, and the remaining resources are released for subsequent low-priority jobs to run. This not only improves the scheduling efficiency and system fault tolerance, but also makes full use of the advantage of high main scheduling execution frequency, optimizes the job response time, and avoids resource waste problems. This carefully designed scheduling scheme will greatly improve the system performance and bring a significant improvement to the job processing and system response speed.

[0116] Figure 4 It is a schematic diagram of a resource scheduling process provided by the embodiments of the present application. By Figure 4 and Figure 2 Comparing, it can be seen that when applying the solution provided by the embodiments of the present application for heterogeneous partitions, when a job cannot run due to insufficient idle resources, the entire partition is no longer pulled into the blacklist for reservation, but refined to the node level, and only the nodes related to the job are reserved, and the remaining resources can continue to attempt to schedule subsequent jobs.

[0117] Figure 5 It is a flowchart of a complete resource scheduling method provided by the embodiments of the present application. In this flowchart, the first type of resource is described by taking the CPU core as an example. For the cases where the first type of resource is memory or heterogeneous card, the situation is similar and will not be elaborated here.

[0118] S501: Determine that the idle resources do not meet the running requirements of the current job.

[0119] S502: According to the specifications of the CPU cores deployed on each node, when it is determined that the number of CPU cores that meet the request and the number of nodes required is not greater than the number of nodes requested, the maximum number of nodes required to execute the current job is the first node quantity threshold.

[0120] S503-1: If the number of nodes of the first type of resources in the idle resources is less than the first node quantity threshold, then reserve all the first type of resources in the idle resources;

[0121] S503-2: If the number of nodes of the first type of resources in the idle resources is greater than or equal to the first node quantity threshold, then reserve the first type of resources in the idle resources according to the first node quantity threshold.

[0122] S504: Integrate the resource reservation strategies of various types of resources to determine the reserved resources.

[0123] S505: Apply the reserved resources to execute the current job.

[0124] Among them, the implementation manners of each step can be referred to the foregoing embodiments and will not be elaborated here.

[0125] In the embodiments of the present application, the blacklist mechanism is refined to the node level; accurately calculate the resource types and resource quantities required by the job; implement resource reservation to ensure the operation of high-priority jobs. Specifically, the scheduling system can accurately reserve the corresponding node resources according to the resource specifications applied by the job; while retaining the priority use right of the resources of high-priority jobs, low-priority jobs in the scheduling partition attempt to use nodes of other specifications. This optimization improves the resource utilization rate in the heterogeneous resource partition scenario, reduces the queuing time of user jobs, and improves the user experience.

[0126] As Figure 6 shown, based on the same inventive concept as the above resource scheduling method, the embodiments of the present application further provide a resource scheduling device, including a data processing unit 61 and a data calculation unit 62.

[0127] Among them, the data processing unit 61 is used to: determine that the idle resources do not meet the running requirements of the current job; wherein, the idle resources are the idle resources of the target heterogeneous partition indicated by the scheduling request, and the scheduling request is generated according to the current job;

[0128] The data calculation unit 62 is used to: calculate the first node quantity threshold of the first type of resources required to execute the current job according to the scheduling parameters carried in the scheduling request; wherein, the first type of resources is any one of the various types of resources required to execute the current job;

[0129] The data processing unit 61 is further used to: compare the first node quantity threshold with the number of nodes of the first type of resources in the idle resources to determine the resource reservation strategy of the first type of resources; wherein, the reserved resources indicated by the resource reservation strategy are used to execute the current job.

[0130] In an optional implementation manner, the data processing unit 61 is specifically used to:

[0131] If the number of nodes of the first type of resources in the idle resources is less than the first node quantity threshold, then all the first type of resources in the idle resources are reserved;

[0132] If the number of nodes of the first type of resources in the idle resources is greater than or equal to the first node quantity threshold, then the first type of resources in the idle resources are reserved according to the first node quantity threshold.

[0133] In an alternative embodiment, the scheduling parameters include the requested number of nodes and the requested number of CPU cores; the first type of resources is CPU cores;

[0134] The data calculation unit 62 is specifically configured to:

[0135] According to the specifications of the CPU cores deployed on each node, when it is determined that the number of CPU cores that meet the request and the required number of nodes is not greater than the requested number of nodes, the maximum number of nodes required to execute the current job is the first node quantity threshold.

[0136] In an alternative embodiment, the specifications of the CPU cores include a first specification and a second specification;

[0137] The data calculation unit 62 is specifically configured to:

[0138] Calculate respectively the number of nodes of the CPU cores of the first specification and the number of nodes of the CPU cores of the second specification required to execute the current job when the number of CPU cores that meet the request and the required number of nodes is not greater than the requested number of nodes;

[0139] Among them, the number of nodes of the CPU cores of the first specification is the first node quantity threshold when the first type of resources is the CPU cores of the first specification, and the number of nodes of the CPU cores of the second specification is the first node quantity threshold when the first type of resources is the CPU cores of the second specification.

[0140] In an alternative embodiment, the scheduling parameters include the target memory occupied by one CPU core, and the data processing unit 61 is further configured to:

[0141] Determine the specification of the CPU cores deployed on the first node according to the relationship between the target memory and the memory of the first node; wherein, the first node is any one node.

[0142] In an alternative embodiment, the scheduling parameters include the requested number of nodes, the type and quantity of the heterogeneous cards requested; the first type of resources is the corresponding type of heterogeneous cards;

[0143] The data calculation unit 62 is specifically configured to:

[0144] Based on the types and quantities of heterogeneous cards deployed by each node, when determining the types and quantities of heterogeneous cards that meet the requirements and the number of required nodes is not greater than the number of requested nodes, the maximum number of nodes required to execute the current job is the first node quantity threshold.

[0145] In an alternative embodiment, the data processing unit 61 is further configured to: after determining the resource reservation policy for the first type of resources:

[0146] Integrate the resource reservation policies of various types of resources to determine the reserved resources;

[0147] Apply the reserved resources to execute the current job.

[0148] The resource scheduling device provided in the embodiments of the present application adopts the same inventive concept as the above-mentioned resource scheduling method and can achieve the same beneficial effects, which will not be elaborated here.

[0149] Based on the same inventive concept as the above-mentioned resource scheduling method, the embodiments of the present application further provide an electronic device, which may specifically be a desktop computer, a portable computer, a smart phone, a tablet computer, a personal digital assistant (Personal Digital Assistant, PDA), a server, etc. As Figure 7 shown, the electronic device may include a processor 71 and a memory 72.

[0150] The processor 71 may be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present application may be directly embodied as being executed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

[0151] The memory 72, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. The memory may include at least one type of storage medium, for example, it may include flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic memory, magnetic disk, optical disc, and so on. The memory is any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 602 in the embodiments of the present application may also be a circuit or any other device capable of implementing a storage function, for storing program instructions and / or data.

[0152] Those of ordinary skill in the art can understand that all or part of the steps for implementing the above method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps including the above method embodiments; the above computer storage medium can be any available medium or data storage device accessible by a computer, including but not limited to: removable storage devices, random access memory (RAM), magnetic memory (such as floppy disks, hard disks, magnetic tapes, magneto-optical discs (MO), etc.), optical memory (such as CDs, DVDs, BDs, HVDs, etc.), and semiconductor memories (such as ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid state drives (SSD)), and other various media that can store program code.

[0153] Alternatively, if the above integrated units of the present application are implemented in the form of software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiments of the present application, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods of the various embodiments of the present application. The aforementioned storage medium includes: removable storage devices, random access memory (RAM), magnetic memories (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical memories (such as CDs, DVDs, BDs, HVDs, etc.), and semiconductor memories (such as ROM, EPROM, EEPROM, non-volatile memories (NAND FLASH), solid-state drives (SSD)), etc., various media that can store program codes.

[0154] Based on the same inventive concept, the embodiments of the present application also provide a computer program product, which includes: computer program code. When the computer program code runs on a computer, it causes the computer to execute the data reading method of the hard disk as described in any of the foregoing discussions. Since the principle of the above computer program product for solving problems is similar to that of the data reading method of the hard disk, the implementation of the above computer program product can refer to the implementation of the method, and the repeated parts will not be elaborated.

[0155] The above embodiments are only used to introduce the technical solution of the present application in detail, but the description of the above embodiments is only for helping to understand the method of the embodiments of the present application and should not be construed as a limitation of the embodiments of the present application. Any changes or substitutions that can be easily thought of by those skilled in the art should be covered within the protection scope of the embodiments of the present application.

Claims

1. A resource scheduling method, characterized in that, Including: Determine that the idle resources do not meet the running requirements of the current job; wherein, the idle resources are the idle resources of the target heterogeneous partition indicated by the scheduling request, and the scheduling request is generated according to the current job; Calculate a first node quantity threshold of a first type of resources required to execute the current job according to the scheduling parameters carried in the scheduling request; wherein, the first type of resources is any one of various resources required to execute the current job; Compare the first node quantity threshold with the number of nodes of the first type of resources in the idle resources to determine the resource reservation policy for the first type of resources; wherein, the reserved resources indicated by the resource reservation policy are used to execute the current job.

2. The method according to claim 1, characterized in that, The comparing the first node quantity threshold with the number of nodes of the first type of resources in the idle resources to determine the resource reservation policy for the first type of resources includes: If the number of nodes of the first type of resources in the idle resources is less than the first node quantity threshold, then reserve all of the first type of resources in the idle resources; If the number of nodes of the first type of resources in the idle resources is greater than or equal to the first node quantity threshold, then reserve the first type of resources in the idle resources according to the first node quantity threshold.

3. The method according to claim 1, characterized in that, The scheduling parameters include the requested number of nodes and the requested number of CPU cores; the first type of resources is CPU cores; The calculating a first node quantity threshold of a first type of resources required to execute the current job according to the scheduling parameters carried in the scheduling request includes: According to the specifications of the CPU cores deployed on each node, when determining that the number of CPU cores that meet the request and the required number of nodes is not greater than the requested number of nodes, the maximum number of nodes required to execute the current job is the first node quantity threshold.

4. The method according to claim 3, wherein The specifications of the CPU cores include a first specification and a second specification; The according to the specifications of the CPU cores, when determining that the number of CPU cores that meet the request and the required number of nodes is not greater than the requested number of nodes, the maximum number of nodes required to execute the current job is the first node quantity threshold includes: Calculate respectively the number of nodes of the first specification of CPU cores and the number of nodes of the second specification of CPU cores required to execute the current job when the number of CPU cores that meet the request and the required number of nodes is not greater than the requested number of nodes; Wherein, the number of nodes of the first specification of CPU cores is the first node quantity threshold when the first type of resources is the first specification of CPU cores, and the number of nodes of the second specification of CPU cores is the first node quantity threshold when the first type of resources is the second specification of CPU cores.

5. The method according to claim 4, wherein The scheduling parameters include the target memory occupied by one CPU core, and the method further includes: Determine the specification of the CPU cores deployed on the first node according to the relationship between the target memory and the memory of the first node; wherein, the first node is any one node.

6. The method according to claim 1, characterized in that The scheduling parameters include the requested number of nodes, the type and quantity of the requested heterogeneous cards; the first type of resources is the corresponding type of heterogeneous cards; Calculating a first node quantity threshold of a first type of resources required to execute the current job according to scheduling parameters carried in the scheduling request includes: Determining, according to types and quantities of heterogeneous cards deployed on each node, that when types and quantities of heterogeneous cards satisfying the request are met and the required number of nodes is not greater than the number of nodes in the request, the maximum number of nodes required to execute the current job is the first node quantity threshold.

7. The method according to any one of claims 1 to 6, characterized in that, After determining the resource reservation policy for the first type of resources, the method further includes: Fusing resource reservation policies of various types of resources to determine reserved resources; Applying the reserved resources to execute the current job.

8. A resource scheduling device, characterized in that, Including: A data processing unit, configured to: determine that idle resources do not meet the running requirements of the current job; wherein the idle resources are idle resources of a target heterogeneous partition indicated by a scheduling request, and the scheduling request is generated according to the current job; A data calculation unit, configured to: calculate a first node quantity threshold of a first type of resources required to execute the current job according to scheduling parameters carried in the scheduling request; wherein the first type of resources is any one of various types of resources required to execute the current job; The data processing unit is further configured to: compare the first node quantity threshold with the number of nodes of the first type of resources in the idle resources to determine the resource reservation policy for the first type of resources; wherein the reserved resources indicated by the resource reservation policy are used to execute the current job.

9. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images