Method and apparatus for determining resource scheduling policy, storage medium, electronic device and program product

Abstract

Embodiments of the present application provide a method and device for determining a resource scheduling strategy, a storage medium, an electronic device and a program product, wherein the method comprises: obtaining a to-be-processed job submitted by a client to a scheduling system; determining a job type of the to-be-processed job and a resource requirement quantity of a required computing resource of the to-be-processed job; determining resource scheduled data of a plurality of computing resources in a computing node, and determining a resource scheduling strategy corresponding to the to-be-processed job based on the resource scheduled data, the job type and the resource requirement quantity, wherein the resource scheduled data is used to represent a usage of the plurality of computing resources, and the resource scheduling strategy is used to represent a rule of scheduling the required computing resource of the to-be-processed job from the computing node. Through the present application, the technical problem of low scheduling efficiency of computing resources is solved.

Description

Technical Field

[0001] This application relates to the field of high-performance computing clusters, and more specifically, to a method, apparatus, storage medium, electronic device, and program product for determining a resource scheduling strategy. Background Technology

[0002] Currently, in the field of high-performance computing, especially when it comes to tasks involving large-scale scientific computing and deep learning, cluster resource scheduling faces severe challenges.

[0003] In related technologies, elastic scaling and job preemption of job resources are usually handled through simple job suspension or restart mechanisms. Once a job is preempted, it is either completely suspended or terminated directly, resulting in the loss of existing computation progress. However, the above methods only manage resources in a coarse-grained manner and fail to delve into the details of process states, thus resulting in the technical problem of low scheduling efficiency of computing resources. Summary of the Invention

[0004] This application provides a method, apparatus, storage medium, electronic device, and program product for determining resource scheduling strategies, in order to at least solve the problem of low efficiency in recycling internal objects in related technologies.

[0005] According to one embodiment of this application, a method for determining a resource scheduling strategy is provided. The method may include: obtaining a pending job submitted by a client to a scheduling system; determining the job type of the pending job and the resource requirement of the computing resources required by the pending job; determining the resource scheduling data of multiple computing resources in a computing node, and determining the resource scheduling strategy corresponding to the pending job based on the resource scheduling data, the job type, and the resource requirement, wherein the resource scheduling data is used to represent the usage of multiple computing resources, and the resource scheduling strategy is used to represent the rules for scheduling the computing resources required by the pending job from the computing node.

[0006] In an exemplary embodiment, determining the resource scheduling strategy corresponding to the job to be processed based on resource-scheduled data, job type, and resource requirement quantity includes: determining the number of schedulable resources in the computing node based on resource-scheduled data, wherein the number of schedulable resources is used to characterize the number of computing resources that can be scheduled among multiple computing resources; and determining the resource scheduling strategy based on job type, resource requirement quantity, and schedulable resource quantity.

[0007] In one exemplary embodiment, determining a resource scheduling strategy based on job type, resource demand quantity, and schedulable resource quantity includes: comparing resource demand quantity and schedulable resource quantity to obtain a comparison result; and determining a scheduling strategy based on the comparison result and job type.

[0008] In an exemplary embodiment, determining a scheduling strategy based on the comparison result and the job type includes: in response to the job type being a non-real-time job type and the comparison result being that the number of schedulable resources is greater than or equal to the number of resource requirements, determining the resource scheduling strategy as: using the allowed schedulable computing resources to execute the pending job.

[0009] In an exemplary embodiment, determining a scheduling strategy based on a comparison result and a job type includes: in response to the job type being a non-real-time job type and the comparison result indicating that the number of schedulable resources is less than the resource requirement, determining whether shared resources exist in the computing node; in response to the existence of shared resources in the computing node, determining the sum of the number of shared resources and schedulable resources, and in response to the sum of the number of shared resources and schedulable resources being greater than or equal to the resource requirement, determining the resource scheduling strategy as: using the schedulable computing resources and shared resources to execute the pending job; in response to the absence of shared resources in the computing node, or the sum of the number of shared resources and schedulable resources being less than the resource requirement, determining resources to be occupied; and based on the number of resources to be occupied, determining the resource scheduling strategy as: using the schedulable computing resources, shared resources, and resources to be occupied to execute the pending job.

[0010] In one exemplary embodiment, determining the resource to be occupied includes: determining at least one job being processed among a plurality of computing resources based on resource scheduling data; determining at least one target job from the at least one job, wherein the priority of the target job is lower than the priority of the job to be processed; and determining the computing resource corresponding to the target job as the resource to be occupied.

[0011] In one exemplary embodiment, the method may further include: in response to the computing resources of a job being occupied by a job to be processed, creating a process snapshot for the target job, wherein the process snapshot is used to save the execution state of the computing resources before they were occupied.

[0012] In an exemplary embodiment, determining a scheduling strategy based on the comparison result and the job type includes: in response to the job type being a real-time job type and the comparison result being that the number of schedulable resources is greater than or equal to the number of resource requirements, determining the resource scheduling strategy as: using the allowed schedulable computing resources to execute the pending job.

[0013] In an exemplary embodiment, determining a scheduling strategy based on a comparison result and a job type includes: in response to the job type being a real-time job type and the comparison result being that the number of schedulable resources is less than the number of resource requirements, determining whether there are any available computing resources among the scheduled computing resources that can be occupied; and in response to the existence of available computing resources, determining the scheduling strategy as: using the schedulable computing resources and available computing resources to execute the pending job.

[0014] In one exemplary embodiment, the method may further include: sending an occupancy request to a proxy process according to a resource scheduling policy, wherein the occupancy request includes identity information of the available computing resources; obtaining a successful occupancy instruction returned by the proxy process; and in response to the successful occupancy instruction, saving the execution status of the job processes in the available computing resources.

[0015] In one exemplary embodiment, the method may further include: executing a pending job according to a resource scheduling policy; releasing the computing resources occupied by the pending job in response to the completion of the pending job execution; and restoring historical job processes in the available computing resources, wherein the historical job processes are used to characterize the execution status of jobs executed on the available computing resources before they were occupied.

[0016] According to another embodiment of this application, a resource scheduling strategy determination apparatus is also provided. The apparatus may include: an acquisition unit for acquiring pending jobs submitted by a client to a scheduling system; a first determination unit for determining the job type of the pending job and the resource requirement of the computing resources required by the pending job; and a second determination unit for determining the resource scheduling data of multiple computing resources in a computing node, and determining the resource scheduling strategy corresponding to the pending job based on the resource scheduling data, the job type, and the resource requirement, wherein the resource scheduling data is used to represent the usage of multiple computing resources, and the resource scheduling strategy is used to represent the rules for scheduling the computing resources required by the pending job from the computing node.

[0017] According to yet another embodiment of this application, a computer-readable storage medium is also provided, in which a computer program is stored, wherein the computer program is configured to perform the steps in any of the above method embodiments when it is run.

[0018] According to yet another embodiment of this application, an electronic device is also provided, including a memory and a processor, wherein a computer program is stored in the memory and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0019] According to yet another embodiment of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0020] This application obtains pending jobs submitted by clients to the scheduling system; determines the job type and resource requirements of the pending jobs; determines the resource scheduling data of multiple computing resources in computing nodes; and, based on the resource scheduling data, job type, and resource requirements, determines the resource scheduling strategy corresponding to the pending jobs. The resource scheduling data represents the usage of multiple computing resources, and the resource scheduling strategy represents the rules for scheduling the computing resources required for the pending jobs from the computing nodes. In other words, in this embodiment, after obtaining the pending jobs, the job type and resource requirements for executing the pending jobs can be determined, and the resource scheduling data of the computing resources can be determined. Based on the scheduled data, job type, and resource requirements, the resource scheduling strategy corresponding to the pending jobs can be determined, thereby solving the technical problem of low scheduling efficiency of computing resources and achieving the technical effect of improving the scheduling efficiency of computing resources. Attached Figure Description

[0021] Figure 1 This is a hardware structure block diagram of a server device for a method of determining a resource scheduling strategy according to an embodiment of this application.

[0022] Figure 2 This is a flowchart of a method for determining a resource scheduling strategy according to an embodiment of this application;

[0023] Figure 3 This is a schematic diagram of a snapshot of the creation process according to an embodiment of this application;

[0024] Figure 4 This is a structural block diagram of a resource scheduling strategy determination device according to an embodiment of this application;

[0025] Figure 5 This is a computer system architecture block diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0026] The embodiments of this application will be described in detail below with reference to the accompanying drawings and examples.

[0027] It should be noted that the terms "first," "second," etc., in the specification, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0028] First, some nouns or terms that appear in the description of the embodiments of this application shall be interpreted as follows:

[0029] The cluster manager and job scheduling system (Simple Linux Utility for Resource Management, or Slurm for short) can be a highly scalable and fault-tolerant cluster manager and job scheduling system that can be used for large-scale compute node clusters;

[0030] Job: In a job scheduling system, a job can refer to a task or a group of tasks that requires computing resources (such as CPU, memory, disk space, etc.) and time to execute. It can consist of one or more processes that execute on computing nodes and share resources. Jobs can be of various types, such as scientific simulation, data analysis, machine learning, etc. In the Slurm job scheduling system, after a user submits a job, Slurm can schedule the job to be executed on a suitable computing node based on the job requirements and the availability of system resources. Users can submit and manage jobs through Slurm command-line tools (such as sbatch, srun, etc.).

[0031] Real-time jobs are tasks that need to be completed within a specific timeframe, otherwise serious consequences may result. Examples include financial market simulations and real-time data analysis. Real-time jobs are typically scheduled with priority over non-real-time jobs.

[0032] Non-real-time jobs are tasks without strict deadlines and can be completed at any time. Examples include scientific simulations and data analysis. The scheduling of these non-real-time jobs is typically based on resource availability and priority.

[0033] Preemption, in high-performance computing (HPC) scheduling systems, is a mechanism that allows the system to preempt running jobs when needed. It can be used to optimize resource utilization and improve job completion speed, and can occur between jobs, nodes, or computing resources. When a high-priority job is submitted to the system, the scheduling system may preempt one or more running low-priority jobs to allocate resources for the newly submitted job. The preempted job will be suspended and rescheduled when resources become available.

[0034] The Compute Unified Device Architecture (CUDA) is a general-purpose parallel computing architecture that enables GPUs to solve complex computational problems. It can include the CUDA instruction set architecture (ISA) and the parallel computing engine inside the GPU.

[0035] The CUDA library is a database containing a set of pre-compiled functions and classes, which can help developers more easily develop parallel computing applications on the CUDA platform.

[0036] Parallel computing (CUDA kernel) functions refer to functions in CUDA programs used to perform parallel computing on the GPU. A CUDA kernel function is a function designed to execute in parallel on multiple threads, which are organized into a specific execution model, such as thread blocks, grids, and multidimensional indexes. CUDA kernel functions can be declared using special function call syntax (such as __global__) and launched on the GPU by calling the CUDA Application Programming Interface (API).

[0037] A software tool (checkpoint / restore in userspace, abbreviated as CRU) is a tool that runs on the operating system and can perform checkpoint / restore functions in user space. Using this tool, you can freeze a running program and checkpoint to a series of files associated with the program. Then, you can use these files to restore the program to the point when it was frozen on any host. In other words, it is a backup and restore of the running program environment.

[0038] This embodiment also provides a method for determining a resource scheduling strategy. This system is used to implement the embodiments and preferred embodiments, and details already described will not be repeated. As used below, the terms "module" and "unit" refer to a combination of software and / or hardware capable of performing a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0039] As an optional implementation, the method embodiments provided in this application can be executed on a server device or a similar computing device. Taking running on a server device as an example, Figure 1 This is a hardware structure block diagram of a server device for a method of determining a resource scheduling strategy according to an embodiment of this application. For example... Figure 1 As shown, the server device may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The server device may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1The structure shown is for illustrative purposes only and does not limit the structure of the server equipment described above. For example, the server equipment may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0040] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the resource scheduling strategy determination method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, thus implementing the above-described method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to server devices via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0041] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider for the server device. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0042] This embodiment provides a method for determining a resource scheduling strategy. Figure 2 This is a flowchart of a method for determining a resource scheduling strategy according to an embodiment of this application, such as... Figure 2 As shown, the method may include the following steps:

[0043] Step S202: Obtain the pending jobs submitted by the client to the scheduling system;

[0044] Step S204: Determine the job type of the job to be processed and the resource requirements of the computing resources required by the job to be processed.

[0045] Step S206: Determine the resource scheduling data of multiple computing resources in the computing node, and determine the resource scheduling strategy corresponding to the job to be processed based on the resource scheduling data, job type and resource requirement. The resource scheduling data is used to represent the usage of multiple computing resources, and the resource scheduling strategy is used to represent the rules for scheduling the computing resources required for the job to be processed from the computing node.

[0046] In this embodiment, the aforementioned pending jobs can be computational tasks (hereinafter referred to as tasks) submitted by users to the scheduling system, awaiting resource allocation and execution. These tasks can be any program or data processing task requiring computational resources (such as CPU, GPU, memory, etc.). For example, the pending job could be a user-submitted deep learning model training job, which requires 8 CPU cores and 2 GPUs for training. The job types can include real-time and non-real-time job types. Real-time jobs have strict time response requirements and are typically high-priority, used in applications requiring immediate feedback. Non-real-time jobs do not have such time constraints, can tolerate latency or interruptions, and are typically low-priority. The resource requirements can be used to determine the amount of resources needed to process pending jobs. The computational resources can refer to the hardware devices used to execute jobs in a high-performance computing cluster, including but not limited to CPUs, GPUs, storage devices (such as hard drives and SSDs), and network resources. The resource scheduling strategy described above can be used to represent the rules for scheduling the computing resources required for pending jobs from computing nodes. It can refer to the rules and logic by which the scheduling system determines how to allocate and manage computing resources based on the current resource status, job type, and job requirements. It can be used to optimize resource utilization efficiency and meet job execution needs, and may include real-time job preemption strategies or resource sharing strategies. For example, the resource scheduling strategy described above can be used to determine the usage and scheduling status of computing resources at different locations within a computing node. It should be noted that this is only an illustrative example, and no specific limitations are imposed on the types of pending jobs, computing resources, or the content of the resource scheduling strategy described above.

[0047] Optionally, the scheduling system can categorize pending jobs into two types: real-time jobs and non-real-time jobs. Real-time jobs have dedicated CPU time and do not share CPU resources with other jobs; non-real-time jobs can share CPU resources. When a user submits a pending job to the scheduling system, the system can determine in advance whether the job is real-time or non-real-time and the amount of CPU and GPU resources required. The above steps allow the system to obtain the job type and the required computing resources.

[0048] Optionally, the scheduling system can receive job requests submitted by users or automated tasks. These job requests may include a job description, the required resource type (e.g., CPU, GPU), and quantity. Based on this request, jobs to be processed can be obtained. By analyzing the jobs to be processed, the job type and the required quantity of computing resources can be determined. At this point, the resource scheduling data of the computing resources in the computing nodes needs to be confirmed to determine the available computing resources in the computing nodes. Further, the scheduling system considers the current resource usage on each computing node (i.e., resource scheduling data). Based on the scheduled data, job type, and resource requirement quantity, the scheduling system can determine how to allocate resources to the jobs to be processed to obtain a resource scheduling strategy. For example, if a real-time job requests resources, the scheduling system will prioritize preempting resources from non-real-time jobs.

[0049] Optionally, the above-mentioned job types can be divided into real-time jobs and non-real-time jobs, which can be classified according to the nature and requirements of the job. The resource requirements mentioned above can refer to the specific quantity and type of resources needed to complete the job, such as the number of CPU cores, the number of GPUs, and their models. It should be noted that this is only an illustrative example, and there are no specific restrictions on the types of resource requirements.

[0050] For example, suppose in an HPC cluster, non-real-time job 1 is using 5 CPU cores and 1 GPU on node 1, while real-time job 2 requires 10 CPU cores and 1 GPU to run. The scheduling system receives the submission of real-time job 2. Analyzing job 2, the scheduling system determines that it is a real-time job requiring 10 CPU cores and 1 GPU. The scheduling system can then check the resource scheduling data and find the resource usage of non-real-time job 1 on node 1. The scheduling system can decide that, since job 2 is a real-time job, it will pause or scale down job 1, releasing the GPU and some CPU resources used by job 1 on node 1 to meet the requirements of job 2. For example, job 1 can retain 2 CPU cores to continue running, while job 2 can use the remaining 8 CPU cores and the GPU.

[0051] In this embodiment, by allowing non-real-time jobs to share CPU resources, resource idleness can be avoided, improving the overall efficiency of the cluster. The resource requirements of real-time jobs are prioritized to ensure their completion within the specified time. The job elastic scaling scheme allows for dynamic resource adjustment, enabling the scheduling system to respond more flexibly to fluctuations in resource demand, improving the cluster's adaptability and responsiveness. More efficient resource management and scheduling can reduce unnecessary resource waste and lower operating costs. Even under resource constraints, non-real-time jobs can continue to run, avoiding complete job cancellation and improving job completion rates and user satisfaction.

[0052] Optionally, through the resource scheduling strategy described above, the cluster can ensure that real-time jobs receive resources first, meeting time-sensitive computing needs and improving the real-time performance of data processing. Through these steps, resources allocated to jobs can be dynamically adjusted according to job requirements, improving resource utilization efficiency and reducing waste. This method allows for rapid response to changes in computing resource demands, effectively handling sudden tasks, improving the overall computing power of the cluster, and enhancing cluster flexibility.

[0053] Optionally, intelligent scheduling algorithms can maximize the utilization of all computing resources in the cluster, improving the overall performance and economic efficiency of the cluster. Even under resource constraints, it ensures that user-submitted jobs have a chance to be executed, reducing waiting time and job failure rates.

[0054] By using the above method, after obtaining the job to be processed, we can determine the job type and the amount of resources required to execute the job, and determine the resource scheduling data in the computing resources. Based on the scheduled data, job type and resource requirement, we can determine the resource scheduling strategy corresponding to the job to be processed, thereby solving the technical problem of low scheduling efficiency of computing resources and achieving the technical effect of improving the scheduling efficiency of computing resources.

[0055] In an exemplary embodiment, step S206, determining the resource scheduling strategy corresponding to the job to be processed based on the resource-scheduled data, job type, and resource requirement quantity, includes: determining the number of schedulable resources in the computing node based on the resource-scheduled data, wherein the number of schedulable resources is used to characterize the number of computing resources that can be scheduled among multiple computing resources; and determining the resource scheduling strategy based on the job type, resource requirement quantity, and schedulable resource quantity.

[0056] In this embodiment, resource scheduling data is obtained, and the number of schedulable resources in the computing node can be determined based on the resource scheduling data. The number of schedulable resources can be used to characterize the number of computing resources that can be scheduled among multiple computing resources. Furthermore, based on the job type, the number of resource requirements, and the number of schedulable resources, a resource scheduling strategy can be determined.

[0057] Optionally, the scheduling system determines the resource scheduling strategy for the jobs to be processed based on the resource-already-scheduled data, job type, and resource requirement quantity. This process can be divided into two sub-steps: determining the number of schedulable resources in the computing node based on the resource-already-scheduled data, and further determining the resource scheduling strategy based on the job type, resource requirement quantity, and schedulable resource quantity.

[0058] Optionally, the aforementioned resource scheduling data includes the usage of various computing resources (such as CPU and GPU) on the current computing node. By analyzing this data, the scheduling system can calculate which resources are idle, which resources can be further utilized, and which resources are occupied by existing jobs but may be partially released. The number of schedulable resources reflects the amount of resources that a computing node can provide for jobs to be processed based on existing jobs. Therefore, based on this, the scheduling system can determine how to allocate computing resources (which can be simply referred to as resources) in the computing node according to the job type (real-time or non-real-time), the amount of resources required by the job, and the number of schedulable resources on the computing node. For real-time jobs, the scheduling system tends to exclusively occupy resources and prioritizes preemption mechanisms; for non-real-time jobs, the scheduling system will more flexibly adopt resource sharing or partial resource allocation strategies.

[0059] For example, suppose node one has 16 CPU cores, of which 8 CPU cores are used by non-real-time job two, and job two only uses 60% of the CPU core performance. Meanwhile, the GPU resources on node one are not occupied by any job. The scheduling system analyzes the resource scheduling data and determines that the amount of resources available on node one for real-time job one is the remaining 8 CPU cores and 1 GPU. Additionally, it can consider temporarily sharing some CPU cores (e.g., 2) from job two to meet job one's needs. Further, it can be determined that real-time job one requires 10 CPU cores and 1 GPU. After analyzing the schedulable resources on node one, the scheduling system finds that the resources directly meeting job one's needs are insufficient. Therefore, the scheduling system formulates the following resource scheduling strategy: Real-time preemption strategy: The scheduling system decides to suspend or scale down non-real-time job two, releasing some of the CPU resources occupied by job two to meet job one's resource needs. For example, job one can use the remaining 8 CPU cores and the additional 2 CPU cores shared from job two, as well as the idle GPU resources on node one. Dynamic resource allocation strategy: If the resource requirements of job one cannot be fully met, the scheduling system can consider dynamically searching for or borrowing resources from other non-real-time jobs or nodes until the minimum resource requirements of job one are met, so as to ensure the timely execution of job one.

[0060] In summary, through the steps outlined above, different resource scheduling strategies are provided for different job types. This precise resource scheduling allows for more efficient utilization of resources on computing nodes, avoiding resource idleness or over-allocation. Furthermore, the priority of real-time jobs is guaranteed, ensuring their completion in the shortest possible time to meet time-sensitive requirements. Non-real-time jobs can continue execution under resource constraints through partial resource allocation or resource sharing mechanisms, avoiding complete waiting or task cancellation and improving job completion rates. Effective resource management and scheduling strategies reduce resource waste and help control cluster operating costs. Even in highly resource-constrained environments, jobs are ensured to be rationally scheduled and executed, improving the system's support for multiple users and multiple tasks, and enhancing cluster availability and user satisfaction.

[0061] Through the above steps, the scheduling system can achieve intelligent management of resources, dynamically adjust resource allocation according to job characteristics and resource status, and achieve the goals of optimizing resource use, improving cluster efficiency, and meeting different job requirements.

[0062] In one exemplary embodiment, determining a resource scheduling strategy based on job type, resource demand quantity, and schedulable resource quantity includes: comparing resource demand quantity and schedulable resource quantity to obtain a comparison result; and determining a scheduling strategy based on the comparison result and job type.

[0063] In this embodiment, the resource demand and the number of schedulable resources are compared to obtain a comparison result. This comparison result can be used to characterize the difference between the resource demand and the number of schedulable resources. Based on the comparison result, it can be determined whether the number of schedulable resources in the current computing node can meet the requirements of the job to be processed. Furthermore, based on the comparison result and the job type, a scheduling strategy can be determined.

[0064] Optionally, the scheduling system first needs to assess whether the demands of the pending job can be met by the currently available schedulable resources. This involves comparing the resources required by the job (such as the number of CPU cores and GPUs) with the available unallocated resources on the current node to obtain a comparison result. This comparison result can be used to guide the scheduling system in taking action. If the resource demand is less than or equal to the number of available schedulable resources, the scheduling system can directly allocate resources to start the job. If the resource demand is greater than the number of available schedulable resources, the scheduling system can decide, based on the job type (real-time or non-real-time), whether to preempt resources from non-real-time jobs, share CPU resources, or wait for resources to become available to meet the resource demand.

[0065] In summary, through intelligent comparison and decision-making in the above steps, the scheduling system can maximize the utilization of cluster computing resources and avoid resource idleness and waste. The scheduling strategy described above can be flexibly adjusted according to changes in job type and resource requirements, improving the cluster's adaptability to different loads.

[0066] Optionally, through the above steps, even when resources are scarce, user-submitted non-real-time jobs can continue to execute in some form, reducing job waiting time and failure rate, and improving user satisfaction with cluster resource allocation. Dynamic resource scheduling strategies help reduce unnecessary resource allocation, avoid over-investing in hardware to cope with occasional peak demand, and thus control operating costs.

[0067] The following section further explains the processing procedure for non-real-time job types.

[0068] In an exemplary embodiment, determining a scheduling strategy based on the comparison result and the job type includes: in response to the job type being a non-real-time job type and the comparison result being that the number of schedulable resources is greater than or equal to the number of resource requirements, determining the resource scheduling strategy as: using the allowed schedulable computing resources to execute the pending job.

[0069] In an exemplary embodiment, determining a scheduling strategy based on a comparison result and a job type includes: in response to the job type being a non-real-time job type and the comparison result indicating that the number of schedulable resources is less than the resource requirement, determining whether shared resources exist in the computing node; in response to the existence of shared resources in the computing node, determining the sum of the number of shared resources and schedulable resources, and in response to the sum of the number of shared resources and schedulable resources being greater than or equal to the resource requirement, determining the resource scheduling strategy as: using the schedulable computing resources and shared resources to execute the pending job; in response to the absence of shared resources in the computing node, or the sum of the number of shared resources and schedulable resources being less than the resource requirement, determining resources to be occupied; and based on the number of resources to be occupied, determining the resource scheduling strategy as: using the schedulable computing resources, shared resources, and resources to be occupied to execute the pending job.

[0070] In one exemplary embodiment, determining the resource to be occupied includes: determining at least one job being processed among a plurality of computing resources based on resource scheduling data; determining at least one target job from the at least one job, wherein the priority of the target job is lower than the priority of the job to be processed; and determining the computing resource corresponding to the target job as the resource to be occupied.

[0071] In one exemplary embodiment, the method may further include: in response to the computing resources of a job being occupied by a job to be processed, creating a process snapshot for the target job, wherein the process snapshot is used to save the execution state of the computing resources before they were occupied.

[0072] In this embodiment, it is determined whether the job type is a non-real-time job type. If the job type is a non-real-time job type and the comparison result is the number of schedulable resources and the number of resource requirements, then it can be determined that the current number of computing resources can meet the needs of the job to be processed. Therefore, the scheduling strategy can be determined to use the allowed schedulable computing resources to execute the job to be processed.

[0073] In this embodiment, if the comparison result shows that the number of schedulable resources is less than the resource requirement, it can be determined that the number of currently allowed schedulable computing resources cannot meet the needs of the pending job. Further, it can be determined whether there are shared resources in the computing node, which can be shared resources of other non-real-time jobs. If there are shared resources in the computing node, it can be determined whether the sum of the shared resources and the schedulable resources meets the required resource quantity. If the sum meets the required resource quantity, the scheduling strategy can be determined as using the shared resources and schedulable resources to execute the pending task. If the sum of the shared resources and the schedulable resources does not meet the required resource quantity, it can be further determined whether there are resources to be occupied in the computing node. These resources can be resources allowed to be occupied in the computing node, or resources currently being used by non-real-time jobs with a lower priority than the pending job.

[0074] Optionally, after determining the resources to be occupied, the tasks to be processed can be handled among schedulable resources, resources to be occupied, and shared resources. The resources to be occupied can be computing resources allowed to be used within a computing node. The schedulable resources can be idle resources within a computing node that are allowed to be invoked.

[0075] In this embodiment, the resource to be occupied can be determined through the following steps: based on the resource scheduling data, at least one job being processed among multiple computing resources can be determined, and at least one target job can be determined from the at least one job, wherein the target job can be a non-real-time job with a lower priority than the job to be processed, and the computing resources used by the target job can be determined as the resource to be occupied.

[0076] In this embodiment, a process snapshot can be created for the target job before occupying the resource to be occupied. This process snapshot can be used to save the execution state of the computing resource before it is occupied, and can be constructed using a CUDA agent, and may include the context information of the target job.

[0077] Optionally, the scheduling system can categorize pending jobs into two types: real-time jobs and non-real-time jobs. Real-time jobs have exclusive access to the CPU and do not share CPU resources with other jobs; non-real-time jobs can share CPU resources. Users can submit pending jobs to the scheduling system, specifying whether the job is real-time or non-real-time, and the amount of CPU and GPU resources required. For non-real-time jobs, after submission, the scheduling system checks for available CPU and GPU resources (i.e., computing resources). If available, the job is immediately scheduled for execution; if insufficient resources are available, the system considers whether it can share resources with other non-real-time jobs to achieve execution. If so, the job is run by time-sharing CPU resources with other non-real-time jobs.

[0078] Optionally, during job preemption, if only CRIU is used to save and restore the job process, there is a problem that the context state of the GPU being used by the job cannot be saved. Therefore, in this embodiment, for jobs using GPUs for CUDA computation, the saving and restoration of the GPU context can be accomplished through a CUDA proxy. That is, when the job process calls the CUDA API, it does not directly call the CUDA library, but instead calls the CUDA library through the CUDA proxy. The CUDA proxy can include a dynamic link library (hereinafter referred to as the proxy library) and a proxy process. The dynamic link library can be used to control the job process's calls to the CUDA API. The proxy process, through which the proxy library can forward the hijacked calls to the CUDA API from the job process, ultimately uses the CUDA library to issue requests.

[0079] For example, when a job process creates a CUDA context, the proxy library intercepts the request and forwards it to the proxy process. The proxy process can then call the CUDA library to create a CUDA context. Subsequently, when the job process calls the CUDA API, the proxy library will intercept and forward the request to the proxy process. The proxy process will then use this context to forward the job process's API calls to the CUDA library. Figure 3 This is a schematic diagram of a snapshot of the creation process according to an embodiment of this application, such as... Figure 3As shown, when job process 301 creates a CUDA context, proxy library 302 intercepts the request and forwards it to proxy process 303, which can then call the CUDA library to create the CUDA context. When job process 304 creates a CUDA context, proxy library 305 intercepts the request and forwards it to proxy process 303, which can then call the CUDA library to create the CUDA context. The created context can then be transferred to image processor 306.

[0080] Optionally, for jobs that use GPUs for CUDA computation, the CUDA dynamic link library that the job process depends on is not the CUDA library that is provided for use, but a CUDA proxy link library; and a proxy process also needs to be started at the same time as the job process.

[0081] Alternatively, to conserve resources, only one agent process can be started on each compute node (physical machine), and all job processes running on that node can use this agent process. The agent process can establish an independent CUDA context for each proxied job process.

[0082] Optionally, when creating a snapshot of the job process and saving the GPU state of the job process, the scheduling system can send instructions to the agent process; the agent link library can also forward CUDA APL to the agent process, which requires communication with the agent process. Therefore, the agent process can listen to a certain port of the host after it starts and communicate with the agent process by sending requests to that port.

[0083] Optionally, when the scheduling system starts the job process, it can set the port number that the agent process listens on to the job process's environment variables. When the agent library forwards the CUDA API to the agent process, it can obtain the port number that the agent process listens on through the environment variables and send a request to that port to complete the forwarding of the CUDA API request.

[0084] In this embodiment, when the job type is a non-real-time job type and the number of schedulable resources is greater than or equal to the number of resource requirements, the scheduling system can adopt a strategy of directly allocating resources.

[0085] Optionally, the scheduling system can also reserve computing resources for non-real-time jobs to ensure that computing resources are available when pending jobs start. At the same time, once a pending job is completed or paused, the resources are immediately released for use by other jobs to improve resource turnover.

[0086] Optionally, during the execution of the pending job, if other non-real-time jobs request resources, and the scheduling system detects that there are sufficient resources on the computing node to meet the needs of other non-real-time jobs, the scheduling system can dynamically adjust the resources to allocate additional resources for job Q without affecting the execution of the pending job, thereby achieving flexible reallocation of resources.

[0087] Optionally, when determining the resources to be occupied, the scheduling system may also consider the following factors: the job's running status and remaining workload to minimize the impact on the target job; the type of resource and its impact on job performance, for example, prioritizing CPU over GPU preemption because saving and restoring GPU state may be more complex and have a greater impact on the job; dynamic adjustment of shared resources, i.e., allowing running jobs to release some resources for other jobs to use when resources are scarce, without affecting critical performance; and a job recovery plan after resource preemption to ensure that the target job can be restored to its state before preemption after the resources are released, thereby reducing the loss and impact of job interruption. It should be noted that the above methods for determining the resources to be occupied are only illustrative examples, and no specific restrictions are imposed on the methods for determining the resources to be occupied here.

[0088] In summary, through the intelligent preemption mechanism, the system can ensure that the resource requirements of critical tasks (such as real-time tasks) are met, preventing important tasks from being delayed due to insufficient resources and achieving efficient resource utilization. Even under highly resource-constrained conditions, the scheduling system can still provide necessary resources for pending tasks through resource preemption, reducing job waiting time and job execution delays.

[0089] Optionally, the aforementioned resource preemption strategy ensures the effective allocation and utilization of resources, reducing job failures due to insufficient resources and thus improving job success rates. The scheduling system can make intelligent resource preemption decisions based on job priority, resource requirements, and current resource usage, avoiding resource waste and unnecessary job interruptions. Simultaneously, by reducing job waiting time and increasing success rates, the scheduling system enhances cluster availability and efficiency, optimizing the user experience for submitting and executing jobs.

[0090] In this embodiment, the above steps not only provide a flexible and scalable resource scheduling strategy but also enable intelligent saving and recovery of job states, greatly enhancing the resource scheduling capabilities and job execution continuity of the high-performance computing cluster, providing users with a more efficient, flexible, and secure computing environment. Furthermore, the scheduling system, through these steps, can not only handle the discrepancy between resource demand and available resources but also ensure the smooth execution of all jobs in the cluster through an intelligent resource preemption mechanism, thereby achieving efficient resource utilization and optimizing the user experience.

[0091] The following section further explains the processing procedure for pending jobs of the real-time job type.

[0092] In an exemplary embodiment, determining a scheduling strategy based on the comparison result and the job type includes: in response to the job type being a real-time job type and the comparison result being that the number of schedulable resources is greater than or equal to the number of resource requirements, determining the resource scheduling strategy as: using the allowed schedulable computing resources to execute the pending job.

[0093] In an exemplary embodiment, determining a scheduling strategy based on a comparison result and a job type includes: in response to the job type being a real-time job type and the comparison result being that the number of schedulable resources is less than the number of resource requirements, determining whether there are any available computing resources among the scheduled computing resources that can be occupied; and in response to the existence of available computing resources, determining the scheduling strategy as: using the schedulable computing resources and available computing resources to execute the pending job.

[0094] In one exemplary embodiment, the method may further include: sending an occupancy request to a proxy process according to a resource scheduling policy, wherein the occupancy request includes identity information of the available computing resources; obtaining a successful occupancy instruction returned by the proxy process; and in response to the successful occupancy instruction, saving the execution status of the job processes in the available computing resources.

[0095] In this embodiment, if the job type is a real-time job and the comparison result shows that the number of schedulable resources is greater than or equal to the resource requirement, the resource scheduling strategy can be determined as directly using the schedulable computing resources to process the job. If the comparison result shows that the number of schedulable resources is less than the resource requirement, it can be determined whether there are any occupiable computing resources allowed in the computing node. In response to the existence of occupiable computing resources allowed in the computing node, the scheduling strategy can be determined as: using the schedulable computing resources and the occupiable computing resources to execute the job. Furthermore, the task to be executed can be processed on the corresponding computing resources according to the scheduling strategy.

[0096] Optionally, after determining the resource scheduling strategy, if the scheduling strategy is to use schedulable and available computing resources to execute pending jobs, a request for allocation can be sent to the agent process. This request may include the identity information of the person occupying the computing resources. A successful allocation instruction is returned by the agent process, and in response to this instruction, the execution status of the job processes within the available computing resources can be saved. That is, in this embodiment, the execution status of the job processes within the computing resources can be stored before the computing resources are allocated. Therefore, after the pending jobs are processed on the available computing resources, the stored job processes can be used to restore the previously processed data on the available computing resources.

[0097] Optionally, for real-time jobs, after a job is submitted, the scheduling system checks whether there are idle CPU or GPU resources in the computing nodes. If there are idle resources, the job can be scheduled to execute immediately. If there are not enough resources, it can consider whether sufficient resources can be obtained by preempting resources currently being used by non-real-time jobs. If sufficient resources can be obtained by preempting one or more non-real-time jobs, the non-real-time jobs can be preempted to release resources, and then the submitted real-time job can be started.

[0098] Optionally, the preemption process for computing resources for non-real-time jobs may include the following: The scheduling system connects to the agent process's port and sends a request to the agent process. Because the agent process can proxy multiple job processes, the request carries the job process's identity information, which can be used to identify the process that needs to be snapshotted. Upon receiving the request, the agent process checks if there is a CUDA kernel function currently executing. If not, the agent process directly returns success to the scheduling system; if there is a CUDA kernel function currently executing, it waits for it to complete, and after completion, the agent process directly returns success to the scheduling system. During this process, the agent program may receive CUDA requests forwarded by the proxy library, but the agent program does not process these CUDA requests forwarded by the proxy library; instead, it temporarily stores them in memory.

[0099] Optionally, after the scheduling system requests a successful response from the agent process, it executes a command provided by CRIU to save the state of the job process. CRIU can save the process's CPU state and memory to a file, typically a shared network file system across all compute nodes for easy recovery. Further, the scheduling system can send another request to the agent process, asking it to save any temporarily stored CUDA requests forwarded by the agent's libraries (if any). The agent process then saves these requests to a file (hereinafter referred to as the GPU snapshot file). These steps complete the saving of the preempted process's state, allowing the resources occupied by the preempted job process to be released and made available to the preempting process.

[0100] Regarding the processing of real-time jobs, the embodiments of the preceding claims provide two different strategies, determining the job scheduling strategy based on a comparison of resource requirements and the number of schedulable resources. Below, we will explain, illustrate, and expand upon these steps in detail, drawing upon the content of the technical disclosure, and demonstrate their beneficial effects.

[0101] Optionally, when the real-time job type is determined and the number of schedulable resources is greater than or equal to the resource requirement, the pending job can be executed directly using the allowed schedulable computing resources. However, when the real-time job type is determined and the number of schedulable resources is less than the resource requirement, the system first checks whether there are any available computing resources that can be preempted from the scheduled computing resources. If so, these available computing resources and the schedulable resources are used to execute the real-time job. However, before preempting the aforementioned available computing resources, the scheduling system can send a preemption request to the agent process. This request may contain the identity information of the computing resources preempted from the non-real-time job. This information is used to indicate to the agent process which resources need to be snapshotted. Further, after receiving the request, the agent process suspends the GPU context state of the non-real-time job, waits for the current CUDA kernel function to finish executing, and then, in cooperation with the CRIU tool, creates a snapshot of the non-real-time job and saves its state. After completion, the agent process returns a successful preemption instruction to the scheduling system. After receiving a successful acquisition instruction from the agent process, the scheduling system uses CRIU to save the execution status of non-real-time jobs, including CPU status and memory data, to ensure that non-real-time jobs can be restored to their state before being preempted after resources are acquired.

[0102] Optionally, to further improve the efficiency of resource management and scheduling, a specific resource pool can be reserved for real-time jobs. When a real-time job is submitted, resources are preferentially allocated from this resource pool, reducing the possibility of resource preemption. Simultaneously, the resource preemption strategy can be dynamically adjusted based on the cluster load. For example, resource preemption can be reduced when the load is low to avoid impacting non-real-time jobs. When resources become available again, non-real-time jobs that had been preempted can be prioritized for recovery to minimize the impact of job interruptions.

[0103] In summary, through the above steps, even under resource shortages, real-time jobs can acquire sufficient resources through resource preemption, ensuring job continuity and the elastic scaling capability of cluster resources. After non-real-time jobs are preempted, their state is preserved, avoiding resource waste. Simultaneously, the scheduling system can reallocate the released resources, improving overall resource utilization. This strategy ensures the priority of real-time jobs while also considering the continuity of non-real-time jobs, achieving a balance between fairness and efficiency in cluster scheduling.

[0104] In one exemplary embodiment, the method may further include: executing a pending job according to a resource scheduling policy; releasing the computing resources occupied by the pending job in response to the completion of the pending job execution; and restoring historical job processes in the available computing resources, wherein the historical job processes are used to characterize the execution status of jobs executed on the available computing resources before they were occupied.

[0105] In this embodiment, once the pending job has been completed, the computing resources occupied by the pending job can be released, and the historical job processes in the available computing resources can be restored. These historical job processes can be pre-saved job processes, which can be used to characterize the execution status of jobs executed before the available computing resources were occupied. This execution status can be used to determine the job's running result, progress, and other information.

[0106] Optionally, the scheduling system can execute CRIU commands to restore the preempted job process from a previously saved file.

[0107] Optionally, once a pending job completes, the resources it occupies are released, and the scheduling system can reschedule. At this point, it first checks if there are any real-time jobs to schedule. If there are no real-time jobs to schedule or the conditions for real-time jobs to run are not met, then non-real-time jobs are scheduled. If sufficient resources are available, previously saved preempted job processes can be restored and continue execution.

[0108] Optionally, when resuming a job process, the CPU selection logic is as follows: If there are enough idle CPUs on the node (idle CPUs refer to CPUs not being used by any job), then the computing resources for executing the pending job are selected from the idle CPUs. If there are not enough idle CPUs on the compute node (which can be simply referred to as a node), then sharing the CPU with other non-real-time jobs can be considered.

[0109] Alternatively, assuming the number of idle CPUs plus shared CPUs on a node is m, and the number of CPUs requested by the job is n, if m is less than n, then it is advisable to allocate m CPUs to the job for it to run. Doing so may result in poorer job performance, but it avoids wasting resources.

[0110] Optionally, the job process can be restored through the following steps: If no agent process is running on the compute node where the job process is to be restored, the scheduling system starts an agent process on that node. The scheduling system sends a request to the agent process, requesting it to restore the GPU context state, specifying a GPU snapshot file in the request. Upon receiving the request, the agent process calls the NVIDIA CUDA API to recreate a new CUDA context. If the GPU snapshot file contains previously saved but not yet executed CUDAKernel functions, these functions are executed in the newly created CUDA context. The agent process then returns a success message to the scheduling system.

[0111] Optionally, when the real-time job type is determined and the number of schedulable resources is greater than or equal to the resource requirement, the pending job can be executed directly using the allowed schedulable computing resources. However, when the real-time job type is determined and the number of schedulable resources is less than the resource requirement, firstly, it can be checked whether there are any available computing resources that can be occupied among the scheduled computing resources. If so, the real-time job can be executed using these available computing resources and the schedulable resources.

[0112] For example, before occupying the aforementioned available computing resources to execute a pending job, a request for allocation can be sent to the agent process. The scheduling system sends this request to the agent process, and the request can include the identity information of the four CPU cores and one GPU resource preempted from the non-real-time job. This information is used to instruct the agent process which resources need to be snapshotted. After receiving the request, the agent process can pause the GPU context state of the non-real-time job, wait for the current CUDA kernel function to finish executing, and then, in conjunction with the CRIU tool, create a snapshot of the aforementioned non-real-time job and save its state. After completion, the agent process can return a successful allocation instruction to the scheduling system. Furthermore, after receiving the successful allocation instruction from the agent process, the scheduling system uses CRIU to save the execution state of the aforementioned non-real-time job. This execution state can include CPU state and memory data, which can be used to ensure that the non-real-time job can be restored to its state before preemption after the resources are occupied.

[0113] In summary, by following the steps outlined above, the resource requirements of real-time jobs can be handled more intelligently and efficiently, while maintaining the flexibility and fairness of cluster resource scheduling, thus providing strong support for high-performance computing scenarios.

[0114] Optionally, even in situations of resource scarcity, real-time jobs can acquire sufficient resources through resource preemption, ensuring job continuity and the elastic scaling capability of cluster resources. After non-real-time jobs are preempted, their state is preserved, preventing resource waste. Simultaneously, the scheduling system can reallocate released resources, improving overall resource utilization.

[0115] Optionally, the defined resource scheduling strategy ensures the priority of real-time jobs while also taking into account the continuity of non-real-time jobs, thus achieving a balance between fairness and efficiency in cluster scheduling.

[0116] Optionally, the above method can be applied to the field of high-performance computing clusters, and can effectively support the shared use of GPU computing resources by multiple users in high-performance computing scenarios.

[0117] In this embodiment of the application, after obtaining the job to be processed, the job type and the amount of resources required to execute the job can be determined, and the resource scheduling data in the computing resources can be determined. Based on the scheduling data, job type and resource requirement, the resource scheduling strategy corresponding to the job to be processed can be determined, thereby solving the technical problem of low scheduling efficiency of computing resources and achieving the technical effect of improving the scheduling efficiency of computing resources.

[0118] To facilitate understanding of the implementation methods of this application, relevant scenarios are explained below, but these explanations do not limit the scope of this application.

[0119] Currently, high-performance computing plays a wide and important role in many industries such as life sciences, manufacturing simulation, chemical engineering, aerospace, materials, and meteorology. These fields generally involve a large amount of computation and require the computation tasks to be completed within a certain time. A single general-purpose server node cannot meet the requirements. Multiple high-performance servers are usually used and interconnected through a high-speed network to form a high-performance computing cluster, which processes large amounts of data in parallel at extremely high speeds.

[0120] For high-performance computing cluster job scheduling systems, such as Slurm, this system maintains a queue of pending user job scripts and manages the overall resource utilization of these jobs. It manages available compute node resources in a shared or non-shared manner for users to execute jobs. The system allocates resources appropriately to the job queue and monitors jobs until they are completed.

[0121] In related technologies, it is impossible to achieve elastic scaling of the nodes or CPU resources used by a job. While existing cluster scheduling systems allow job preemption based on priority or other conditions, they cannot automatically scale down or migrate preempted jobs to other host nodes for continued execution. The only solutions for handling preempted jobs are either to suspend them or cancel them.

[0122] However, canceling a preempted job results in the loss of all executed results, requiring the job to be resubmitted and executed from the beginning. Suspending a preempted job typically only releases the CPU resources it uses, not the memory and GPU resources. While this releases CPU resources for other jobs, some may not utilize them fully, leading to waste. Therefore, the above method still suffers from low scheduling efficiency.

[0123] To address the aforementioned issues, this invention proposes a job scheduling method based on CUDA agents that supports elastic scaling of CPU / GPU jobs. This method solves the problem that in high-performance computing scenarios, the resources used by nodes cannot be elastically scaled, thus preventing finer-grained scheduling of resources in the cluster.

[0124] Optionally, the method can be implemented by a job scheduling system that manages and uses CPU, GPU and other resources in the entire cluster. The scheduling system allocates CPU and GPU resources to user jobs that require GPU resources, providing a real-time job usage scheme. It also supports the creation and recovery of snapshots for jobs using GPUs, improving the utilization of resources in the cluster and ensuring that real-time jobs are executed with priority.

[0125] In this embodiment, jobs are divided into real-time jobs and non-real-time jobs. Non-real-time jobs are allowed to share CPU resources, while real-time jobs do not share CPU resources with other jobs. Considering that real-time jobs have a higher priority than non-real-time jobs, real-time jobs are allowed to preempt resources used by running non-real-time jobs. Furthermore, the state of the preempted job (including the CPU, GPU memory, registers, etc. used by the job process) can be saved to a file via CRUI. When idle CPU resources become available in the system, or by sharing CPU resources with other non-real-time jobs, the preempted job can be rescheduled and restored from the saved file to continue execution. This solves the technical problem of low scheduling efficiency of computing resources and achieves the technical effect of improving the scheduling efficiency of computing resources.

[0126] Optionally, this embodiment uses a CUDA agent to save and restore the state of GPU job processes, enabling inter-node migration of GPU jobs and achieving elastic scaling of jobs. Furthermore, it can be combined with preemptive scheduling strategies in existing HPC scheduling systems, improving scheduling accuracy and flexibility, and increasing resource utilization in the cluster. This solves the problem that in high-performance computing scenarios, the resources used by computing nodes cannot be elastically scaled, thus preventing finer-grained scheduling of resources within the cluster.

[0127] Optionally, this embodiment uses a CUDA agent to save and restore job processes, especially GPU job processes, solving the problems of job migration and elastic scaling that traditional HPC scheduling systems cannot handle. It has significant benefits in multiple HPC scheduling scenarios, improving scheduling accuracy and resource utilization efficiency.

[0128] Optionally, the above method can be used in the following scenario: A job cannot be recovered due to a node crash during execution. Using this method, the job process state can be saved periodically, restoring it to the previously saved state after a crash. Another scenario is when a low-priority job is preempted by a high-priority job, and traditional scheduling systems cannot migrate the job process, causing the low-priority job to be unable to continue execution. Using the solution of this patent, the preempted job can be migrated to another node and continue execution from the previously saved state.

[0129] Optionally, in this embodiment, the scheduling system categorizes jobs into real-time jobs and non-real-time jobs. Real-time jobs are allowed to preempt non-real-time jobs. Real-time jobs have exclusive access to the CPU, while non-real-time jobs can share the CPU.

[0130] Optionally, this embodiment uses a CUDA agent (comprising an agent library and an agent process) to save and restore the GPU state. By working in conjunction with CRIU, it performs checkpoint / restore functions for the job process. Of course, CRIU can be replaced with other similar software that has checkpoint / restore capabilities.

[0131] This embodiment provides a scheme for elastic scaling of jobs in high-performance computing scenarios. Based on this scheme, real-time jobs can be completed in the shortest possible time. By assigning dedicated CPU resources to real-time jobs and allowing them to preempt non-real-time jobs, and by ensuring that real-time jobs receive resources before non-real-time jobs, real-time jobs can be completed as quickly as possible.

[0132] Optionally, this embodiment allows non-real-time jobs to share CPU resources, improving resource utilization and avoiding resource waste. For example, if a non-real-time job requires 10 CPUs, but the system only has 5 idle CPUs, then if CPU resource sharing is not allowed, the non-real-time job cannot run due to insufficient resources, and the 5 idle CPUs cannot be utilized, resulting in resource waste. However, in this embodiment, by allowing CPU resource sharing, this job can not only use the 5 idle CPUs but also obtain 5 shared CPUs by sharing with other non-real-time jobs, thus meeting the requirement of 10 CPUs and enabling the job to run.

[0133] Optionally, this embodiment allows the number of CPUs allocated to a non-real-time job to be less than the number of CPUs it requests, thereby improving resource utilization and avoiding resource waste. For example, if a non-real-time job requests 10 CPUs, and the scheduling system allocates 10 CPUs to it, and then 5 of those CPUs are preempted by a real-time job, leaving only 5 CPUs available in the system, then we can allow the non-real-time job to continue running using the remaining 5 unpreempted CPUs.

[0134] Optionally, a parameter can be added when submitting a non-real-time job: minimum number of CPUs.

[0135] In this embodiment, the problem of CRIU's inability to save GPU state is solved by the above method. Moreover, the scheme for saving and restoring GPU state is not related to a specific physical GPU, meaning that this embodiment can be applied to various different GPU models.

[0136] Optionally, this embodiment is transparent to the job process. That is, no modification to the job program code is required, and there is no intrusion into the job process.

[0137] In this embodiment, without a proxy, the CUDA context held by the job process would be unusable after recovery, requiring its re-creation. Therefore, a CUDA proxy can be used to intercept and proxy the job process's CUDA API calls. This embodiment uses a proxy-based approach, where the proxy maintains the CUDA context. When the proxy process recovers the GPU state, it automatically creates a new CUDA context. This process is transparent to the job process, as it is automatically handled by the proxy process, and the job process is unaware of the context changes before and after recovery. Subsequent calls to the CUDA API by the job process will automatically use the newly created CUDA context.

[0138] In this embodiment, the CUDA agent is divided into two parts: an agent library and an agent process. This allows the calling system to send requests to the agent process through the port when the agent process is listening on a certain port. Furthermore, multiple job processes on the same host can use the same agent process. The above method can start the agent process before resuming the job process, and the agent process can create a new CUDA context.

[0139] In this embodiment, the scheduling system orchestrates the entire process by combining CRIU with a scheme for saving and restoring GPU state. The scheduling system first sends a request to the agent, instructing the agent process to stop executing subsequent CUDA kernel functions after completing the current one. Then, it executes a CRIU command to save the job process state and requests the agent process to save any unexecuted CUDA kernel functions to a file.

[0140] In this embodiment, jobs can be divided into real-time jobs and non-real-time jobs, allowing real-time jobs to preempt non-real-time jobs, thereby ensuring that real-time jobs can be executed before non-real-time jobs. Furthermore, this embodiment allows non-real-time jobs to share the CPU, thus improving CPU resource utilization.

[0141] In this embodiment, for real-time jobs, each job can be assigned a priority upon submission, with higher-priority jobs being scheduled first. That is, for real-time jobs, scheduling is performed from highest to lowest priority. Higher-priority jobs are scheduled first. If resources remain after scheduling high-priority jobs (or resources that can be preempted from non-real-time jobs), lower-priority jobs are then scheduled. If no resources remain and resources cannot be preempted from non-real-time jobs, lower-priority jobs wait until resources become available before being scheduled for execution. For real-time jobs of the same priority, they can be scheduled and executed based on their submission time, with earlier submissions being executed first.

[0142] Optionally, to prevent excessive over-segmentation when non-real-time jobs share CPU resources, a maximum over-segmentation ratio can be set in the scheduling system. For example, if the maximum over-segmentation ratio is set to 4, then a CPU can be used by a maximum of 4 non-real-time jobs. Once the maximum over-segmentation ratio is reached, subsequent non-real-time jobs will not be executed immediately but will wait for available resources.

[0143] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.

[0144] This embodiment also provides a resource scheduling strategy determination device, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0145] Figure 4 This is a structural block diagram of a resource scheduling strategy determination device according to an embodiment of this application, such as... Figure 4 As shown, the device may include: an acquisition unit 42, a first determination unit 44, and a second determination unit 46.

[0146] The acquisition unit 42 is used to acquire the pending jobs submitted by the client to the scheduling system.

[0147] The first determining unit 44 is used to determine the job type of the job to be processed and the resource requirement of the computing resources required by the job to be processed.

[0148] The second determining unit 46 is used to determine the resource scheduling data of multiple computing resources in the computing node, and to determine the resource scheduling strategy corresponding to the job to be processed based on the resource scheduling data, job type and resource requirement quantity. The resource scheduling data is used to represent the usage of multiple computing resources, and the resource scheduling strategy is used to represent the rules for scheduling the computing resources required for the job to be processed from the computing node.

[0149] After obtaining the job to be processed through the above-mentioned device, the job type and the amount of resources required to execute the job can be determined, and the resource scheduling data in the computing resources can be determined. Based on the scheduling data, job type and resource requirement, the resource scheduling strategy corresponding to the job to be processed can be determined, thereby solving the technical problem of low scheduling efficiency of computing resources and achieving the technical effect of improving the scheduling efficiency of computing resources.

[0150] It should be noted that the above modules can be implemented by software or hardware. For the latter, they can be implemented in the following ways, but are not limited to: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.

[0151] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above method embodiments when run.

[0152] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0153] Embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0154] Optionally, Figure 5 This is a computer system architecture block diagram of an electronic device according to an embodiment of this application. For example... Figure 5 As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 502 or programs loaded from storage section 508 into random access memory (RAM) 503. The RAM 503 also stores various programs and data required for system operation. The CPU 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output interface 505 (I / O interface) is also connected to the bus 504.

[0155] The following components are connected to the input / output interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a local area network card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the input / output interface 505 as needed. A removable medium 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 510 as needed so that computer programs read from it can be installed into the storage section 508 as needed.

[0156] In one exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0157] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0158] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0159] The embodiments described herein also provide a computer program that includes computer instructions stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps in any of the above method embodiments.

[0160] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0161] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0162] The above are merely preferred embodiments of this application and are not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application should be included within the protection scope of this application.

Claims

1. A method for determining a resource scheduling strategy, characterized in that, include: Retrieve pending jobs submitted by the client to the scheduling system; Determine the job type of the job to be processed, and the amount of computing resources required by the job to be processed; The resource scheduling data of multiple computing resources in a computing node is determined, and based on the resource scheduling data, the job type, and the resource requirement, a resource scheduling strategy corresponding to the job to be processed is determined. The resource scheduling data is used to represent the usage status of the multiple computing resources; the resource scheduling strategy is used to represent the rules for scheduling the computing resources required for the job to be processed from the computing node. The method further includes: In response to the computing resources of a job being occupied by the job to be processed, a process snapshot is created for the target job, wherein the process snapshot is used to save the execution state of the computing resources before they were occupied, including the context information of the target job; According to the resource scheduling strategy, an occupation request is sent to the agent process, wherein the occupation request contains the identity information of the computing resources that can be occupied, and the agent process establishes an independent CUDA context for each proxied job process; Obtain the successful occupancy instruction returned by the agent process; In response to the successful occupancy instruction, the execution status of the job processes in the available computing resources is saved.

2. The method according to claim 1, characterized in that, The step of determining the resource scheduling strategy corresponding to the job to be processed based on the resource scheduling data, the job type, and the resource demand includes: Based on the resource scheduling data, the number of schedulable resources in the computing node is determined, wherein the number of schedulable resources is used to characterize the number of computing resources that can be scheduled among the multiple computing resources; The resource scheduling strategy is determined based on the job type, the resource requirement, and the number of schedulable resources.

3. The method according to claim 2, characterized in that, The step of determining the resource scheduling strategy based on the job type, the resource requirement quantity, and the schedulable resource quantity includes: The comparison result is obtained by comparing the resource demand quantity with the schedulable resource quantity. Based on the comparison results and the job type, the scheduling strategy is determined.

4. The method according to claim 3, characterized in that, Determining the scheduling strategy based on the comparison result and the job type includes: In response to the fact that the job type is a non-real-time job type, and the comparison result is that the number of schedulable resources is greater than or equal to the number of resource requirements, the resource scheduling strategy is determined to be: to use the computing resources that are allowed to be scheduled to execute the job to be processed.

5. The method according to claim 4, characterized in that, Determining the scheduling strategy based on the comparison result and the job type includes: In response to the fact that the job type is the non-real-time job type and the comparison result is that the number of schedulable resources is less than the number of resources required, it is determined whether there are shared resources in the computing node; In response to the existence of the shared resources in the computing node, the sum of the number of shared resources and the number of schedulable resources is determined, and in response to the sum of the number of shared resources and the number of schedulable resources being greater than or equal to the resource demand, the resource scheduling strategy is determined to be: to execute the pending job using the computing resources and shared resources that are allowed to be scheduled; In response to the absence of the shared resource in the computing node, or the sum of the number of the shared resources and the number of schedulable resources being less than the resource requirement, a resource to be occupied is determined; Based on the quantity of resources to be occupied, the resource scheduling strategy is determined as follows: the pending job is executed using the schedulable computing resources, the shared resources, and the resources to be occupied.

6. The method according to claim 5, characterized in that, The determination of the resources to be occupied includes: Based on the resource scheduling data, at least one of the jobs being processed among the plurality of computing resources is identified; From at least one of the stated jobs, at least one of the stated target jobs is determined, wherein the priority of the target job is lower than the priority of the job to be processed; The computing resources corresponding to the target job are identified as the resources to be occupied.

7. The method according to claim 3, characterized in that, Determining the scheduling strategy based on the comparison result and the job type includes: In response to the fact that the job type is a real-time job type, and the comparison result is that the number of schedulable resources is greater than or equal to the number of resource requirements, the resource scheduling strategy is determined to be: to use the computing resources that are allowed to be scheduled to execute the job to be processed.

8. The method according to claim 7, characterized in that, Determining the scheduling strategy based on the comparison result and the job type includes: In response to the fact that the job type is a real-time job type and the comparison result is that the number of schedulable resources is less than the number of resource requirements, it is determined whether there are any available computing resources among the scheduled computing resources that can be occupied. In response to the existence of the available computing resources, the scheduling strategy is determined to be: to execute the pending job using the schedulable computing resources and the available computing resources.

9. The method according to claim 1, characterized in that, The method further includes: The pending jobs will be executed according to the resource scheduling strategy. In response to the completion of the pending job, the computing resources occupied by the pending job are released; Restore the historical job processes in the available computing resources, wherein the historical job processes are used to characterize the execution status of jobs executed on the available computing resources before they were occupied.

10. A device for determining a resource scheduling strategy, characterized in that, include, The acquisition unit is used to acquire pending jobs submitted by the client to the scheduling system; The first determining unit is used to determine the job type of the job to be processed and the resource requirement of the computing resources required by the job to be processed. The second determining unit is used to determine the resource scheduling data of multiple computing resources in the computing node, and to determine the resource scheduling strategy corresponding to the job to be processed based on the resource scheduling data, the job type and the resource requirement quantity, wherein the resource scheduling data is used to represent the usage status of the multiple computing resources, and the resource scheduling strategy is used to represent the rules for scheduling the computing resources required for the job to be processed from the computing node; The second determining unit is further configured to: in response to the computing resources of the job being occupied by the job to be processed, create a process snapshot for the target job, wherein the process snapshot is used to save the execution state of the computing resources before they were occupied, including the context information of the target job; According to the resource scheduling strategy, an occupation request is sent to the agent process, wherein the occupation request contains the identity information of the computing resources that can be occupied, and the agent process establishes an independent CUDA context for each proxied job process; Obtain the successful occupancy instruction returned by the agent process; In response to the successful occupancy instruction, the execution status of the job processes in the available computing resources is saved.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the method described in any one of claims 1 to 9.

12. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method described in any one of claims 1 to 9.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method described in any one of claims 1 to 9.

Images