Cloud-native job scheduling method and system, and storage medium
By using cloud-native job scheduling methods, computing resources can be dynamically created and released using public cloud resources. This solves the resource limitations and security vulnerabilities of traditional job scheduling systems in public cloud environments, and achieves efficient and isolated high-performance computing.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHENZHEN CLOUD SOFTWARE CO LTD
- Filing Date
- 2022-03-02
- Publication Date
- 2026-06-02
AI Technical Summary
Traditional high-performance computing job scheduling systems suffer from resource limitations, security vulnerabilities, and job interference in public cloud environments, making it impossible to effectively utilize the elastic computing resources of public clouds.
It adopts a cloud-native job scheduling method, receiving user requests through the Cloud Shell subsystem, scheduling and monitoring computing resources through the Job Manager subsystem, and dynamically creating and releasing various types of public cloud computing resources through the Resource Manager subsystem, thereby achieving isolated scheduling of jobs across multiple cloud vendors or regions.
It achieves efficient utilization of public cloud resources, avoids resource idleness and security vulnerabilities, supports multiple types of computing resources, eliminates dependence on head nodes, and reduces mutual interference between jobs.
Smart Images
Figure CN114661436B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of high-performance computing, and in particular to a cloud-native job scheduling method, system, and storage medium. Background Technology
[0002] Traditional high-performance computing (HPC) job scheduling systems, including SLURM, SGE, PBS, Condor, LSF, etc., typically include two main categories of functions:
[0003] First, high-performance computing cluster management: mainly based on the management of computer nodes using CPUs and GPUs, used to build a cluster environment for job execution;
[0004] Second, job management: This mainly allows cluster users to submit, view, cancel, and perform other related functions of jobs via command line (CLI).
[0005] Current job scheduling systems are mainly used in scenarios with offline server cluster resources, such as supercomputing centers and enterprise high-performance computing clusters, to provide multi-user shared job scheduling functions for a static, fixed computer node cluster.
[0006] Meanwhile, public cloud service providers such as AWS, Google Cloud, Microsoft Azure, Alibaba Cloud, Tencent Cloud, and Huawei Cloud offer a variety of elastic and dynamic computing resource services, such as Alibaba Cloud Elastic Compute Service (EC2) and AWS EC2. Currently, the main method for building a high-performance computing system using public clouds is to deploy traditional job scheduling systems such as SLURM to virtual machines in the cloud and utilize their cloud bursting capabilities to connect to the cloud vendor's API to dynamically scale the cluster. However, this approach has the following obvious drawbacks:
[0007] 1. Traditional job scheduling systems require at least one control node to run the control manager, usually called the head node or master node. The pre-determination of this node means that the entire high-performance computing system in the cloud is limited to the region where the control node is located, i.e., a certain cloud region.
[0008] 2. Traditional job scheduling systems' high-performance computing cluster management functions can only be used to manage virtual machine resources from cloud vendors. Using them means that only one type of computing resource, the virtual machine service from the cloud vendor, can be used. However, cloud vendors now offer a variety of resources that can be used for high-performance computing tasks, such as container services like Alibaba Cloud ECI, function computing services like Alibaba Cloud Function Compute FC, and AWS Lambda.
[0009] 3. Traditional job scheduling systems are designed for multiple users to share a limited computer cluster. Users can see each other's job execution information, which increases security vulnerabilities and can cause mutual interference. For example, if user A submits a very large job that causes the scheduling system to crash, other users' running jobs may also crash. Cloud providers offer almost unlimited computing resources, making such a shared-use design impractical and causing the aforementioned problems. Summary of the Invention
[0010] The main objective of this invention is to provide a cloud-native job scheduling method, system, and storage medium. Jobs are completely isolated and have no shared resources. It is specifically designed for high-performance computing systems built using public cloud resources. It abandons the cluster management function in traditional high-performance computing job scheduling systems and instead utilizes the management function of public cloud computing resources themselves. It also supports multiple types of public cloud computing resources, enabling a job to be scheduled to multiple cloud vendors or different regions of the same cloud vendor for simultaneous computation.
[0011] To achieve the above objectives, this invention proposes a cloud-native job scheduling method. The method is applied to a cloud-native job scheduling system, which includes a Cloud Shell subsystem and a job and resource management subsystem. The method includes the following steps:
[0012] The Cloud Shell subsystem sends the user's job submission request to the job and resource management subsystem. The job submission request specifies the relevant information that the job needs to run, including: computing resource parameters and the number of tasks to run in the job.
[0013] The job and resource management subsystem receives the job submission request, schedules the corresponding computing resources of the cloud provider, schedules the programs / instructions that the job needs to execute to run on the corresponding computing resources, monitors the status of job execution at the same time, and releases the computing resources after the job program / instruction has finished executing.
[0014] The job and resource management subsystem includes: a Job Manager subsystem and a Resource Manager subsystem.
[0015] The steps of receiving the job submission request through the job and resource management subsystem, scheduling the corresponding computing resources of the cloud provider, scheduling the programs / instructions that the job needs to execute to run on the corresponding computing resources, and simultaneously monitoring the job's running status, and releasing the computing resources after the job program / instruction has finished executing, include:
[0016] The Job Manager subsystem receives the job submission request and sends a request to the Resource Manager subsystem to schedule computing resources based on the computing resource parameters in the job submission request.
[0017] The Resource Manager subsystem, based on the request to schedule computing resources, uses the cloud vendor's API interface to start the corresponding computing resources to create them. After the resources are created, it returns a list of the created resource IDs to the Job Manager subsystem.
[0018] The Job Manager subsystem schedules the programs / instructions that the job needs to execute to the corresponding computing resources based on the returned computing resource information, and at the same time monitors the status of the job execution. After the job program / instruction finishes execution, it notifies the Resource Manager subsystem to release the computing resources.
[0019] The steps of the Resource Manager subsystem in starting the corresponding computing resources using the cloud vendor's API interface based on the request to schedule computing resources, creating the corresponding computing resources, and returning the list of created resource IDs to the Job Manager subsystem after the resource creation is completed include:
[0020] The Resource Manager subsystem executes a scheduling strategy based on the request for scheduling computing resources, and determines the cloud vendors, regions, and resource quantities that need to be scheduled for this job.
[0021] The Resource Manager subsystem calls the corresponding computing resources by calling the API of the cloud provider for the corresponding resource type, based on the determined cloud provider, its region, and the amount of resources.
[0022] After the resource creation is complete, the list of corresponding created resource IDs will be returned to the Job Manager subsystem.
[0023] The Job Manager subsystem, based on the returned computing resource information, schedules the programs / instructions required by the job to run on the corresponding computing resources, and simultaneously monitors the job's running status. The step of notifying the Resource Manager subsystem to release computing resources after the job program / instruction execution is complete includes:
[0024] The Job Manager subsystem runs the JobAgent subsystem on each computing resource based on the returned computing resource information, and combines the computing resources running the JobAgent subsystem into a computing cluster.
[0025] After resource startup is complete, the Job Manager subsystem schedules the programs / instructions that the job needs to execute to the corresponding computing resources, runs them through the Job Agent subsystem, and monitors the status of job execution at the same time. After the job program / instruction finishes execution, it notifies the Resource Manager subsystem to release the computing resources.
[0026] The method further includes:
[0027] The corresponding cloud vendor computing resources are destroyed through the Resource Manager subsystem.
[0028] The method further includes:
[0029] The Cloud Shell subsystem provides users with job management capabilities via command-line CLI; the cloud provider's computing resources include virtual machines, containers, and serverless function resources; the commands submitted via the command-line CLI include: number of tasks, resource metrics, and resource categories.
[0030] The method further includes:
[0031] The Cloud Shell subsystem sends user job submission requests to the Job Manager subsystem via the SSH over WebSocket protocol or as a Restful API request.
[0032] This invention also proposes a cloud-native job scheduling system, which includes: a Cloud Shell subsystem and a job and resource management subsystem, wherein:
[0033] The Cloud Shell subsystem is used to send the user's job submission request to the job and resource management subsystem. The job submission request specifies the relevant information that the job needs to run, including: computing resource parameters and job execution task information.
[0034] The job and resource management subsystem is used to receive the job submission request, schedule the computing resources corresponding to the cloud provider, schedule the programs / instructions that the job needs to execute to run on the corresponding computing resources, monitor the status of job execution at the same time, and release the computing resources after the job program / instruction has finished executing.
[0035] The job and resource management subsystem includes: a Job Manager subsystem and a Resource Manager subsystem, wherein:
[0036] The Job Manager subsystem is used to receive the job submission request and send a request to schedule computing resources to the Resource Manager subsystem based on the computing resource parameters in the job submission request.
[0037] The Resource Manager subsystem is used to start the corresponding computing resources using the cloud vendor's API interface according to the request for scheduling computing resources, so as to create the corresponding computing resources. After the resource creation is completed, the list of corresponding created resource IDs is returned to the Job Manager subsystem.
[0038] The Job Manager subsystem is also used to schedule the programs / instructions that the job needs to execute to the corresponding computing resources based on the returned computing resource information, and at the same time monitor the status of the job operation. After the job program / instruction is completed, it notifies the Resource Manager subsystem to release the computing resources.
[0039] The present invention also proposes a computer storage medium storing a computer program, which, when executed by a processor, implements the steps of the method described above.
[0040] The cloud-native job scheduling method, system, and storage medium proposed in this invention provide complete isolation and no sharing between jobs. It is specifically designed for high-performance computing systems built using public cloud resources. It completely abandons the cluster management function in traditional high-performance computing job scheduling systems and instead utilizes the management function of the public cloud computing resources themselves, while simultaneously supporting multiple types of public cloud computing resources. It also eliminates the dependence on head node requirements, so that a job can be scheduled to multiple cloud vendors or different regions of the same cloud vendor for simultaneous computation.
[0041] Compared to existing technologies, the job scheduling system of this invention completely overcomes the drawbacks of traditional job scheduling systems. First, it does not have a pre-started, static computing cluster, which reduces resource idleness and waste, and eliminates the need for cluster management functions. After a user submits a job, the job scheduling system dynamically creates the computing resources required by the cloud provider. There is no concept of a head node or control node, and the created computing resources may be located in any region of any cloud provider, avoiding the idleness caused by head nodes. In addition, this job scheduling system can schedule various computing resources such as virtual machines, containers, and serverless functions, while traditional job scheduling systems can only schedule virtual machine resources. Finally, computing resources are always scheduled according to the job, and no computing resources are shared between jobs. After the job finishes running, the corresponding resource set is released, thereby avoiding the negative factors such as security and job interference caused by multi-user resource sharing in traditional job scheduling systems. Attached Figure Description
[0042] Figure 1 This is a flowchart illustrating the cloud-native job scheduling method of the present invention;
[0043] Figure 2 This is a schematic diagram of the overall architecture of the cloud-native job scheduling system of the present invention;
[0044] Figure 3 This is a detailed flowchart illustrating the cloud-native job scheduling method of the present invention.
[0045] Figure 4 This is a schematic diagram of the process of scheduling computing resources in the present invention. Detailed Implementation
[0046] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
[0047] Reference Figures 1-4 As shown, this invention proposes a cloud-native job scheduling method, which is applied to a cloud-native job scheduling system. The job scheduling system includes a Cloud Shell subsystem and a job and resource management subsystem. The overall architecture of the cloud-native job scheduling system of this invention is as follows: Figure 2 As shown.
[0048] like Figure 1 As shown, the method includes the following steps:
[0049] S10, the user's job submission request is sent to the job and resource management subsystem through the Cloud Shell subsystem. The job submission request specifies the relevant information that the job needs to run, including: computing resource parameters and job execution task information.
[0050] S20: Receive the job submission request through the job and resource management subsystem, schedule the computing resources corresponding to the cloud provider, schedule the programs / instructions that the job needs to execute to run on the corresponding computing resources, and monitor the status of the job operation at the same time. After the job program / instruction is executed, release the computing resources.
[0051] The job and resource management subsystem includes: a Job Manager subsystem and a Resource Manager subsystem.
[0052] The steps of receiving the job submission request through the job and resource management subsystem, scheduling the corresponding computing resources of the cloud provider, scheduling the programs / instructions that the job needs to execute to run on the corresponding computing resources, and simultaneously monitoring the job's running status, and releasing the computing resources after the job program / instruction has finished executing, include:
[0053] The Job Manager subsystem receives the job submission request and sends a request to the Resource Manager subsystem to schedule computing resources based on the computing resource parameters in the job submission request.
[0054] The Resource Manager subsystem, based on the request to schedule computing resources, uses the cloud vendor's API interface to start the corresponding computing resources to create them. After the resources are created, it returns a list of the created resource IDs to the Job Manager subsystem.
[0055] The Job Manager subsystem schedules the programs / instructions that the job needs to execute to the corresponding computing resources based on the returned computing resource information, and at the same time monitors the status of the job execution. After the job program / instruction finishes execution, it notifies the Resource Manager subsystem to release the computing resources.
[0056] The steps of the Resource Manager subsystem in starting the corresponding computing resources using the cloud vendor's API interface based on the request to schedule computing resources, creating the corresponding computing resources, and returning the list of created resource IDs to the Job Manager subsystem after the resource creation is completed include:
[0057] The Resource Manager subsystem executes a scheduling strategy based on the request for scheduling computing resources, and determines the cloud vendors, regions, and resource quantities that need to be scheduled for this job.
[0058] The Resource Manager subsystem calls the corresponding computing resources by calling the API of the cloud provider for the corresponding resource type, based on the determined cloud provider, its region, and the amount of resources.
[0059] After the resource creation is complete, the list of corresponding created resource IDs will be returned to the Job Manager subsystem.
[0060] The Job Manager subsystem, based on the returned computing resource information, schedules the programs / instructions required by the job to run on the corresponding computing resources, and simultaneously monitors the job's running status. The step of notifying the Resource Manager subsystem to release computing resources after the job program / instruction execution is complete includes:
[0061] The Job Manager subsystem runs the JobAgent subsystem on each computing resource based on the returned computing resource information, and combines the computing resources running the JobAgent subsystem into a computing cluster.
[0062] After resource startup is complete, the Job Manager subsystem schedules the programs / instructions that the job needs to execute to the corresponding computing resources, runs them through the Job Agent subsystem, and monitors the status of job execution at the same time. After the job program / instruction finishes execution, it notifies the Resource Manager subsystem to release the computing resources.
[0063] The method further includes:
[0064] The corresponding cloud vendor computing resources are destroyed through the Resource Manager subsystem.
[0065] The method further includes:
[0066] The Cloud Shell subsystem provides users with job management capabilities via command-line CLI; the cloud provider's computing resources include virtual machines, containers, and serverless function resources; the commands submitted via the command-line CLI include: number of tasks, resource metrics, and resource categories.
[0067] The method further includes:
[0068] The Cloud Shell subsystem sends user job submission requests to the Job Manager subsystem via the SSH over WebSocket protocol or as a Restful API request.
[0069] This invention also proposes a cloud-native job scheduling system, which includes: a Cloud Shell subsystem and a job and resource management subsystem, wherein:
[0070] The Cloud Shell subsystem is used to send the user's job submission request to the job and resource management subsystem. The job submission request specifies the relevant information that the job needs to run, including: computing resource parameters and job execution task information.
[0071] The job and resource management subsystem is used to receive the job submission request, schedule the computing resources corresponding to the cloud provider, schedule the programs / instructions that the job needs to execute to run on the corresponding computing resources, monitor the status of job execution at the same time, and release the computing resources after the job program / instruction has finished executing.
[0072] The job and resource management subsystem includes: a Job Manager subsystem and a ResourceManager subsystem, wherein:
[0073] The Job Manager subsystem is used to receive the job submission request and send a request to schedule computing resources to the Resource Manager subsystem based on the computing resource parameters in the job submission request.
[0074] The Resource Manager subsystem is used to start the corresponding computing resources using the cloud vendor's API interface according to the request for scheduling computing resources, so as to create the corresponding computing resources. After the resource creation is completed, the list of corresponding created resource IDs is returned to the Job Manager subsystem.
[0075] The Job Manager subsystem is also used to schedule the programs / instructions that the job needs to execute to the corresponding computing resources based on the returned computing resource information, and at the same time monitor the status of the job operation. After the job program / instruction is completed, it notifies the Resource Manager subsystem to release the computing resources.
[0076] The cloud-native job scheduling method proposed in this invention completely isolates jobs without sharing, and is specifically designed to serve high-performance computing systems built using public cloud resources. It completely abandons the cluster management function in traditional high-performance computing job scheduling systems, and instead utilizes the management function of the public cloud computing resources themselves, while simultaneously supporting multiple types of public cloud computing resources. It also eliminates the dependence on head node requirements, so that a job can be scheduled to multiple cloud vendors or different regions of the same cloud vendor for simultaneous computation.
[0077] Compared to existing technologies, the job scheduling system of this invention completely overcomes the drawbacks of traditional job scheduling systems. First, it does not have a pre-started, static computing cluster, which reduces resource idleness and waste, and eliminates the need for cluster management functions. After a user submits a job, the job scheduling system dynamically creates the computing resources required by the cloud provider. There is no concept of a head node or control node, and the created computing resources may be located in any region of any cloud provider, avoiding the idleness caused by head nodes. In addition, this job scheduling system can schedule various computing resources such as virtual machines, containers, and serverless functions, while traditional job scheduling systems can only schedule virtual machine resources. Finally, computing resources are always scheduled according to the job, and no computing resources are shared between jobs. After the job finishes running, the corresponding resource set is released, thereby avoiding the negative factors such as security and job interference caused by multi-user resource sharing in traditional job scheduling systems.
[0078] The following is a detailed description of the present invention:
[0079] This invention implements a cloud-native high-performance computing job scheduling system and method. Jobs are completely isolated and have no sharing, and it is specifically designed to serve high-performance computing systems built using public cloud resources. It completely abandons the cluster management function in traditional high-performance computing job scheduling systems, and instead utilizes the management function of the public cloud computing resources themselves, while simultaneously supporting multiple types of public cloud computing resources. It also eliminates the dependence on head node requirements, so that a job can be scheduled to multiple cloud vendors or different regions of the same cloud vendor for simultaneous computation.
[0080] like Figure 2 As shown, the job scheduling system of the present invention comprises three subsystems or modules, namely:
[0081] A web-based Cloud Shell that allows users to manage jobs using the command-line interface (CLI) provided by Cloud Shell.
[0082] A Job Manager subsystem receives requests from Cloud Shell and is responsible for executing jobs. Specifically, the Resource Manager subsystem schedules cloud provider computing resources such as virtual machines, containers, or serverless functions. Once the computing resources are ready, the Job Manager is responsible for loading the specific computing tasks onto the resources and running them.
[0083] A Resource Manager subsystem is used to schedule computing resources from cloud providers, such as virtual machines, containers, or serverless functions.
[0084] This invention also provides a job scheduling method utilizing public cloud resources. Users send job submission requests using Cloud Shell, specifying the computing resources required for the job, the number of tasks to be executed, and other information in the request. Upon receiving the submission request, the JobManager subsystem first calls the Resource Manager subsystem to schedule the corresponding computing resources from the cloud provider. The Resource Manager is responsible for using the cloud provider's API to start the corresponding computing resources. After the resources are started, the Job Manager schedules the programs / instructions required by the job to run on the corresponding computing resources and monitors the job's running status. After the job programs / instructions have finished executing, the Job Manager is notified to release the computing resources.
[0085] The job scheduling system of the present invention is as follows: Figure 2 As shown:
[0086] exist Figure 2 In this context, Web CloudShell is a web-based CLI tool that provides a command-line interface similar to the Linux Shell for users to operate. In its implementation, it is a Linux Shell-like command-line page that opens by entering an address URL in a browser. This page connects to a remote Linux machine or a Linux container via the SSH over WebSocket protocol. Web CloudShell runs on the user's local computer's browser.
[0087] On the other side of the network is the server subsystem of this invention, in which the Job Manager is responsible for accepting job submission requests sent from WebCloudShell. Based on the computing resource parameters in the request, it calls the ResourceManager subsystem to create the corresponding computing resources from the cloud provider. After the resources are successfully started, the Job Agent subsystem is run on each computing resource. These computing resources running the Job Agent are combined into a computing cluster. At this time, the JobManager will run the job execution instructions submitted by the user through the Job Agent to complete the job execution. After the job execution is completed, the Job Manager will notify the Resource Manager to release the corresponding computing resources.
[0088] The Resource Manager subsystem is responsible for the creation and destruction of cloud resources. By calling the APIs provided by cloud vendors, Resource Manager can create and destroy various cloud vendor computing resources, mainly including virtual machines, containers, and serverless functions, including but not limited to the following resources:
[0089] Alibaba Cloud: ECS cloud server, Function Compute (FC), Elastic Container Instance (ECI);
[0090] Tencent Cloud: Cloud Servers, Container Services, Cloud Functions;
[0091] Huawei Cloud: Elastic Cloud Server (ECS), Cloud Container Instance (ECI);
[0092] Amazon Web Service:ec2,ecs,Lambda;
[0093] Microsoft Azure:Virtual Machines,Azure Container Instances,AzureFunctions;
[0094] Google Cloud: Compute Engine, Google Kubernetes Engine, Cloud Functions.
[0095] The Job Agent subsystem runs within the started cloud computing resources. Each started computing resource runs a Job Agent subsystem, which is usually pre-deployed on the cloud provider's side in the form of a virtual machine image, container image, or function application package. It is responsible for interacting with the Job Manager and receiving the run instructions sent from the Job Manager to run the user job programs to be executed. At the same time, the Job Manager monitors the status of the user job programs and notifies the Job Manager of the running status.
[0096] The aforementioned job scheduling system completely overcomes the drawbacks of traditional job scheduling systems. Firstly, it does not have a pre-started, static computing cluster, which reduces resource idleness and waste, and eliminates the need for cluster management functions. After a user submits a job, the job scheduling system dynamically creates the computing resources required by the cloud provider. There is no concept of a head node or control node; the job and resource management functions originally undertaken by the head node are separated into an independent subsystem. This decouples job submission from specific cloud providers and regions, and the created computing resources can be located in any region of any cloud provider, avoiding the idleness caused by head nodes. Furthermore, this job scheduling system can schedule various computing resources such as virtual machines, containers, and serverless functions, while traditional job scheduling systems can only schedule virtual machine resources. Finally, computing resources are always scheduled according to the job, without sharing any computing resources between jobs, thus avoiding the negative factors such as security vulnerabilities and job interference caused by multi-user resource sharing in traditional job scheduling systems.
[0097] This invention also provides a job scheduling method for using cloud resources to run computing jobs, such as... Figure 3 As shown:
[0098] Figure 3 There are 5 steps in total to complete the entire process of job submission, execution, and resource release. The detailed process of these 5 steps is described in detail below.
[0099] S1: Job Submission The Cloud WebShell command-line CLI is used to submit jobs. Command-line is the most common method for submitting jobs in high-performance computing (HPC). The submitted command contains three basic pieces of information:
[0100] The `-tasks` option specifies the number of tasks the job needs to run. For example, `-tasks 3` means the job needs to run 3 tasks.
[0101] Resource metrics - res: The amount of resources required for each task. Resources are named in the format "Memory size - Number of CPU cores - Disk size - Disk type - Number of GPU cards - GPU type". The number of GPU cards and GPU type are optional. For example, "2G-4-40G-SSD" means that each task requires 2GB of memory, 4 CPU cores, and 40G of SSD disk storage. Another example is "1G-8-20G-ESSD-2-v100", which means that it requires 1GB of memory, 8 CPU cores, 20GB of ESSD storage, and 2 Tesla V100 GPU accelerator cards.
[0102] Resource Category - type: The category of cloud computing resources, which is one of VM, Container, or Function, where VM represents Virtual Machine, Container represents container, and Function represents serverless function resources.
[0103] In one possible implementation, the existing HPC job management system can be extended to support the above information, such as extending SLURM's sbatch to submit jobs as shown below:
[0104] sbatch-tasks 3-res 2G-8-160G-SSD-type VM myJob.sbatch
[0105] The above implementation extends the support of the sbatch command to include the information mentioned in this invention. In the example above, the myJob.sbatch job will run 3 tasks, each of which uses 2GB of memory, 8 CPU cores, and 160GB of SSD storage. The computing resources are virtual machines, such as AWS EC2 or Alibaba Cloud ECS.
[0106] After the job instruction is submitted, CloudShell sends it to the backend Job Manager subsystem. Instructions can be sent using the SSH over WebSocket protocol or as a RESTful API request.
[0107] After the job submission command is submitted from the Web CloudShell to the Job Manager subsystem, step S2 is executed. The request to schedule computing resources is sent to Resource Manager and executed:
[0108] S2: Scheduling computing resources, the specific process is as follows: Figure 4 As shown:
[0109] S2-1 First, the scheduling strategy is executed to determine the cloud vendors and regions that need to be scheduled for this job. The scheduling strategy can be a simple random scheduling, round-robin scheduling, or a more complex customized scheduling algorithm. By running the scheduling strategy, the list of cloud vendors and regions for resources is determined, as well as the number of resources that need to be scheduled for each cloud vendor and region. It should be emphasized that a job may use resources from multiple regions of multiple cloud vendors for simultaneous scheduling.
[0110] S2-2, based on the determined cloud vendor, region, and quantity information, call the corresponding resource type API of the cloud vendor to create resources. For example, for VM type, it may use Alibaba Cloud's ECS cloud server API to create the corresponding number of virtual machines to the specified cloud vendor and region. In another implementation method, it may call the cloud vendor's Agnostic SDK (multi-cloud resource orchestration tool) to perform resource creation, such as using HashiCorp's Terraform to create resources.
[0111] In the previous example, it will create 3 virtual machines, each with 2GB of memory, 8 CPU cores, and 160GB of SSD storage.
[0112] S2-3 After the resource creation is complete, return the list of the corresponding created resource IDs to the Job Manager subsystem to complete this step.
[0113] Step S3: Based on the returned computing resource information, run the Job Agent subsystem on each computing resource to complete the computing resource cluster preparation. The Job Agent can run using a predefined VM Image, Container Image, or Function package along with the computing resources created in step S2, or by using a hook method on the started computing resources to download the Job Agent package directly from the network during initialization and then run the Job Agent startup after the download is complete. The download method can use commands such as curl / wget. After each JobAgent starts, it will establish communication with the Job Manager, for example, using gRPC or Socket communication. After successful startup, the JobAgent will send a "Ready" notification to the Job Manager, indicating that it can accept task execution.
[0114] After receiving Ready notifications from all Job Agents, the Job Manager executes job execution in step S4. It sends a run task notification to each Job Agent. Upon receiving the run notification, the Job Agent runs the job script to be executed. The execution of the job script is similar to that of a traditional job scheduling system, such as using the operating system's fork mechanism to start a child process and run the job script in the child process.
[0115] Referring to the above task, the myJob.sbatch script will run in a child process. The Job Agent will monitor the running job script at all times and periodically notify the Job Manager of the status information Running until the job script finishes running. After the Job Agent detects that the job script has finished running, it will send the status information Completed or failed to the JobManager.
[0116] After receiving a Completed message from the Job Agent, the Job Manager executes step S5 to release the computing resources running the Job Agent. The release request is sent by the Job Manager to the Resource Manager subsystem. The Resource Manager will execute the release operation API according to the computing resource ID and the corresponding region of the corresponding cloud provider. Similar to scheduling resources, the release operation may be implemented using the API of the cloud provider, such as the API of Alibaba Cloud ECS cloud server, or using a multi-cloud resource orchestration tool SDK, such as HashiCorp's Terraform. This step will be executed cyclically after each Completed message is received. After all computing resources have been released, the entire job scheduling and execution process ends.
[0117] It should be noted that in other embodiments, the Job Manager and Resource Manager mentioned above may be combined into a single system, enabling this single system to handle both job management and resource management capabilities. In addition, the Web CloudShell may be replaced by direct program calls such as RESTful API calls or gRPC calls, but the call parameters will always include the number of tasks, resource metrics, and resource category information.
[0118] The present invention also proposes a computer storage medium storing a computer program, which, when executed by a processor, implements the steps of the method described above.
[0119] The cloud-native job scheduling method, system, and storage medium proposed in this invention provide complete isolation and no sharing between jobs. It is specifically designed for high-performance computing systems built using public cloud resources. It completely abandons the cluster management function in traditional high-performance computing job scheduling systems and instead utilizes the management function of the public cloud computing resources themselves, while simultaneously supporting multiple types of public cloud computing resources. It also eliminates the dependence on head node requirements, so that a job can be scheduled to multiple cloud vendors or different regions of the same cloud vendor for simultaneous computation.
[0120] Compared to existing technologies, the job scheduling system of this invention completely overcomes the drawbacks of traditional job scheduling systems. First, it does not have a pre-started, static computing cluster, which reduces resource idleness and waste, and eliminates the need for cluster management functions. After a user submits a job, the job scheduling system dynamically creates the computing resources required by the cloud provider. There is no concept of a head node or control node, and the created computing resources may be located in any region of any cloud provider, avoiding the idleness caused by head nodes. In addition, this job scheduling system can schedule various computing resources such as virtual machines, containers, and serverless functions, while traditional job scheduling systems can only schedule virtual machine resources. Finally, computing resources are always scheduled according to the job, and no computing resources are shared between jobs. After the job finishes running, the corresponding resource set is released, thereby avoiding the negative factors such as security and job interference caused by multi-user resource sharing in traditional job scheduling systems.
[0121] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.
Claims
1. A cloud-native job scheduling method, characterized in that, The method is applied to a cloud-native job scheduling system, which includes a Cloud Shell subsystem and a job and resource management subsystem. The method includes the following steps: The Cloud Shell subsystem sends the user's job submission request to the job and resource management subsystem. The job submission request specifies the relevant information that the job needs to run, including: computing resource parameters and the number of tasks to run in the job. The job and resource management subsystem receives the job submission request, schedules the corresponding computing resources of the cloud provider, schedules the programs / instructions that the job needs to execute to run on the corresponding computing resources, and monitors the status of the job execution at the same time. After the job program / instruction finishes execution, the computing resources are released. The Cloud Shell subsystem provides users with job management capabilities via command-line CLI; the cloud provider's computing resources include virtual machines, containers, and serverless function resources; the commands submitted via the command-line CLI include: number of tasks, resource metrics, and resource categories; The Cloud Shell subsystem sends user job submission requests to the Job Manager subsystem via the SSH over WebSocket protocol or in the form of a Restful API request; The job and resource management subsystem includes: a Job Manager subsystem and a Resource Manager subsystem. The steps of receiving the job submission request through the job and resource management subsystem, scheduling the corresponding computing resources of the cloud provider, scheduling the programs / instructions that the job needs to execute to run on the corresponding computing resources, and simultaneously monitoring the job's running status, and releasing the computing resources after the job program / instruction has finished executing, include: The Job Manager subsystem receives the job submission request and sends a request to the Resource Manager subsystem to schedule computing resources based on the computing resource parameters in the job submission request. The Resource Manager subsystem, based on the request to schedule computing resources, uses the cloud vendor's API interface to start the corresponding computing resources to create them. After the resources are created, it returns a list of the created resource IDs to the Job Manager subsystem. The Job Manager subsystem schedules the programs / instructions that the job needs to execute to the corresponding computing resources based on the returned computing resource information, and at the same time monitors the status of the job execution. After the job program / instruction finishes execution, it notifies the Resource Manager subsystem to release the computing resources. The Resource Manager subsystem, based on the request to schedule computing resources, uses the cloud vendor's API interface to start the corresponding computing resources to create them. After resource creation is complete, the step of returning the list of created resource IDs to the Job Manager subsystem includes: The Resource Manager subsystem executes a scheduling strategy based on the request for scheduling computing resources, and determines the cloud vendors, regions, and resource quantities that need to be scheduled for this job. The Resource Manager subsystem calls the corresponding computing resources by calling the API of the cloud provider for the corresponding resource type, based on the determined cloud provider, its region, and the amount of resources. After the resource creation is complete, the list of corresponding created resource IDs will be returned to the Job Manager subsystem; The Job Manager subsystem schedules the programs / instructions required by the job to run on the corresponding computing resources based on the returned computing resource information, and simultaneously monitors the job's running status. The step of notifying the Resource Manager subsystem to release computing resources after the job program / instruction has finished executing includes: The Job Manager subsystem runs the Job Agent subsystem on each computing resource based on the returned computing resource information, and combines the computing resources running the Job Agent subsystem into a computing cluster. After resource startup is complete, the Job Manager subsystem schedules the programs / instructions that the job needs to execute to the corresponding computing resources, runs them through the Job Agent subsystem, and monitors the status of job execution at the same time. After the job program / instruction finishes execution, it notifies the Resource Manager subsystem to release the computing resources.
2. The method according to claim 1, characterized in that, The method further includes: The corresponding cloud vendor computing resources are destroyed through the Resource Manager subsystem.
3. A cloud-native job scheduling system, characterized in that, The job scheduling system includes: a Cloud Shell subsystem and a job and resource management subsystem, wherein: The Cloud Shell subsystem is used to send user job submission requests to the job and resource management subsystem. The job submission request specifies relevant information for the job to run, including computing resource parameters and the number of tasks to be executed. The Cloud Shell subsystem also provides users with job management capabilities via a command-line interface (CLI). The cloud provider's computing resources include virtual machines, containers, and serverless function resources. The commands submitted via the command-line CLI include the number of tasks, resource metrics, and resource categories. The Cloud Shell subsystem also sends user job submission requests to the Job Manager subsystem via the SSH over WebSocket protocol or as a RESTful API request. The job and resource management subsystem is used to receive the job submission request, schedule the computing resources corresponding to the cloud provider, schedule the programs / instructions that the job needs to execute to run on the corresponding computing resources, monitor the status of job execution at the same time, and release the computing resources after the job program / instruction has finished executing. The job and resource management subsystem includes: a Job Manager subsystem and a Resource Manager subsystem, wherein: The Job Manager subsystem is used to receive the job submission request and send a request to schedule computing resources to the Resource Manager subsystem based on the computing resource parameters in the job submission request. The Resource Manager subsystem is used to start the corresponding computing resources using the cloud vendor's API interface according to the request for scheduling computing resources, so as to create the corresponding computing resources. After the resource creation is completed, the list of corresponding created resource IDs is returned to the Job Manager subsystem. The Job Manager subsystem is also used to schedule the programs / instructions that the job needs to execute to the corresponding computing resources based on the returned computing resource information, and at the same time monitor the status of the job operation. After the job program / instruction finishes execution, it notifies the Resource Manager subsystem to release the computing resources. The Resource Manager subsystem is also used to execute scheduling strategies based on the request for scheduling computing resources, and to determine the cloud vendors, their regions, and the amount of resources that need to be scheduled for this job. The Resource Manager subsystem is also used to call the API of the cloud vendor for the corresponding resource type to create the corresponding computing resources based on the determined cloud vendor, its region, and resource quantity information; After the resource creation is complete, the list of corresponding created resource IDs will be returned to the Job Manager subsystem; The Job Manager subsystem is also used to run the Job Agent subsystem on each computing resource based on the returned computing resource information, and combine the computing resources running the Job Agent subsystem into a computing cluster. After the resource startup is complete, the Job Manager subsystem is also used to schedule the programs / instructions that the job needs to execute to the corresponding computing resources, run them through the Job Agent subsystem, and monitor the status of the job execution at the same time. After the job program / instruction finishes execution, it notifies the Resource Manager subsystem to release the computing resources.
4. A computer storage medium, characterized in that, The computer storage medium stores a computer program, which, when executed by a processor, implements the steps of the method as described in any one of claims 1-2.