A scheduling method of a GPU server cluster

Abstract

The application is suitable for the technical field of GPU server, and provides a scheduling method of GPU server cluster.The application receives an uploading task, and generates standard task information through standardization analysis;state monitoring, idle analysis and information recording are performed on multiple GPU servers in the GPU server cluster;comparison and selection are performed on the multiple GPU servers to select a target server;task deployment and environment construction are performed;the uploading task is executed, and real-time monitoring and dynamic adjustment are performed on the target server.The application can perform state monitoring, idle analysis and information recording on the GPU server cluster, comparison and selection on the multiple GPU servers according to the standard task information obtained through standardization analysis, selection of the target server, and then task deployment, environment construction and task execution, and real-time monitoring and dynamic adjustment during the task execution process, thereby effectively solving the problems of static and extensive allocation, low utilization and mutual blocking of tasks.

Description

Technical Field

[0001] This invention belongs to the field of GPU server technology, and in particular relates to a scheduling method for a GPU server cluster. Background Technology

[0002] A GPU server is a high-performance computing server built around a graphics processing unit (GPU) as its core computing resource. It can provide computing efficiency far exceeding that of a CPU in parallel computing scenarios and is particularly suitable for large-scale data processing and complex computing tasks.

[0003] GPU servers typically consist of a CPU, GPU accelerator card, high-speed memory, storage system, and high-speed network interface. The CPU is mainly responsible for scheduling and controlling tasks, while the GPU is responsible for performing large-scale parallel computing, such as matrix operations and vector processing. GPUs have hundreds or thousands of computing cores and can process large amounts of data simultaneously, thus having significant advantages in fields such as artificial intelligence training, deep learning inference, scientific computing, image processing, and video encoding.

[0004] In existing technologies, GPU server cluster scheduling typically employs static allocation or simple round-robin allocation, which can easily lead to some GPU server nodes being overloaded with GPUs while other nodes are idle and wasted. This results in problems such as static and inefficient allocation, low utilization, and a lack of prediction of future resources, which can easily lead to tasks blocking each other. Summary of the Invention

[0005] The purpose of this invention is to provide a scheduling method for a GPU server cluster, which aims to solve the technical problems existing in the prior art mentioned in the background.

[0006] The embodiments of the present invention are implemented as follows:

[0007] A scheduling method for a GPU server cluster, the method specifically includes the following steps:

[0008] Receive uploaded tasks, perform standardized parsing, and generate standard task information;

[0009] Perform status monitoring, idle analysis, and information recording on multiple GPU servers in a GPU server cluster, and obtain cluster resource snapshots;

[0010] Based on the standard task information and the cluster resource snapshot, multiple GPU servers are compared and selected, and a target server is selected from the multiple GPU servers;

[0011] Plan the task deployment time for the target server, and perform task deployment and environment setup during the task deployment time;

[0012] The upload task is executed on the target server, and the target server is monitored and dynamically adjusted in real time during the task execution.

[0013] As a further limitation of the technical solution of this embodiment of the invention, the step of receiving the uploaded task, performing standardized parsing, and generating standard task information specifically includes the following steps:

[0014] Receive upload tasks;

[0015] Perform integrity and permission checks on the uploaded task, and determine whether the checks pass.

[0016] After the verification is passed, the resource requirements of the uploaded task are parsed to generate a resource requirements table;

[0017] According to the resource requirement table, the upload task is standardized and packaged to generate standard task information.

[0018] As a further limitation of the technical solution of the embodiments of the present invention, the standard task information includes: task ID, task type, GPU requirements, memory requirements, model requirements, estimated running time and priority index.

[0019] As a further limitation of the technical solution of this invention, the step of performing status monitoring, idle analysis and information recording on multiple GPU servers in the GPU server cluster to obtain a cluster resource snapshot specifically includes the following steps:

[0020] Obtain multiple raw performance data points corresponding to multiple GPU servers in a GPU server cluster;

[0021] Perform status monitoring on multiple GPU servers in the GPU server cluster and obtain multiple status monitoring data.

[0022] Get queued task information;

[0023] By combining the queuing task information and multiple status monitoring data, idle time analysis is performed on multiple GPU servers to predict the corresponding future resource idle time.

[0024] Based on multiple sets of raw performance data and multiple future resource idle periods, information is recorded for multiple GPU servers to obtain cluster resource snapshots.

[0025] As a further limitation of the technical solution of this embodiment of the invention, the step of comparing and selecting a target server from multiple GPU servers based on the standard task information and the cluster resource snapshot specifically includes the following steps:

[0026] Based on the standard task information and the cluster resource snapshot, select multiple available servers from the multiple GPU servers;

[0027] The operation of multiple available servers is evaluated to obtain multiple corresponding operation scores;

[0028] The target server is selected from the multiple GPU servers by comparing the various performance scores.

[0029] As a further limitation of the technical solution of this embodiment of the invention, the step of planning the task deployment time of the target server and performing task deployment and environment construction during the task deployment time specifically includes the following steps:

[0030] Select the target window period corresponding to the target server from multiple future resource availability windows;

[0031] Based on the target idle period, plan the task deployment time for the target server;

[0032] The task is deployed at the specified deployment time;

[0033] A containerized environment is built on the target server.

[0034] As a further limitation of the technical solution of this embodiment of the invention, the step of executing the upload task on the target server and performing real-time monitoring and dynamic adjustment of the target server during the task execution specifically includes the following steps:

[0035] Perform task initiation control and execute the upload task on the target server;

[0036] During task execution, the target server is monitored in real time to obtain real-time monitoring data;

[0037] The real-time monitoring data is analyzed to determine if any execution anomalies exist;

[0038] In the event of an execution exception, dynamic resource adjustments are made to the target server.

[0039] Compared with the prior art, the beneficial effects of the present invention are:

[0040] This invention, through receiving uploaded tasks and performing standardized parsing to generate standard task information, monitors the status, analyzes idle time, and records information of multiple GPU servers in a GPU server cluster. It then compares and selects a target server from among the multiple GPU servers, deploys the task, builds the environment, and executes the uploaded task, while simultaneously monitoring and dynamically adjusting the target server in real time. This effectively solves the problems of static and inefficient allocation, low utilization, and task blocking. Attached Figure Description

[0041] Figure 1 A flowchart of the method provided in an embodiment of the present invention is shown;

[0042] Figure 2 A flowchart illustrating the generation of standard task information in the method provided by an embodiment of the present invention is shown.

[0043] Figure 3 A flowchart illustrating the method for obtaining a cluster resource snapshot provided in an embodiment of the present invention is shown.

[0044] Figure 4 A flowchart illustrating the comparison and selection of multiple GPU servers in the method provided by an embodiment of the present invention is shown.

[0045] Figure 5 A flowchart of task deployment and environment construction in the method provided by an embodiment of the present invention is shown.

[0046] Figure 6 A flowchart illustrating task execution and monitoring adjustments in the method provided by an embodiment of the present invention is shown. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0048] Understandably, in existing technologies, GPU server cluster scheduling typically employs static allocation or simple round-robin allocation, which can easily lead to some GPU server nodes being overloaded with GPUs while other nodes are idle and wasting resources. This results in problems such as static and inefficient allocation, low utilization, and a lack of prediction of future resources, which can easily lead to tasks blocking each other.

[0049] To address the aforementioned issues, this invention receives uploaded tasks, performs standardized parsing to generate standard task information, monitors the status of multiple GPU servers in a GPU server cluster, analyzes their idle time, and records information to obtain a cluster resource snapshot, then compares and selects a target server based on the standard task information and the cluster resource snapshot, plans the task deployment time for the target server, and performs task deployment and environment setup during the deployment time, executing the uploaded task on the target server, and monitoring and dynamically adjusting the target server in real time during task execution. This approach effectively solves the problems of static and inefficient allocation, low utilization, and task blocking by multiple GPU servers by receiving uploaded tasks, performing standardized parsing of standard task information, comparing and selecting a target server, deploying tasks, setting up environments, and executing tasks, and monitoring and dynamically adjusting tasks in real time during execution.

[0050] Specifically, Figure 1 A flowchart of the method provided by an embodiment of the present invention is shown.

[0051] In a preferred embodiment of the present invention, a scheduling method for a GPU server cluster specifically includes the following steps:

[0052] Step S101: Receive the uploaded task, perform standardized parsing, and generate standard task information.

[0053] In this embodiment of the invention, an upload task is received, and then the integrity and permission of the upload task are verified. If the verification is successful, the resource requirements of the upload task are parsed to generate a resource requirement table. Then, according to the resource requirement table, the upload task is standardized and encapsulated to generate standard task information for the upload task. The standard task information generated by encapsulation includes information such as the upload task ID, task type, GPU requirements, memory requirements, model requirements, estimated running time, and priority index.

[0054] Specifically, Figure 2 A flowchart illustrating the generation of standard task information in the method provided by an embodiment of the present invention is shown.

[0055] In another preferred embodiment of the present invention, the step of receiving the upload task and performing standardized parsing to generate standard task information specifically includes the following steps:

[0056] Step S1011: Receive upload task;

[0057] Step S1012: Perform integrity and permission verification on the uploaded task, and determine whether the verification passes;

[0058] Step S1013: After the verification is passed, the resource requirements of the uploaded task are parsed to generate a resource requirements table;

[0059] Step S1014: According to the resource requirement table, the upload task is standardized and packaged to generate standard task information.

[0060] Furthermore, the scheduling method for the GPU server cluster also includes the following steps:

[0061] Step S102: Perform status monitoring, idle analysis and information recording on multiple GPU servers in the GPU server cluster, and obtain a snapshot of cluster resources.

[0062] In this embodiment of the invention, multiple GPU servers in a GPU server cluster are identified, and multiple raw performance data corresponding to the multiple GPU servers are obtained. At the same time, the status of the multiple GPU servers in the GPU server cluster is monitored, and multiple status monitoring data corresponding to the multiple GPU servers are obtained. Queuing task information is also obtained. By combining the queuing task information and multiple status monitoring data, idle time analysis is performed on the multiple GPU servers to predict the future resource idle time corresponding to the multiple GPU servers. Then, based on the multiple raw performance data and multiple future resource idle time, information is recorded for the multiple GPU servers to obtain a cluster resource snapshot.

[0063] Specifically, Figure 3 A flowchart illustrating the method for obtaining a cluster resource snapshot provided in an embodiment of the present invention is shown.

[0064] In yet another preferred embodiment of the present invention, the step of performing status monitoring, idle analysis, and information recording on multiple GPU servers in a GPU server cluster to obtain a cluster resource snapshot specifically includes the following steps:

[0065] Step S1021: Obtain multiple raw performance data corresponding to multiple GPU servers in the GPU server cluster;

[0066] Step S1022: Perform status monitoring on multiple GPU servers in the GPU server cluster and obtain multiple status monitoring data.

[0067] Step S1023: Obtain queued task information;

[0068] Step S1024: Combining the queued task information and multiple status monitoring data, perform idle analysis on multiple GPU servers to predict the corresponding future resource idle periods;

[0069] Step S1025: Based on the multiple original performance data and multiple future resource idle periods, record information for multiple GPU servers to obtain a cluster resource snapshot.

[0070] Furthermore, the scheduling method for the GPU server cluster also includes the following steps:

[0071] Step S103: Based on the standard task information and the cluster resource snapshot, compare and select from the multiple GPU servers to select the target server.

[0072] In this embodiment of the invention, by analyzing standard task information and cluster resource snapshots, multiple available servers capable of executing upload tasks are selected from multiple GPU servers. The multiple available servers are evaluated based on the principles of minimum load priority, fastest release priority, and data locality priority to obtain multiple corresponding running scores. The multiple running scores are then compared, and the target server with the highest running score is selected from the multiple GPU servers.

[0073] Specifically, Figure 4 A flowchart illustrating the comparison and selection of multiple GPU servers in the method provided by an embodiment of the present invention is shown.

[0074] In another preferred embodiment of the present invention, the step of comparing and selecting a target server from the multiple GPU servers based on the standard task information and the cluster resource snapshot specifically includes the following steps:

[0075] Step S1031: Based on the standard task information and the cluster resource snapshot, select multiple available servers from the multiple GPU servers;

[0076] Step S1032: Evaluate the operation of the multiple available servers and obtain multiple corresponding operation scores;

[0077] Step S1033: Compare the multiple running scores and select the target server from the multiple GPU servers.

[0078] Furthermore, the scheduling method for the GPU server cluster also includes the following steps:

[0079] Step S104: Plan the task deployment time of the target server, and perform task deployment and environment construction during the task deployment time.

[0080] In this embodiment of the invention, a target window corresponding to the target server is selected from multiple future resource window periods. Then, within the target window period, the task deployment time of the target server is planned. After the task deployment time is reached, the upload task is deployed. On the target server, the specified image is pulled, the dataset is mounted, environment variables are set, and GPU resources are allocated to realize the construction of a containerized environment.

[0081] Specifically, Figure 5 A flowchart of task deployment and environment construction in the method provided by an embodiment of the present invention is shown.

[0082] In another preferred embodiment of the present invention, the step of planning the task deployment time of the target server and performing task deployment and environment construction during the task deployment time specifically includes the following steps:

[0083] Step S1041: Select the target window period corresponding to the target server from the multiple future resource window periods;

[0084] Step S1042: Based on the target idle period, plan the task deployment time of the target server;

[0085] Step S1043: Deploy the task at the specified task deployment time;

[0086] Step S1044: Build a containerized environment on the target server.

[0087] Furthermore, the scheduling method for the GPU server cluster also includes the following steps:

[0088] Step S105: On the target server, execute the upload task, and during the task execution process, monitor and dynamically adjust the target server in real time.

[0089] In this embodiment of the invention, the upload task is started and controlled. The upload task is executed on the target server. During the execution of the upload task, the target server is monitored in real time to obtain real-time monitoring data. By analyzing the real-time monitoring data, it is determined whether there are execution anomalies such as low GPU utilization or slow data loading. If an execution anomaly is determined to exist, dynamic resource adjustments are made to the target server.

[0090] Specifically, Figure 6 A flowchart illustrating task execution and monitoring adjustments in the method provided by an embodiment of the present invention is shown.

[0091] In another preferred embodiment of the present invention, the step of executing the upload task on the target server and performing real-time monitoring and dynamic adjustment of the target server during the task execution specifically includes the following steps:

[0092] Step S1051: Perform task startup control and execute the upload task on the target server;

[0093] Step S1052: During the task execution process, the target server is monitored in real time to obtain real-time monitoring data;

[0094] Step S1053: Analyze the real-time monitoring data to determine if there are any execution anomalies;

[0095] Step S1054: When an execution exception occurs, dynamically adjust the resources of the target server.

[0096] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A scheduling method for a GPU server cluster, characterized in that, The method specifically includes the following steps: Receive uploaded tasks, perform standardized parsing, and generate standard task information; Perform status monitoring, idle analysis, and information recording on multiple GPU servers in a GPU server cluster, and obtain cluster resource snapshots; Based on the standard task information and the cluster resource snapshot, multiple GPU servers are compared and selected, and a target server is selected from the multiple GPU servers; Plan the task deployment time for the target server, and perform task deployment and environment construction during the task deployment time; The upload task is executed on the target server, and the target server is monitored and dynamically adjusted in real time during the task execution.

2. The scheduling method for a GPU server cluster according to claim 1, characterized in that, The process of receiving uploaded tasks, performing standardized parsing, and generating standard task information specifically includes the following steps: Receive upload tasks; Perform integrity and permission checks on the uploaded task, and determine whether the checks pass. After the verification is passed, the resource requirements of the uploaded task are parsed to generate a resource requirements table; According to the resource requirement table, the upload task is standardized and packaged to generate standard task information.

3. The scheduling method for a GPU server cluster according to claim 2, characterized in that, The standard task information includes: task ID, task type, GPU requirements, memory requirements, model requirements, estimated runtime, and priority index.

4. The scheduling method for a GPU server cluster according to claim 1, characterized in that, The process of monitoring the status of multiple GPU servers in a GPU server cluster, analyzing their idle time, recording information, and obtaining a snapshot of cluster resources specifically includes the following steps: Obtain multiple raw performance data points corresponding to multiple GPU servers in a GPU server cluster; Perform status monitoring on multiple GPU servers in the GPU server cluster and obtain multiple status monitoring data. Get queued task information; By combining the queuing task information and multiple status monitoring data, idle time analysis is performed on multiple GPU servers to predict the corresponding future resource idle time. Based on multiple sets of raw performance data and multiple future resource idle periods, information is recorded for multiple GPU servers to obtain cluster resource snapshots.

5. The scheduling method for a GPU server cluster according to claim 1, characterized in that, The step of comparing and selecting a target server from multiple GPU servers based on the standard task information and the cluster resource snapshot specifically includes the following steps: Based on the standard task information and the cluster resource snapshot, select multiple available servers from the multiple GPU servers; The operation of multiple available servers is evaluated to obtain multiple corresponding operation scores; The target server is selected from the multiple GPU servers by comparing the various performance scores.

6. The scheduling method for a GPU server cluster according to claim 4, characterized in that, The process of planning the task deployment time for the target server and performing task deployment and environment construction during the task deployment time specifically includes the following steps: Select the target window period corresponding to the target server from multiple future resource availability windows; Based on the target idle period, plan the task deployment time for the target server; The task is deployed at the specified deployment time. A containerized environment is built on the target server.

7. The scheduling method for a GPU server cluster according to claim 1, characterized in that, The process of executing the upload task on the target server and performing real-time monitoring and dynamic adjustment of the target server during task execution specifically includes the following steps: Perform task initiation control and execute the upload task on the target server; During task execution, the target server is monitored in real time to obtain real-time monitoring data; The real-time monitoring data is analyzed to determine if any execution anomalies exist; In the event of an execution exception, dynamic resource adjustments are made to the target server.

Images