A GPU cluster service management system and scheduling method

Abstract

The present invention discloses a GPU cluster service management system and a scheduling method, which relates to the field of computer science and is used to solve the problem of untimely data processing caused by abnormal judgment of important data, collect task resource feature information and GPU feature information, establish a data analysis model, generate a priority evaluation coefficient, and compare and analyze it with a priority threshold to obtain a comparison result, and statistically divide it into a primary processing task and a secondary processing task, and divide it into a primary task group processing time and a remaining task group processing time, and compare it with the required time, adjust the primary task group according to the comparison result, compare the remaining time with the remaining task group, and after dividing the remaining task group, perform processing services until the remaining time shrinks to 0, so that even when it exceeds the user's required time, the divided primary task group and the remaining task group are still screened and the structure is adjusted, so as to realize the maximum number of GPU processing services and meet the basic needs of users.

Description

Technical Field

[0001] The present invention relates to the field of computer science, and more specifically, to a GPU cluster service management system and a scheduling method. Background Art

[0002] GPU cluster service is a cloud service that provides GPU computing resources. Generally, this service allows users to rent GPU clusters to perform tasks that require a large amount of computing resources, such as machine learning training, deep learning model training, large-scale data analysis, etc. The GPU cluster service management system and scheduling method are systems used to manage and schedule computing resources and tasks in GPU clusters. By managing the resources in the GPU cluster, tasks are scheduled according to task priority, resource requirements and the current status of the cluster to maximize the utilization of cluster resources and the execution efficiency of tasks.

[0003] The prior art has the following deficiencies:

[0004] At present, GPU cluster service management systems and scheduling methods are usually applied to big data analysis. Common dynamic scheduling algorithms include preemptive priority-based scheduling. When the application scenario data is huge, a single priority algorithm will cause important data judgment errors, resulting in untimely data processing. When processing huge data, it will also lead to too long processing time, thus affecting user experience. Therefore, a GPU cluster service management system and scheduling method are proposed.

[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not constitute the prior art that is already known to one of ordinary skill in the art. Summary of the invention

[0006] In order to overcome the above-mentioned defects of the prior art, an embodiment of the present invention provides a GPU cluster service management system and a scheduling method, which solves the problems raised in the above-mentioned background technology by using different product inspection methods.

[0007] To achieve the above object, the present invention provides the following technical solution, a GPU cluster service management system and scheduling method, including S1: collecting task resource feature information and GPU feature information, and performing data processing on them to obtain task resource processing time, task waiting time, GPU utilization rate and GPU temperature;

[0008] S2: Obtain task resource feature information and GPU feature information, establish a data analysis model, and generate a priority evaluation coefficient;

[0009] S3: Obtain the priority evaluation coefficient, compare and analyze it with the priority threshold, obtain the comparison result, determine the task priority, and then determine the total number of tasks, and divide them into primary processing tasks and secondary processing tasks;

[0010] S4: Based on the primary processing tasks and the secondary processing tasks, the task resource processing time is obtained, and it is divided into the primary task group processing time and the remaining task group processing time, and the results are compared with the required time to obtain the comparison results, and the primary task group is adjusted according to the comparison results;

[0011] S5: Compare the remaining time with the remaining task groups, divide the remaining task groups, and then perform processing services until the remaining time shrinks to 0.

[0012] In a preferred embodiment, the task resource processing time Pd is estimated by historical data or model prediction based on factors such as the type, size, and complexity of the task when the task is submitted. g , where g is the number of tasks, g = 1, 2, 3, 4, ... M, and M is the total number of tasks;

[0013] The task waiting time is obtained by recording the task submission time when the user submits the task, and subtracting the task submission time from the current time to get the task waiting time Wt. g ;

[0014] GPU utilization Use GPU monitoring tools to obtain GPU utilization Ut g ;

[0015] The GPU temperature is obtained by using the GPU monitoring tool.

[0016] In a preferred embodiment, the task resource processing time Pd g , Task waiting time Wt g 、GPU utilization Ut g And GPU temperature T to establish a data analysis model and generate the priority evaluation coefficient Pr g , based on the formula:

[0017]

[0018] In the formula, Pr g is the priority evaluation coefficient, as well as The processing time Pd for the task resources g , Task waiting time Wt g 、GPU utilization Ut g and a preset scaling factor for the GPU temperature T, and as well as All are greater than 0;

[0019] Compare and analyze the priority evaluation coefficient with the iterative priority threshold;

[0020] If the priority evaluation coefficient is greater than or equal to the priority threshold, the task is marked as a priority task and a priority signal is generated. If the priority evaluation coefficient is less than the priority threshold, the task is marked as a secondary task and a secondary signal is generated.

[0021] In a preferred embodiment, the priority evaluation coefficient is compared and analyzed with the priority threshold value which is continuously iterated;

[0022] If the priority evaluation coefficient is greater than or equal to the priority threshold, the task is marked as a priority task and a priority signal is generated. If the priority evaluation coefficient is less than the priority threshold, the task is marked as a secondary task and a secondary signal is generated.

[0023] Statistical calculation is performed on multiple priority tasks and secondary tasks, the number of each task is calculated, the priority tasks are marked as the first task group, and the secondary tasks are marked as the remaining task group.

[0024] In a preferred embodiment, the task resource processing time Pd g Divide into first-level task groups according to service level and processing time Remaining task group processing time And the processing time of all task groups And obtain the required time Ti set by the user;

[0025] Processing time of all task groups Comparing with the required time Ti, we get:

[0026] If the processing time of all task groups is greater than or equal to the required time, subtract the processing time of the remaining task groups from the processing time of all task groups to obtain the processing time of the first-level task group and compare it with the required time;

[0027] If the processing time of all task groups is less than the required time, all task groups will be processed;

[0028] When the processing time of the first-level task group is less than the required time, the remaining time is obtained. When the processing time of the first-level task group is greater than or equal to the required time, the task structure within the first-level task group is adjusted, and the tasks with higher priority evaluation coefficient values ​​are processed and served, and the remaining tasks within the first-level task group are divided into the remaining classification groups.

[0029] In a preferred embodiment, the remaining time Comparing this with the processing time of the remaining task groups, we get a possibility:

[0030] The remaining task group processing time is greater than the remaining time Then adjust the original remaining task group structure, and according to the above method, process and serve the tasks with higher priority evaluation coefficient values, divide them into a new first-level task group and a new remaining task group, and compare the new first-level task group with the remaining time and process it, then calculate the remaining time again, and so on, until the remaining time is 0, then end the calculation.

[0031] A GPU cluster service management system includes a collection module, a processing module, a priority module, a comparison module, and an output window;

[0032] The acquisition module is used to collect task resource feature information and GPU feature information, obtain task resource processing time, task waiting time, GPU utilization rate and GPU temperature, and send them to the processing module;

[0033] The processing module is used to obtain task resource feature information and GPU feature information, establish a data analysis model, generate a priority evaluation coefficient, and send it to the priority module;

[0034] The priority module is used to obtain the priority evaluation coefficient, compare and analyze it with the priority threshold, obtain the comparison result, and statistically divide it into primary processing tasks and secondary processing tasks and send them to the comparison module, and divide the primary processing tasks and secondary processing tasks again according to the adjustment information and send them to the comparison module;

[0035] The comparison module is used to obtain the task resource processing time based on the first-level processing tasks and the second-level processing tasks, and divide it into the first-level task group processing time and the remaining task group processing time, and compare it with the required time, and send adjustment information to the priority module based on the comparison results; the output window is used to receive the final processing service results and display them.

[0036] Technical effects and advantages of the present invention:

[0037] 1. The present invention obtains the priority evaluation coefficient of the task and compares it with the priority threshold to obtain the priority task and the secondary task, and counts their number, divides them into the primary task group and the remaining task group, and gives priority to the processing service of the primary task group, thereby improving the GPU service efficiency and meeting the user's usage requirements.

[0038] 2. The present invention obtains the task resource processing time and combines it with the first-level task group and the remaining task groups, and compares and analyzes it with the user's required time. In this way, even if it exceeds the user's required time, the divided first-level task group and the remaining task groups can still be screened and the structure adjusted, thereby maximizing the number of services processed by the GPU and meeting the basic needs of users. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a method flow chart of a GPU cluster service scheduling method of the present invention.

[0040] Figure 2 This is a module schematic diagram of a GPU cluster service management system according to the present invention. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] Example 1

[0043] See also Figure 1 ,A GPU cluster service scheduling method, usually, schedules tasks according to task priority, resource requirements and the current state of the cluster to maximize the utilization of cluster resources and the execution efficiency of tasks. The specific operation process is as follows:

[0044] S1: Collect task resource feature information and GPU feature information, and process them into data to obtain task resource processing time, task waiting time, GPU utilization rate and GPU temperature;

[0045] The GPU cluster service will schedule tasks according to the priority of the task, resource requirements and the current status of the cluster to maximize the utilization of cluster resources and the execution efficiency of the task. It is usually scheduled through the dynamic adjustment method priority scheduling. It will determine the execution order of tasks according to their priorities. In priority scheduling, the system will assign a priority to each task. Tasks with high priorities will be executed first, and the priority will also dynamically change the priority evaluation criteria as the historical data set changes;

[0046] Among them, the task resource characteristic information includes the task resource processing time and the task waiting time;

[0047] The logic for obtaining the task resource processing time is to estimate the time required for the task to be executed in the GPU cluster based on factors such as the task type, size, and complexity when the task is submitted, using historical data or model prediction methods;

[0048] It should be noted that the estimation of task execution time usually uses a linear regression model to construct a feature vector based on task size, task type, and resource requirements, and obtain the data set X = [X1, X2, ..., X n ], where X iis the feature of the task; where i is the number of features, i = 1, 2, 3, 4, ... N, and N is the total number of features;

[0049] Construct the task execution time into label y;

[0050] The linear regression model is used to represent the relationship between task execution time and task features: y = X·w+b, where W is the weight vector and b is the bias term;

[0051] Use mean squared error as the loss function:

[0052]

[0053] Use the gradient descent optimization algorithm to minimize the loss function and obtain the optimal weight vector

[0054] w and bias term b;

[0055] The following is a simplified formula for the above steps. Assume that there are m pieces of historical task data, and each piece of task data has n features:

[0056] Eigenvector

[0057] Label Use the trained model to predict the execution time of the task based on the feature data of the task: Thus, the task resource processing time Pd is obtained g , where g is the number of tasks, g = 1, 2, 3, 4, ... M, and M is the total number of tasks;

[0058] It should be noted that the task resource processing time refers to the processing time in the GPU cluster service. Generally, the longer the task processing time is, the larger the GPU capacity occupied by the task is, and the lower the priority is.

[0059] The logic of obtaining the task waiting time is that when the user submits a task, the submission time of the task is recorded. The system serves a task queue and queues the tasks according to their priorities. When a new task is submitted, the task is added to the queue according to its priority. The system selects the next task to be executed from the queue according to the priority of the task in the scheduling method. When the task is selected for execution, the waiting time of the task in the queue is calculated. The waiting time wait_time can be obtained by subtracting the task submission time submit_time from the current time current_time: wait_time = current_time-submit_time, thereby obtaining the task waiting time Wt g ;

[0060] It should be noted that the longer the task waiting time is, the lower the priority is, indicating that the user does not consider the task to be very important.

[0061] The GPU feature information includes GPU utilization and GPU temperature;

[0062] The logic of obtaining GPU utilization is to use GPU monitoring tools to obtain GPU utilization information. Usually, NVIDIA's nvidia-smi tool is used to obtain GPU utilization information. By calling the GPU monitoring tool or API, the current GPU utilization data can be obtained, usually expressed as a percentage. Among them, nvidia-smi is a command tool for monitoring and managing NVIDIA GPU devices. nvidia-smi provides a variety of useful functions, including viewing GPU utilization, temperature, memory usage, driver information, etc. It is commonly used by system administrators, developers, and users to view and manage GPU devices;

[0063] It should be noted that GPU utilization is the utilization when processing the current task. Therefore, GPU utilization is to obtain GPU utilization data periodically according to the task to monitor GPU utilization in real time. g ;

[0064] It should be noted that when the GPU utilization rate is higher, it means that the GPU resources are tight, the processing speed is slower, and tasks with higher priority cannot be assigned;

[0065] The logic of obtaining GPU temperature is to use GPU monitoring tools to obtain GPU temperature information. Usually, NVIDIA's nvidia-smi tool is used to obtain GPU temperature information. By calling the GPU monitoring tool or API, the current GPU temperature T can be obtained.

[0066] It should be noted that the GPU temperature is collected in real time. Real-time collection of GPU temperature can more accurately control the situation where the GPU temperature is too high. I will not go into details here.

[0067] It should be noted that when the GPU is higher, it means that the GPU's task execution efficiency is reduced, or even the hardware may be damaged and cannot be assigned tasks with higher priority;

[0068] S2: Obtain task resource feature information and GPU feature information, establish a data analysis model, and generate a priority evaluation coefficient;

[0069] The task resource processing time Pd g , Task waiting time Wt g 、GPU utilization Utg And GPU temperature T to establish a data analysis model and generate the priority evaluation coefficient Pr g , based on the formula:

[0070] In the formula, Pr g is the priority evaluation coefficient, as well as The processing time Pd for the task resources g , Task waiting time Wt g 、GPU utilization Ut g and a preset scaling factor for the GPU temperature T, and as well as All are greater than 0;

[0071] Among them, the task resource processing time Pd g , Task waiting time Wt g 、GPU utilization Ut g and GPU temperature T are both data-based manifestations of task priority;

[0072] From the formula, we can know that the task resource processing time Pd g , Task waiting time Wt g 、GPU utilization Ut g And when the GPU temperature T is higher, it means that the current task priority is lower and the priority evaluation coefficient is lower;

[0073] S3: Obtain the priority evaluation coefficient, compare and analyze it with the priority threshold, obtain the comparison result, determine the task priority, and then determine the total number of tasks, and divide them into primary processing tasks and secondary processing tasks;

[0074] The priority evaluation coefficient expresses the difference between a single task and the priority threshold. The priority threshold is the ideal GPU state obtained by continuous iteration of the historical data set. In the ideal GPU state, the task resource processing time is the lowest and the GPU utilization is the lowest.

[0075] The logic for obtaining the priority threshold is to collect a set of distributed samples from the historical task database, divide the data set into a training set and a test set, set the evaluation index and clustering algorithm, and in each round of cross-validation, train the model on the training set and evaluate the model performance on the test set. Then, adjust the balance threshold based on the performance of the validation set. Therefore, the priority threshold is constantly updated.

[0076] In the present invention, clustering algorithm is a kind of unsupervised learning algorithm, which is used to divide the task groups in the data set into groups or clusters with similarities; a common one is K-means clustering, which divides the task groups in the data set into K clusters, so that the distance between each task and the center point (center of mass) of its cluster is minimized, and finally the priority of the task is measured by the Euclidean distance, so as to set the priority threshold;

[0077] After obtaining the priority evaluation coefficient, the priority evaluation coefficient is compared and analyzed with the continuously iterated priority threshold;

[0078] If the priority evaluation coefficient is greater than or equal to the priority threshold, the task is marked as a priority task and a priority signal is generated;

[0079] If the priority evaluation coefficient is less than the priority threshold, the task is marked as a secondary task and a secondary signal is generated;

[0080] When multiple priority evaluation coefficient operations are performed to obtain multiple tasks whose priority evaluation coefficients are greater than or equal to the priority threshold, multiple priority tasks are extracted;

[0081] Performing statistical calculations on multiple priority tasks and secondary tasks, calculating the number of each task, marking the priority tasks as the first task group, and marking the secondary tasks as the remaining task group;

[0082] Assume that there are 100 groups of tasks, of which 25 are priority tasks and 75 are secondary tasks determined by the above method. Figure 1 As shown:

[0083] Priority evaluation coefficient Service Level Number of Services ≥ Priority Threshold First-level task force 25 <Priority Threshold Remaining Task Group 75

[0084] Figure 1

[0085] As shown in the figure, for task service priority, the first-level task group is served first, and then the remaining task groups are served, which improves the processing time of important tasks, quickly completes the division of task priorities and services, and thus meets user needs;

[0086] The present invention obtains the priority evaluation coefficient of the task and compares it with the priority threshold to obtain the priority task and the secondary task, and counts their number, divides them into the primary task group and the remaining task group, and gives priority to the processing service of the primary task group, thereby improving the GPU service efficiency and meeting the user's usage requirements.

[0087] Example 2

[0088] In the first embodiment of the present invention, an example is given to illustrate that when facing a huge task set, a priority evaluation coefficient is obtained through task feature information and GPU feature information, so as to evaluate and divide the first-level task group and the remaining task group in the task set, and give priority to the operation strategy of processing services for the first-level task group; however, in the first embodiment, it is only based on an ideal state. When the user puts forward a strict time requirement, the actual user index cannot be met, resulting in a service timeout; in view of the above problem, the second embodiment of the present invention is further refined;

[0089] The above embodiment 1 has collected the task resource processing time Pd of each task g ;

[0090] S4: Based on the primary processing tasks and the secondary processing tasks, obtain the task resource processing time, and divide it into the primary task group processing time and the remaining task group processing time, and compare it with the required time to obtain the comparison result, and adjust the primary task group according to the comparison result; the task resource processing time Pd g Divide into first-level task groups according to service level and processing time And the remaining task group processing time

[0091] Among them, the request duration is the request time Ti proposed by the user;

[0092] It should be noted that the required duration is the required time entered by the user according to his / her own needs. The specific setting range is set based on the fastest processing time of the historical task set as the lower limit. This can be understood to avoid the system misjudgment caused by the user entering too short a time.

[0093] The processing time of the first-level task group And the remaining task group processing time Add up the processing time of all task groups

[0094] It is understandable that the formula for the processing time of all task groups is expressed as Processing time of all task groups Comparing with the required time Ti, we get:

[0095] If the processing time of all task groups is greater than or equal to the required time, subtract the processing time of the remaining task groups from the processing time of all task groups to obtain the processing time of the first-level task group and compare it with the required time;

[0096] If the processing time of all task groups is less than the required time, all task groups will be processed;

[0097] It is understandable that if the processing time of all task groups is equal to the required time, there may be a situation where the current network is poor or the GPU utilization fluctuates, resulting in the actual processing time of all task groups being longer than the required time, which will not be elaborated here;

[0098] When the processing time of the first-level task group is less than the required time, the remaining time is obtained. When the processing time of the first-level task group is greater than or equal to the required time, the task structure within the first-level task group is adjusted, and the tasks with higher priority evaluation coefficients are processed and served, and the remaining tasks within the first-level task group are divided into the remaining classification groups;

[0099] It can be understood that if there are 25 tasks in the first-level task group, the processing time of the first-level task group is 60 minutes, and the required time is 45 minutes, then the first 70% of the tasks in the first-level task group are reclassified as the processing time of the first-level task group;

[0100] It should be noted that the division rule is obtained by dividing the historical task set and the ratio of the required time to the processing time, which can be the top 70% or 60% of the tasks in the first-level task group. The specific implementation method is not limited and will not be repeated here.

[0101] S5: Compare the remaining time with the remaining task groups, divide the remaining task groups, and then perform processing services until the remaining time shrinks to 0;

[0102] The remaining time It indicates the time remaining after the first-level task group has finished processing;

[0103] The remaining time Comparing this with the processing time of the remaining task groups, we get a possibility:

[0104] The remaining task group processing time is greater than the remaining time Then adjust the original structure of the remaining task groups, and according to the above method, process the tasks with the highest priority evaluation coefficients, divide them into a new first-level task group and a new remaining task group, and compare the new first-level task group with the remaining time and process them;

[0105] It can be understood that, consistent with the above method, the division rule is obtained by dividing the historical task set and the ratio of the remaining time to the processing time, which can be the top 20% or 30% of the tasks in the remaining task group. The specific implementation method is not limited and will not be repeated here;

[0106] For example, if the processing time of the adjusted first-level task group is 3.43 minutes and the required time is 8 minutes, the remaining time is 4.57 minutes obtained by subtracting the required time from the processing time of the first-level task group. The processing time of the remaining task group is calculated again to be 5.15 minutes. Since the processing time of the remaining task group of 5.15 minutes is greater than the remaining time of 4.57 minutes, the original remaining task group is divided into a new first-level task group and a new remaining task group. The processing time of the new first-level task group is calculated again to be 3.5 minutes. The new first-level task group is processed and the remaining time is calculated again. This is repeated until the remaining time is 0, at which time the calculation ends.

[0107] The present invention obtains the task resource processing duration and combines it with the primary task group and the remaining task groups, and compares and analyzes it with the user's required time. In this way, even if it exceeds the user's required time, the divided primary task group and the remaining task groups can still be screened and the structure adjusted, thereby maximizing the number of services processed by the GPU and meeting the basic needs of users.

[0108] Example 3

[0109] See also Figure 2 ,A GPU cluster service management system includes a collection module, a processing module, a priority module, a comparison module, and an output window;

[0110] The acquisition module is used to collect task resource feature information and GPU feature information, obtain task resource processing time, task waiting time, GPU utilization rate and GPU temperature, and send them to the processing module;

[0111] The processing module is used to obtain task resource feature information and GPU feature information, establish a data analysis model, generate a priority evaluation coefficient, and send it to the priority module;

[0112] The priority module is used to obtain the priority evaluation coefficient, compare and analyze it with the priority threshold, obtain the comparison result, and statistically divide it into primary processing tasks and secondary processing tasks and send them to the comparison module, and divide the primary processing tasks and secondary processing tasks again according to the adjustment information and send them to the comparison module;

[0113] The comparison module is used to obtain the task resource processing time based on the first-level processing tasks and the second-level processing tasks, and divide it into the first-level task group processing time and the remaining task group processing time, and compare it with the required time, and send adjustment information to the priority module based on the comparison results; the output window is used to receive the final processing service results and display them.

[0114] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0115] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0116] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0117] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0118] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0119] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0120] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0121] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0122] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks or optical disks.

[0123] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A GPU cluster service scheduling method, characterized in that: S1: Collect task resource feature information and GPU feature information, and process them into data to obtain task resource processing time, task waiting time, GPU utilization rate and GPU temperature; The task resource processing time Pdg is estimated by historical data or model prediction based on factors such as the task type, size, and complexity when the task is submitted, where g is the number of tasks, g = 1, 2, 3, 4, ... M, and M is the total number of tasks; The task waiting time is obtained by recording the task submission time when the user submits the task, and subtracting the task submission time from the current time to get the task waiting time Wtg; GPU utilization Use GPU monitoring tools to obtain GPU utilization Utg; GPU temperature: Use the GPU monitoring tool to obtain the current GPU temperature T; S2: Obtain task resource feature information and GPU feature information, establish a data analysis model, and generate a priority evaluation coefficient; The task resource processing time Pdg, task waiting time Wtg, GPU utilization Utg and GPU temperature T are used to establish a data analysis model to generate the priority evaluation coefficient Prg based on the following formula: In the formula, Pr g is the priority evaluation coefficient, as well as The processing time Pd for the task resources g , the preset proportionality coefficient of task waiting time Wtg, GPU utilization Utg, and GPU temperature T, and as well as All are greater than 0; Compare and analyze the priority evaluation coefficient with the iterative priority threshold; If the priority evaluation coefficient is greater than or equal to the priority threshold, the task is marked as a priority task and a priority signal is generated. If the priority evaluation coefficient is less than the priority threshold, the task is marked as a secondary task and a secondary signal is generated. S3: Obtain the priority evaluation coefficient, compare and analyze it with the priority threshold, obtain the comparison result, determine the task priority, and then determine the total number of tasks, and divide them into primary processing tasks and secondary processing tasks; S4: Based on the primary processing tasks and the secondary processing tasks, the task resource processing time is obtained, and it is divided into the primary task group processing time and the remaining task group processing time, and the results are compared with the required time to obtain the comparison results, and the primary task group is adjusted according to the comparison results; S5: Compare the remaining time with the remaining task groups, divide the remaining task groups, and then perform processing services until the remaining time shrinks to 0.

2. The GPU cluster service scheduling method according to claim 1, characterized in that: Compare and analyze the priority evaluation coefficient with the iterative priority threshold; If the priority evaluation coefficient is greater than or equal to the priority threshold, the task is marked as a priority task and a priority signal is generated. If the priority evaluation coefficient is less than the priority threshold, the task is marked as a secondary task and a secondary signal is generated. Statistical calculation is performed on multiple priority tasks and secondary tasks, the number of each task is calculated, the priority tasks are marked as the first task group, and the secondary tasks are marked as the remaining task group.

3. The GPU cluster service scheduling method according to claim 1, characterized in that: The task resource processing time Pd g Divide into first-level task groups according to service level and processing time Remaining task group processing time And the processing time of all task groups And obtain the required time Ti set by the user; Processing time of all task groups Comparing with the required time Ti, we get: If the processing time of all task groups is greater than or equal to the required time, subtract the processing time of the remaining task groups from the processing time of all task groups to obtain the processing time of the first-level task group and compare it with the required time; If the processing time of all task groups is less than the required time, all task groups will be processed; When the processing time of the first-level task group is less than the required time, the remaining time is obtained. When the processing time of the first-level task group is greater than or equal to the required time, the task structure within the first-level task group is adjusted, and the tasks with higher priority evaluation coefficient values ​​are processed and served, and the remaining tasks within the first-level task group are divided into the remaining classification groups.

4. The GPU cluster service scheduling method according to claim 3, characterized in that: The remaining time Comparing this with the processing time of the remaining task groups, we get a possibility: The remaining task group processing time is greater than the remaining time Then adjust the original remaining task group structure, and according to the above method, process and serve the tasks with higher priority evaluation coefficient values, divide them into a new first-level task group and a new remaining task group, and compare the new first-level task group with the remaining time and process it, then calculate the remaining time again, and so on, until the remaining time is 0, then end the calculation.

5. A GPU cluster service management system, used to implement a GPU cluster service scheduling method according to any one of claims 1 to 4, characterized in that: Including acquisition module, processing module, priority module, comparison module and output window; The acquisition module is used to collect task resource feature information and GPU feature information, obtain task resource processing time, task waiting time, GPU utilization rate and GPU temperature, and send them to the processing module; The processing module is used to obtain task resource feature information and GPU feature information, establish a data analysis model, generate a priority evaluation coefficient, and send it to the priority module; The priority module is used to obtain the priority evaluation coefficient, compare and analyze it with the priority threshold, obtain the comparison result, and statistically divide it into primary processing tasks and secondary processing tasks and send them to the comparison module, and divide the primary processing tasks and secondary processing tasks again according to the adjustment information and send them to the comparison module; The comparison module is used to obtain the task resource processing time according to the primary processing tasks and the secondary processing tasks, and divide it into the primary task group processing time and the remaining task group processing time, and compare the statistics with the required time, and send adjustment information to the priority module according to the comparison results; The output window is used to receive the final processing service results and display them.

Images