GPU (Graphics Processing Unit) resource scheduling optimization method and device based on hybrid model and medium

Abstract

The invention provides a GPU resource scheduling optimization method and device based on a hybrid model and a medium, and the method comprises the steps: receiving a plurality of new tasks, and obtaining a target resource demand and a target priority of any new task; obtaining various current performance indexes of the GPU resources, and storing the various performance indexes as target time sequence data; calling a CNN-LSTM hybrid model deployed in a production environment, and performing prediction learning on the target time sequence data to obtain a prediction result; selecting the most suitable target scheduling strategy according to the prediction result, various performance indexes and the target priority of any new task; according to the target scheduling strategy and the target resource demand of any new task, GPU resource allocation is carried out on any new task; the task execution condition and the GPU resource use condition are monitored in real time, and a monitoring result is fed back; and according to a monitoring result, dynamically adjusting a currently used target scheduling strategy so as to realize adaptive optimization of GPU resources.

Description

Technical Field

[0001] The present application relates to the technical field of data monitoring and analysis, and in particular to a hybrid model-based GPU resource scheduling optimization method, device and medium. Background Art

[0002] With the rapid development of deep learning and high-performance computing, GPU has become a key computing resource, playing a vital role in the establishment of intelligent computing centers.

[0003] However, traditional GPU resource management systems mainly rely on static scheduling strategies, such as priority scheduling and load balancing. These methods cannot effectively cope with changes in dynamic loads and complex tasks, leading to problems such as uneven resource utilization, task processing delays, and insufficient prediction accuracy. In addition, traditional load prediction methods are mostly based on simple statistical analysis or linear regression, which makes it difficult to capture complex load change trends. Summary of the invention

[0004] The embodiment of the present application provides a GPU resource scheduling optimization method, device and medium based on a hybrid model to solve the problems existing in the related technologies. The technical solution is as follows:

[0005] In a first aspect, an embodiment of the present application provides a GPU resource scheduling optimization method based on a hybrid model, comprising:

[0006] Receive several new tasks and obtain the target resource requirements and target priority of any new task;

[0007] Obtain various current performance indicators of GPU resources, and store the various performance indicators as target time series data;

[0008] Calling the CNN-LSTM hybrid model deployed in the production environment to perform prediction learning on the target time series data to obtain a prediction result, wherein the prediction result includes future task load and resource demand of GPU resources;

[0009] Selecting the most appropriate target scheduling strategy according to the prediction result, the various performance indicators and the target priority of any new task;

[0010] Allocate GPU resources for any new task according to the target scheduling strategy and the target resource requirement of any new task;

[0011] Monitor task execution and GPU resource usage in real time and provide feedback on monitoring results;

[0012] According to the monitoring results, the currently used target scheduling strategy is dynamically adjusted to achieve adaptive optimization of GPU resources.

[0013] In one embodiment, obtaining the target resource requirement and target priority of any new task includes:

[0014] Referring to historical running data of a designated task, estimating resource requirements of any new task to obtain the target resource requirements, wherein the designated task is a task of the same task type as that of any new task;

[0015] According to the relevant static factors of any new task, the priority of any new task is pre-assigned to obtain the target priority.

[0016] In one implementation, obtaining various current performance indicators of GPU resources and storing the various performance indicators as target time series data includes:

[0017] Collect various first performance indicators of GPU resources at the current system level;

[0018] Collect various secondary performance indicators of the current GPU level of GPU resources;

[0019] According to the various first performance indicators and the various second performance indicators, the various performance indicators are obtained;

[0020] Relevant performance indicators are periodically extracted from the various first performance indicators and the various second performance indicators and stored as the target time series data.

[0021] In one embodiment, the CNN-LSTM hybrid model is obtained through the following training process:

[0022] Collect historical time series data of GPU resources;

[0023] Preprocessing the historical time series data to obtain processed historical time series data;

[0024] Split the processed historical event sequence data into a training set, a validation set, and a test set;

[0025] Build a CNN-LSTM model architecture and initialize the hyperparameters of the CNN-LSTM model architecture and Adam optimizer;

[0026] Input the training set into the CNN-LSTM model architecture for forward propagation, and output the final prediction result;

[0027] Based on the loss function, performing loss calculation according to the final prediction result and the training set to obtain a loss value;

[0028] According to the loss value, the gradient of the loss function with respect to the model parameters of the CNN-LSTM model architecture is calculated, and the gradient is back-propagated to update the model parameters, and the model parameters are adjusted by the Adam optimizer to obtain an initial CNN-LSTM model;

[0029] Using the validation set to perform performance evaluation on the initial CNN-LSTM model, and adjusting model parameters of the initial CNN-LSTM model;

[0030] The test set is used to test the initial CNN-LSTM model after adjusting the model parameters. When it is judged according to the test result that the accuracy of the initial CNN-LSTM model reaches above a preset value, the initial CNN-LSTM model is used as the CNN-LSTM model.

[0031] In one embodiment, the CNN-LSTM hybrid model is deployed in a production environment in the following manner:

[0032] Save the CNN-LSTM hybrid model as a target file;

[0033] Load the target file, obtain and deploy the CNN-LSTM hybrid model as an API service;

[0034] Call the Docker container to package the API service and deploy it to the production environment.

[0035] In one embodiment, selecting the most appropriate target scheduling strategy according to the prediction result, the various performance indicators, and the target priority of any new task includes:

[0036] If it is determined that the target priorities of any new task are significantly different, then a priority scheduling strategy is selected as the target scheduling strategy, wherein the priority scheduling strategy is used to indicate that tasks are allocated according to task priorities;

[0037] If it is determined according to the various performance indicators that the load differences between the GPUs are large, a load balancing scheduling strategy is selected as the target scheduling strategy, where the load balancing scheduling strategy is used to indicate that tasks are evenly distributed to each GPU;

[0038] If it is determined according to the prediction result that the future task load fluctuation of the GPU resources is large, the prediction scheduling strategy is selected as the target scheduling strategy, and the prediction scheduling strategy is used to indicate the dynamic adjustment of task allocation according to the prediction result.

[0039] In one implementation, dynamically adjusting the currently used scheduling strategy according to the monitoring result to achieve adaptive optimization of GPU resources includes:

[0040] If the target scheduling strategy is currently the priority scheduling strategy, when it is determined according to the monitoring result that the load of some GPUs reaches a preset upper limit and the load of the remaining GPUs is lower than a preset lower limit, the target scheduling strategy is switched to the load balancing scheduling strategy to achieve adaptive optimization of GPU resources;

[0041] If the target scheduling strategy is currently the load balancing scheduling strategy, when it is determined according to the monitoring result that the response time of the high-priority task increases, the target scheduling strategy is switched to the predictive scheduling strategy to achieve adaptive optimization of GPU resources;

[0042] If the target scheduling strategy is currently the predicted scheduling strategy, when it is determined according to the monitoring result that the switching condition of the preset scenario is currently met, the target scheduling strategy is switched to the scheduling strategy corresponding to the preset scenario to achieve adaptive optimization of GPU resources.

[0043] In a second aspect, the embodiment of the present application further provides a GPU resource scheduling optimization device based on a hybrid model, comprising:

[0044] Task queue, used to store tasks and manage task priorities;

[0045] Resource manager, used to release GPU resources, allocate GPU resources, and monitor GPU resources;

[0046] Monitoring system, used to monitor GPU status, task status, network status, system performance and storage status;

[0047] A scheduler is connected to the task queue, the resource manager and the monitoring system, and is used to implement the method in any one of the implementation modes of the first aspect above.

[0048] In a third aspect, an embodiment of the present application further provides a computer device, comprising: a memory and a processor, wherein the memory stores instructions, and the instructions are loaded and executed by the processor to implement the method in any one of the embodiments of the first aspect above, wherein the memory and the processor communicate with each other through an internal connection path.

[0049] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. When the computer program runs on a computer, the method in any one of the embodiments of the first aspect described above is implemented.

[0050] The advantages or beneficial effects of the above technical solution include at least:

[0051] By introducing the CNN-LSTM hybrid model, this application can combine deep learning technology for precise load prediction, thereby achieving accurate GPU load prediction, thereby improving the intelligence level and prediction accuracy of the system. At the same time, by adopting a variety of scheduling strategies, the scheduling strategy can be flexibly and dynamically adjusted in real time, which can significantly improve resource utilization efficiency and reduce task processing delays, thereby achieving the purpose of optimizing energy efficiency and reducing operating costs.

[0052] In summary, the hybrid model-based GPU resource scheduling optimization method provided in the embodiments of the present application can improve the flexibility and intelligence of the system while optimizing energy efficiency and reducing operating costs.

[0053] The above summary is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments and features described above, further aspects, embodiments and features of the present application will be readily apparent by reference to the accompanying drawings and the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the multiple drawings represent the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments disclosed in the present application and should not be regarded as limiting the scope of the present application.

[0055] Figure 1 A flowchart of a hybrid model-based GPU resource scheduling optimization method provided in an embodiment of the present application;

[0056] Figure 2 A structural block diagram of a GPU resource scheduling optimization device based on a hybrid model provided in an embodiment of the present application;

[0057] Figure 3 A structural block diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0058] In the following, only some exemplary embodiments are briefly described. As those skilled in the art will appreciate, the described embodiments may be modified in various ways without departing from the spirit or scope of the present application. Therefore, the drawings and descriptions are considered to be exemplary and non-restrictive in nature.

[0059] Various technical solutions provided by the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0060] Before introducing the embodiments of the present application, some terms in the present application are first explained to facilitate understanding by those skilled in the art.

[0061] CNN-LSTM hybrid model: It combines Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) to process complex time series data with spatial and temporal characteristics to improve prediction accuracy.

[0062] Forward propagation: refers to the process in which the input data is calculated through each layer and passed to the output layer in the neural network to generate the final prediction result.

[0063] Backpropagation: refers to the process of adjusting network weights to reduce prediction errors by calculating the gradient of the loss function when training a neural network.

[0064] Convolutional layer: A layer in a convolutional neural network that extracts local features from the input data through local receptive fields and convolution kernels.

[0065] The following will be combined Figure 1 The GPU resource scheduling optimization method based on the hybrid model is explained in detail.

[0066] Figure 1 FIG. 1 is a flowchart of a hybrid model-based GPU resource scheduling optimization method according to an embodiment of the present application. Figure 1 As shown, the method may include the following steps:

[0067] S110, receiving several new tasks, and obtaining the target resource requirement and target priority of any new task.

[0068] In one embodiment, several new tasks may be received from a task queue. After receiving several new tasks, resource requirements and priorities of any new task, ie, target resource requirements and target priority, may be obtained.

[0069] In one implementation, the implementation process of step S110 may include the following steps:

[0070] S111. Referring to the historical operation data of the specified task, the resource requirement of any new task is estimated to obtain the target resource requirement.

[0071] The designated task may be a task similar to any new task, for example, a task of the same task type as any new task.

[0072] As an example, the historical operation data may include, but is not limited to: historical execution time of a specified task, GPU utilization, and video memory occupancy.

[0073] S112. Pre-assign a priority level to any new task based on relevant static factors of any new task to obtain a target priority level.

[0074] As an example, when any new task is created, the priority of the new task may be pre-assigned according to static factors related to the new task.

[0075] In a specific implementation, the relevant static factors may include, but are not limited to: the characteristics, importance and urgency of any new task and user setting information.

[0076] In the embodiment of the present application, by executing step S110, it is possible to facilitate the subsequent scheduling of GPU resources for new tasks based on the resource requirements and priority of any new task.

[0077] S120: Obtain various current performance indicators of GPU resources, and store the various performance indicators as time series data.

[0078] In one implementation, the implementation process of step S120 may include the following steps:

[0079] S121. Collect various first performance indicators of the current system level of GPU resources.

[0080] In specific implementation, the Node Exporter component may be called to collect various first performance indicators of the current system level of GPU resources, wherein the various first performance indicators may include: performance indicators such as GPU, memory, and disk usage.

[0081] In an embodiment of the present application, by executing step S121, necessary background data can be provided for GPU resource scheduling. This is because these system-level first performance indicators can help identify and analyze potential performance bottlenecks, resource consumption, and system stability.

[0082] S122. Collect various second performance indicators of the current GPU level of the GPU resources.

[0083] In specific implementation, the DCGM Exporter component can be called to collect various second performance indicators of the current GPU level of the GPU resource. In this way, the DCGM Exporter component can provide GPU performance data, which helps to identify potential problems in GPU use, such as overheating, abnormal power consumption, or resource waste. Among them, the various second performance indicators may include but are not limited to: GPU load, temperature, and fan speed.

[0084] In an embodiment of the present application, by executing step S122, fine-grained GPU-level second performance indicators can be obtained. These performance indicators are crucial for optimizing task scheduling, improving GPU resource utilization efficiency, extending hardware life, and avoiding performance bottlenecks.

[0085] S123. Obtain various performance indicators according to various first performance indicators and various second performance indicators.

[0086] It can be understood that the various performance indicators include various first performance indicators and various second performance indicators. That is, the various performance indicators may include but are not limited to: the load of each GPU and the status of the task being executed.

[0087] S124. Regularly extract relevant performance indicators from various first performance indicators and various second performance indicators, and store them as target time series data.

[0088] In the specific implementation, Prometheus (an open source service monitoring system and time series database) can be called to regularly pull relevant performance indicators from the indicator data sources collected by Node Exporter and DCGM Exporter and store them as target time series data.

[0089] In an embodiment of the present application, by executing step S124, a time series database can be formed based on the target time series data. The time series database has unique advantages in processing and storing continuous time data, and can effectively improve the accuracy of prediction and the overall performance of the system.

[0090] In an embodiment of the present application, by executing the above step S120, a time series database can be formed based on the obtained relevant performance indicators to support accurate load prediction using a deep learning model.

[0091] S130. Call the CNN-LSTM hybrid model deployed in the production environment to perform prediction learning on the target time series data to obtain a prediction result.

[0092] In one implementation, the prediction result may include, but is not limited to: future task load and resource demand of GPU resources.

[0093] In one embodiment, the CNN-LSTM hybrid model can be obtained through the following training process:

[0094] S131. Collect historical time series data of GPU resources.

[0095] In a specific implementation, the historical time series data may include, but is not limited to: various historical performance indicators of GPU resources, such as GPU load, temperature, fan speed, etc.

[0096] As an example, part of the historical time series data can be shown in Table 1 below.

[0097] Table 1

[0098] Timestamp CPU load (%) Temperature(℃) Fan speed (RPM) 2024-08-13 10:00:00 45 55 1500 2024-08-13 10:01:00 50 60 2000 2024-08-13 10:02:00 60 70 2500

[0099] S132. Preprocess the historical time series data to obtain processed historical time series data.

[0100] In specific implementation, the historical time series data can be normalized to scale the historical time series data to the range of 0-1, so as to improve the stability of CNN-LSTM hybrid model training.

[0101] As an example, after the data in Table 1 above is normalized, the data shown in Table 2 below can be obtained.

[0102] Table 2

[0103]

[0104]

[0105] In a specific implementation, a sliding window may be created to predict the load at the next time point, wherein the window size may be set to 1. Then, the normalized historical time series data may be windowed based on the sliding window to obtain the processed historical time series data.

[0106] It can be understood that the processed historical time series data is obtained by preprocessing the historical time series data according to normalization and sliding window format.

[0107] As an example, after the data in the above Table 2 is processed in a sliding window format, the data shown in the following Table 3 can be obtained.

[0108] Table 3

[0109]

[0110] S133, dividing the processed historical event sequence data into a training set, a validation set and a test set.

[0111] In specific implementation, the processed historical event sequence data can be divided into a training set, a validation set, and a test set at a ratio of 70%, 15%, and 15%. It can be understood that the training set consists of 70% of the processed historical event sequence data, the validation set consists of 15% of the processed historical event sequence data, and the test set consists of 15% of the processed historical event sequence data.

[0112] S134. Build the CNN-LSTM model architecture and initialize the hyperparameters of the CNN-LSTM model architecture and Adam optimizer.

[0113] In a specific implementation, the CNN-LSTM model architecture may include, but is not limited to: an input layer, a convolution layer, a pooling layer, a fully connected layer, and an output layer, wherein the convolution layer, the pooling layer, and the fully connected layer are hidden layers.

[0114] In specific implementation, when initializing the CNN-LSTM model architecture, CNN parameters, LSTM parameters and fully connected layer parameters can be initialized. Among them, CNN parameters may include but are not limited to: convolution kernel weights and convolution kernel biases. LSTM parameters may include but are not limited to: LSTM unit weights and LSTM unit biases. Fully connected layer parameters may include but are not limited to: fully connected layer weights and fully connected layer biases.

[0115] In the specific implementation, the hyperparameters of the Adam optimizer can be initialized as follows: the learning rate is set to 0.001. The learning rate can be adjusted through experiments, the first-order moment attenuation coefficient β1 is set to 0.9, the second-order moment attenuation coefficient β2 is set to 0.999, and ∈ is set to 10 -8 .

[0116] As an example, the above-mentioned correlation weights may be initialized using the He initialization method. For example, the above-mentioned correlation weights may be initialized using the following formula (1).

[0117]

[0118] Among them, W represents weight, N represents normal distribution, n1 represents the number of neurons in the first layer, and the initialization value of the bias is 0.

[0119] S135. Input the training set into the CNN-LSTM model architecture for forward propagation and output the final prediction result.

[0120] In specific implementation, the training set can be first input into the input layer, and then passed to the next hidden layer for forward propagation, so that each hidden layer performs a series of calculations and transformations on the data in the training set, and finally the output layer outputs the final prediction result.

[0121] As an example, in the above forward propagation process, the convolution layer can perform a convolution operation on the input data (the data in the training set) through the convolution kernel. For example, the convolution kernel performs a dot product operation with the input data, and then adds a bias term. The result is passed through an activation function to generate the output of the convolution layer. The specific calculation formula can be shown in the following formula (2).

[0122]

[0123] The representative meanings of the parameters in formula (2) are shown in Table 4 below:

[0124] Table 4

[0125]

[0126]

[0127] As an example, in the above forward propagation process, the pooling layer can perform a dimensionality reduction operation on the output of the convolutional layer by calculating the maximum value (maximum pooling) within the pooling window to reduce the size of the data. The specific calculation formula can be shown as the following formula (3).

[0128] Y i,j,k =max(X i:i+P-1,j:j+P-1,k ) (3)

[0129] Among them, P is the size of the pooling window, and the representative meanings of other parameters are defined as shown in Table 5 below:

[0130] Table 5

[0131]

[0132] As an example, in the above forward propagation process, the fully connected layer can perform a linear transformation on the pooling layer input data, and then add a bias term to map the pooling layer input data to a new space through matrix multiplication. The specific calculation formula can be shown as the following formula (4).

[0133] Z=W·X+b (4)

[0134] Among them, W is the weight matrix, X is the input data, b is the bias term, and Z is the output of the fully connected layer.

[0135] In the above forward propagation process, an activation function can be used to perform a linear transformation on the output of any hidden layer to introduce nonlinear factors so that the network can learn complex patterns. For example, the activation function can be a ReLU activation function, and its expression can be shown in the following formula (5).

[0136] ReLU(x)=max(0,x) (5)

[0137] Where x is the output of any hidden layer.

[0138] After the above forward propagation, the output layer can output the final prediction result.

[0139] S136. Based on the loss function, the loss is calculated according to the final prediction result and the training set to obtain a loss value.

[0140] In specific implementation, based on the loss function, the loss calculation can be performed according to the predicted value in the final prediction result and the true value in the training set, for example, the MSE mean square error calculation is performed to obtain the loss value. The loss function can be shown in the following formula (6).

[0141]

[0142] Among them, MSE is the loss value, n is the number of samples, and y i is the true value of the i-th sample, is the predicted value of the ith sample.

[0143] S137. According to the loss value, the gradient of the loss function with respect to the model parameters of the CNN-LSTM model architecture is calculated, and the gradient is back-propagated to update the model parameters. At the same time, the Adam optimizer is used to adjust the model parameters to obtain the initial CNN-LSTM model.

[0144] In an embodiment of the present application, by using the Adam optimizer to adjust the model parameters, the model parameters can be effectively updated and optimized, and the loss can be reduced, so that the initial CNN-LSTM model can converge to the minimum value of the loss function faster and more accurately, thereby improving the prediction performance of the initial CNN-LSTM model.

[0145] S138. Use the validation set to evaluate the performance of the initial CNN-LSTM model and adjust the model parameters of the initial CNN-LSTM model.

[0146] In specific implementation, after each training cycle, the validation set can be used to evaluate the performance of the initial CNN-LSTM model, record the loss and evaluation indicators, and then adjust the model parameters of the initial CNN-LSTM model based on the loss and evaluation indicators.

[0147] S139, using the test set to test the initial CNN-LSTM model after adjusting the model parameters, and when it is determined according to the test result that the accuracy of the initial CNN-LSTM model reaches a preset value (for example, 90%) or more, the initial CNN-LSTM model is used as the CNN-LSTM model. At this point, the training is stopped.

[0148] In one implementation, after training is completed, the CNN-LSTM hybrid model can be deployed to a production environment for actual prediction tasks.

[0149] In one implementation, the CNN-LSTM hybrid model can be deployed in a production environment in the following ways:

[0150] The first step is to save the CNN-LSTM hybrid model as a target file.

[0151] In the specific implementation, the CNN-LSTM hybrid model can be saved as a target file through the deep learning framework TensorFlow so that it can be loaded during deployment. In this way, the CNN-LSTM hybrid model can be easily deployed to the production environment after training is completed, and the persistence and reusability of the CNN-LSTM hybrid model can be ensured.

[0152] In an embodiment of the present application, by persisting the model parameters of the CNN-LSTM hybrid model, resource waste due to repeated training is avoided, and direct loading and use of the CNN-LSTM hybrid model in different environments is supported.

[0153] The second step is to load the target file, obtain and deploy the CNN-LSTM hybrid model as an API service.

[0154] In an embodiment of the present application, by deploying the CNN-LSTM hybrid model as an API service, other applications or other services can call the CNN-LSTM hybrid model for prediction through HTTP requests.

[0155] The third step is to call the Docker container to package the API service and deploy it to the production environment.

[0156] In an embodiment of the present application, by calling a Docker container to package the API service and deploying it in a production environment, the application can be lightweight, portable, consistent, and efficiently managed, thereby ensuring consistent performance and behavior when running in different environments.

[0157] In one implementation, a CNN-LSTM hybrid model deployed in a production environment is called to perform prediction learning on target time series data to obtain a prediction result.

[0158] In the embodiment of the present application, by executing step S130, the feature extraction capability of CNN (extracting key change patterns from historical load data) and the sequence processing capability of LSTM (using these change patterns to predict future load trends) can be combined to predict future task loads and resource requirements in order to make more effective scheduling decisions. That is, in the embodiment of the present application, the CNN-LSTM hybrid model can be used for feature extraction through CNN and for processing long-term and short-term dependencies of time series data through LSTM, so that time series data can be processed more accurately, and by combining real-time monitoring data (i.e., various performance indicators) with historical time series data, it can better adapt to sudden or slow changes in load, improve prediction accuracy, thereby improving the rationality of GPU resource allocation and reducing GPU resource idleness and waste.

[0159] S140, selecting the most appropriate target scheduling strategy based on the prediction results, various performance indicators and the target priority of any new task.

[0160] In one implementation, the implementation process of step S140 may include the following steps:

[0161] S141. If it is determined that the target priorities of any new task are significantly different from each other, the priority scheduling strategy is selected as the target scheduling strategy.

[0162] In specific implementation, the priority scheduling strategy is used to indicate that tasks are assigned according to their priorities. That is, when there are obvious differences in task priorities, the priority scheduling strategy should be selected as the target scheduling strategy (i.e. the scheduling strategy currently used) to ensure that high-priority tasks are given priority. For example, when the system processes a variety of tasks with different priorities, such as computationally intensive and I / O intensive tasks, scheduling tasks according to their priorities can ensure that important tasks are completed first.

[0163] As an example, the priority scheduling strategy can be shown in the following Table 6.

[0164] Table 6

[0165]

[0166] In the embodiment of the present application, by executing step S141, when system resources are tight, the priority scheduling strategy can ensure the timely execution of key tasks.

[0167] S142: If it is determined based on various performance indicators that the load differences between GPUs are large, a load balancing scheduling strategy is selected as a target scheduling strategy.

[0168] In specific implementation, the load balancing scheduling strategy is used to indicate that tasks are evenly distributed to each GPU. That is, when the load difference between GPUs is large, for example, some GPUs are in a high load state, while other GPUs are idle or have a low load, the load balancing scheduling strategy is selected as the target scheduling strategy to evenly distribute tasks to each GPU to balance the load of each GPU. For example, polling or weighted polling can be used to evenly distribute tasks to each GPU.

[0169] In the embodiment of the present application, by executing step S142, when there is a large difference in load between GPUs, the load of each GPU can be balanced through a load balancing scheduling strategy.

[0170] S143: If it is determined according to the prediction result that the future task load fluctuation of the GPU resources is large, the prediction scheduling strategy is selected as the target scheduling strategy.

[0171] In specific implementation, the prediction scheduling strategy is used to indicate that task allocation is dynamically adjusted according to the prediction results. That is, when the system load fluctuates greatly, the prediction scheduling strategy should be selected as the target scheduling strategy to optimize GPU resource usage.

[0172] For example, if the prediction results show that the GPU load will reach 80% within the next hour, in this case, the upcoming high-load tasks can be assigned to the GPU with lower load in advance, or the current tasks can be adjusted to other time periods to avoid overload.

[0173] In the implementation of this application, by executing step S143, task allocation can be dynamically adjusted according to predicted future load data while paying attention to energy efficiency and GPU resource optimization to avoid overload or idle GPU resources, thereby optimizing GPU resource usage and energy consumption.

[0174] In specific implementation, the above steps S141-S143 can be judged simultaneously, but when all three conditions are met at the same time, the priority scheduling strategy is given priority. The judgment can also be made in the order of the steps S141-S143 above, and the embodiment of the present application is not limited to this.

[0175] In an embodiment of the present application, by executing step S140, a variety of scheduling strategies can be selected instead of relying on a single scheduling strategy. That is, the scheduling strategy can be dynamically selected and adjusted based on real-time monitoring data and prediction results, so that the selected scheduling strategy can adapt to different load conditions and task requirements, reduce task processing delays, and improve the flexibility and adaptability of the system.

[0176] S150 , allocating GPU resources for any new task according to the target scheduling strategy and the target resource requirement of any new task.

[0177] In one implementation, any new task may be assigned to a corresponding GPU according to a target scheduling strategy, and then a GPU resource manager may be called to allocate GPU resources for any new task according to a target resource requirement of any new task.

[0178] S160, monitor the task execution status and GPU resource usage in real time, and feedback the monitoring results.

[0179] During the specific implementation, the monitoring system can be called to monitor the task execution status and GPU resource usage in real time and provide feedback on the monitoring results.

[0180] In the embodiment of the present application, by executing step S160, stable operation of the system can be ensured.

[0181] S170. Dynamically adjust the currently used target scheduling strategy according to the monitoring result to achieve adaptive optimization of GPU resources.

[0182] In one implementation, the implementation process of step S170 may include the following steps:

[0183] S171. If the target scheduling strategy is currently a priority scheduling strategy, when it is determined according to the monitoring results that the load of some GPUs reaches a preset upper limit and the load of the remaining GPUs is lower than a preset lower limit, the target scheduling strategy is switched to a load balancing scheduling strategy to achieve adaptive optimization of GPU resources.

[0184] In specific implementation, the preset upper limit and the preset lower limit can be set according to actual needs, and the embodiment of the present application does not limit this. For example, the preset upper limit can be set to 98% close to 100%, and the preset lower limit can be set to 60%.

[0185] In specific implementation, when the target scheduling strategy is currently the priority scheduling strategy, when the load of some GPUs is close to 100%, it indicates that these GPUs are close to the upper limit of their processing capacity, which may cause slow task execution or failure, while other GPUs have low loads and are in a relatively idle state. In this case, it can be determined that the priority scheduling strategy may cause some task delays and resource waste, and is no longer suitable for the current system state. The target scheduling strategy can be switched from the priority scheduling strategy to the load balancing scheduling strategy.

[0186] S172. If the target scheduling strategy is currently a load balancing scheduling strategy, when it is determined according to the monitoring results that the response time of the high-priority task increases, the target scheduling strategy is switched to a predictive scheduling strategy to achieve adaptive optimization of GPU resources.

[0187] In specific implementation, when the target scheduling strategy is currently the load balancing scheduling strategy, if the response time of high-priority tasks (such as critical tasks) increases, it can be judged that the load balancing scheduling strategy is no longer suitable for the current system state, and the target scheduling strategy can be switched to the predictive scheduling strategy to optimize task allocation based on future load forecasts.

[0188] Of course, when the target scheduling strategy is currently the load balancing scheduling strategy, if the load balancing strategy improves the utilization of GPU resources and reduces task delays, then the target scheduling strategy can remain the load balancing scheduling strategy.

[0189] S173. If the target scheduling strategy is currently the predictive scheduling strategy, when it is determined according to the monitoring results that the switching conditions of the preset scenario are currently met, the target scheduling strategy is switched to the scheduling strategy corresponding to the preset scenario to achieve adaptive optimization of GPU resources.

[0190] In one embodiment, the preset scenarios may be scenarios 1 to 4 in Table 7 below, and the scheduling strategies corresponding to scenarios 1 to 4 are one of priority scheduling strategies, load balancing scheduling strategies, hybrid scheduling strategies (combining priority scheduling strategies and predictive scheduling strategies), and adaptive scheduling strategies.

[0191] Table 7

[0192]

[0193] In the embodiment of the present application, by executing step S173, the scheduling strategy can be dynamically adjusted to cope with corresponding emergencies, thereby achieving adaptive optimization of GPU resources.

[0194] In an embodiment of the present application, by executing step S170, the scheduling strategy can be dynamically adjusted according to real-time monitoring data to achieve adaptive optimization of GPU resources.

[0195] In summary, the GPU resource scheduling optimization method based on the hybrid model provided by the embodiment of the present application, by introducing the CNN-LSTM hybrid model, can be combined with deep learning technology for accurate load prediction, and accurate GPU load prediction is achieved, thereby improving the intelligence level and prediction accuracy of the system. At the same time, by adopting a variety of scheduling strategies, the scheduling strategy can be flexibly and dynamically adjusted in real time, which can significantly improve resource utilization efficiency and reduce task processing delays, thereby achieving the purpose of optimizing energy efficiency and reducing operating costs. In summary, the GPU resource scheduling optimization method based on the hybrid model provided by the embodiment of the present application can improve the flexibility and intelligence of the system, while optimizing energy efficiency and reducing operating costs.

[0196] In general, compared with the traditional load prediction method, as shown in Table 8 below, the GPU resource scheduling optimization method based on the hybrid model provided in the embodiment of the present application (referred to as the method of the present application in Table 8) has obvious advantages.

[0197] Table 8 Comparison between this application method and traditional load prediction method

[0198]

[0199]

[0200] Figure 2 FIG. 2 shows a structural block diagram of a GPU resource scheduling optimization device based on a hybrid model according to an embodiment of the present application. Figure 2 As shown, the device may include:

[0201] Task queue 210, used to store tasks, manage task priorities, etc.;

[0202] Resource manager 220, used to release GPU resources, allocate GPU resources, monitor GPU resources, etc.;

[0203] Monitoring system 230, used to monitor GPU status, task status, network status, system performance, storage status, etc.;

[0204] The scheduler 240 is connected to the task queue 210, the resource manager 220 and the monitoring system 230, and is used to implement the GPU resource scheduling optimization method based on the hybrid model in the above embodiment.

[0205] The functions of the various components in the hybrid model-based GPU resource scheduling optimization device in the embodiment of the present application can be found in the corresponding description in the above method, and will not be repeated here.

[0206] In an embodiment of the present application, a hybrid model-based GPU resource scheduling optimization device integrates the task queue 210, the resource manager 220, the monitoring system 230 and the scheduler 240 to form a complete adaptive GPU scheduling system, which can ensure efficient collaboration and data transmission between the various components.

[0207] Figure 3 FIG. 2 shows a block diagram of a computer device according to an embodiment of the present application. Figure 3 As shown, the computer device includes: a memory 310 and a processor 320. The memory 310 stores instructions, and the instructions are loaded and executed by the processor 320 to implement the GPU resource scheduling optimization method based on the hybrid model in the above embodiment. The number of the memory 310 and the processor 320 can be one or more.

[0208] The computer device also includes:

[0209] The communication interface 330 is used to communicate with external devices and perform data exchange transmission.

[0210] If the memory 310, the processor 320 and the communication interface 330 are implemented independently, the memory 310, the processor 320 and the communication interface 330 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0211] Optionally, in a specific implementation, if the memory 310, the processor 320 and the communication interface 330 are integrated on a chip, the memory 310, the processor 320 and the communication interface 330 can communicate with each other through an internal interface.

[0212] An embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program runs on a computer, the method provided in the embodiment of the present application is implemented.

[0213] An embodiment of the present application also provides a chip, which includes a processor for calling and executing instructions stored in a memory, so that a communication device equipped with the chip executes the method provided by the embodiment of the present application.

[0214] An embodiment of the present application also provides a chip, including: an input interface, an output interface, a processor and a memory, wherein the input interface, the output interface, the processor and the memory are connected via an internal connection path, and the processor is used to execute the code in the memory. When the code is executed, the processor is used to execute the method provided in the embodiment of the application.

[0215] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. It is worth noting that the processor may be a processor supporting the Advanced RISC Machines (ARM) architecture.

[0216] Further, optionally, the above-mentioned memory may include a read-only memory and a random access memory, and may also include a non-volatile random access memory. The memory may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may include a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may include a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available. For example, static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct rambus RAM (DR RAM).

[0217] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.

[0218] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.

[0219] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0220] Any process or method description in the flow chart or otherwise described herein can be understood to represent a module, fragment or portion of a code including one or more executable instructions for implementing the steps of a specific logical function or process. And the scope of the preferred embodiment of the present application includes other implementations, in which the functions may not be performed in the order shown or discussed, including in a substantially simultaneous manner or in a reverse order according to the functions involved.

[0221] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, which can be embodied in any computer-readable medium for use by an instruction execution system, apparatus or device (such as a computer-based system, a system including a processor or other system that can fetch instructions from an instruction execution system, apparatus or device and execute instructions), or used in combination with these instruction execution systems, apparatuses or devices.

[0222] It should be understood that the various parts of the present application can be implemented with hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented with software or firmware stored in a memory and executed by a suitable instruction execution system. All or part of the steps of the above embodiment method can be completed by instructing the relevant hardware through a program, which can be stored in a computer-readable storage medium, and when the program is executed, it includes one of the steps of the method embodiment or a combination thereof.

[0223] In addition, each functional unit in each embodiment of the present application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the above-mentioned integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium can be a read-only memory, a disk or an optical disk, etc.

[0224] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A GPU resource scheduling optimization method based on a hybrid model, characterized in that: include: Receive several new tasks and obtain the target resource requirements and target priority of any new task; Obtain various current performance indicators of GPU resources, and store the various performance indicators as target time series data; Calling the CNN-LSTM hybrid model deployed in the production environment to perform prediction learning on the target time series data to obtain a prediction result, wherein the prediction result includes future task load and resource demand of GPU resources; Selecting the most appropriate target scheduling strategy according to the prediction result, the various performance indicators and the target priority of any new task; Allocate GPU resources for any new task according to the target scheduling strategy and the target resource requirement of any new task; Monitor task execution and GPU resource usage in real time and provide feedback on monitoring results; According to the monitoring results, the currently used target scheduling strategy is dynamically adjusted to achieve adaptive optimization of GPU resources.

2. The method according to claim 1, characterized in that Obtaining the target resource requirements and target priority for any new task involves: Referring to historical running data of a designated task, estimating resource requirements of any new task to obtain the target resource requirements, wherein the designated task is a task of the same task type as that of any new task; According to the relevant static factors of any new task, the priority of any new task is pre-assigned to obtain the target priority.

3. The method according to claim 1, characterized in that Obtaining various current performance indicators of GPU resources and storing the various performance indicators as target time series data includes: Collect various first performance indicators of GPU resources at the current system level; Collect various secondary performance indicators of the current GPU level of GPU resources; According to the various first performance indicators and the various second performance indicators, the various performance indicators are obtained; Relevant performance indicators are periodically extracted from the various first performance indicators and the various second performance indicators and stored as the target time series data.

4. The method according to claim 1, characterized in that: The CNN-LSTM hybrid model is obtained through the following training process: Collect historical time series data of GPU resources; Preprocessing the historical time series data to obtain processed historical time series data; Split the processed historical event sequence data into a training set, a validation set, and a test set; Build a CNN-LSTM model architecture and initialize the hyperparameters of the CNN-LSTM model architecture and Adam optimizer; Input the training set into the CNN-LSTM model architecture for forward propagation, and output the final prediction result; Based on the loss function, performing loss calculation according to the final prediction result and the training set to obtain a loss value; According to the loss value, the gradient of the loss function with respect to the model parameters of the CNN-LSTM model architecture is calculated, and the gradient is back-propagated to update the model parameters, and the model parameters are adjusted by the Adam optimizer to obtain an initial CNN-LSTM model; Using the validation set to perform performance evaluation on the initial CNN-LSTM model, and adjusting model parameters of the initial CNN-LSTM model; The test set is used to test the initial CNN-LSTM model after adjusting the model parameters. When it is judged according to the test result that the accuracy of the initial CNN-LSTM model reaches above a preset value, the initial CNN-LSTM model is used as the CNN-LSTM model.

5. The method according to claim 1, characterized in that The CNN-LSTM hybrid model is deployed in a production environment in the following way: Save the CNN-LSTM hybrid model as a target file; Load the target file, obtain and deploy the CNN-LSTM hybrid model as an API service; Call the Docker container to package the API service and deploy it to the production environment.

6. The method according to any one of claims 1 to 5, characterized in that Selecting the most appropriate target scheduling strategy according to the prediction result, the various performance indicators and the target priority of any new task includes: If it is determined that the target priorities are significantly different according to the target priority of any new task, a priority scheduling strategy is selected as the target scheduling strategy, wherein the priority scheduling strategy is used to indicate that tasks are allocated according to task priorities; If it is determined according to the various performance indicators that the load differences between the GPUs are large, a load balancing scheduling strategy is selected as the target scheduling strategy, where the load balancing scheduling strategy is used to indicate that tasks are evenly distributed to each GPU; If it is determined according to the prediction result that the future task load fluctuation of the GPU resources is large, the prediction scheduling strategy is selected as the target scheduling strategy, and the prediction scheduling strategy is used to indicate the dynamic adjustment of task allocation according to the prediction result.

7. The method according to claim 6, characterized in that According to the monitoring results, dynamically adjusting the currently used scheduling strategy to achieve adaptive optimization of GPU resources includes: If the target scheduling strategy is currently the priority scheduling strategy, when it is determined according to the monitoring result that the load of some GPUs reaches a preset upper limit and the load of the remaining GPUs is lower than a preset lower limit, the target scheduling strategy is switched to the load balancing scheduling strategy to achieve adaptive optimization of GPU resources; If the target scheduling strategy is currently the load balancing scheduling strategy, when it is determined according to the monitoring result that the response time of the high-priority task increases, the target scheduling strategy is switched to the predictive scheduling strategy to achieve adaptive optimization of GPU resources; If the target scheduling strategy is currently the predicted scheduling strategy, when it is determined according to the monitoring result that the switching condition of the preset scenario is currently met, the target scheduling strategy is switched to the scheduling strategy corresponding to the preset scenario to achieve adaptive optimization of GPU resources.

8. A GPU resource scheduling optimization device based on a hybrid model, characterized in that: include: Task queue, used to store tasks and manage task priorities; Resource manager, used to release GPU resources, allocate GPU resources, and monitor GPU resources; Monitoring system, used to monitor GPU status, task status, network status, system performance and storage status; A scheduler, connected to the task queue, the resource manager and the monitoring system, is used to implement the method as described in any one of claims 1-7.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory stores instructions, and the instructions are loaded and executed by the processor to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed on a computer, the method according to any one of claims 1 to 7 is implemented.

Images