Resource scheduling method, device, equipment and storage medium

Abstract

The embodiments of the present invention provide a resource scheduling method, device, equipment and storage medium, which relate to the field of computer technology. The specific implementation scheme is: in response to receiving a target task to be processed, predicting the reference resource data of the designated task corresponding to each node of the GPU cluster; wherein the designated task corresponding to each node is the task currently being processed by the node; for each of the nodes, based on the reference resource data of the designated task corresponding to the node, determining the allocatable resources corresponding to the node in the target time period as the to-be-used resources corresponding to the node; based on the to-be-used resources corresponding to each node, selecting a target node that meets the resource requirements of the target task from the nodes; calling the to-be-used resources of the target node to process the target task. It can be seen that through this scheme, the utilization rate of the resource pool can be improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a resource scheduling method, device, equipment and storage medium. Background Art

[0002] A GPU (graphics processing unit) cluster is a computer cluster. Each node in the GPU cluster is equipped with a GPU chip. The GPU resources provided by all the GPU chips in the GPU cluster constitute a resource pool, which provides computing power for the tasks generated by upper-level applications.

[0003] In the related art, a method of static partitioning of GPU resources is adopted between different upper-layer applications, that is, nodes are statically allocated to processing tasks corresponding to different upper-layer applications to achieve resource allocation and isolation between processing tasks corresponding to different upper-layer applications.

[0004] As upper-layer applications become more complex, nodes in GPU clusters gradually show heterogeneous characteristics, that is, the GPU chips equipped on the nodes are of various types. Then, the processing tasks corresponding to different upper-layer applications in the related technology can only be processed by the GPU resources of fixed nodes, which may make some nodes have excess GPU resources for the assigned processing tasks, and the various tasks assigned to some nodes need to wait in the queue for a long time, resulting in low utilization of the resource pool.

[0005] Therefore, there is an urgent need for a resource scheduling method that improves the utilization rate of the resource pool. Summary of the invention

[0006] The purpose of the embodiments of the present invention is to provide a resource scheduling method, apparatus, device and storage medium to improve the utilization rate of the resource pool. The specific technical solution is as follows:

[0007] In a first aspect, an embodiment of the present invention provides a resource scheduling method, which is applied to a master control node in a graphics processing unit GPU cluster, and the method includes:

[0008] In response to receiving the target task to be processed, predicting the reference resource data of the designated tasks corresponding to each node of the GPU cluster; wherein the designated task corresponding to each node is the task currently being processed by the node; the reference resource data of any designated task is used to characterize the resource usage in the target time period when the designated task is executed by the corresponding node;

[0009] For each of the nodes, based on the reference resource data of the designated task corresponding to the node, determining the allocatable resources corresponding to the node in the target time period as the to-be-used resources corresponding to the node;

[0010] Based on the to-be-used resources corresponding to each node, a target node that meets the resource requirements of the target task is selected from each node;

[0011] Invoke the target node's to-be-utilized resources to process the target task.

[0012] Optionally, the predicting reference resource data of the designated tasks corresponding to each node in the GPU cluster includes:

[0013] For each node in the GPU cluster, historical resource data of a specified task corresponding to the node is obtained; wherein the historical resource data corresponding to any specified task represents resource usage in a historical time period before the target time period when the specified task is executed by the corresponding node;

[0014] The historical resource data of the designated task corresponding to the node is used to predict the reference resource data of the designated task corresponding to the node.

[0015] Optionally, the using the historical resource data of the designated task corresponding to the node to predict the reference resource data of the designated task corresponding to the node includes:

[0016] Determine a resource prediction model corresponding to a target chip type as a resource prediction model to be used; wherein the target chip type is a chip type of a GPU chip configured by the node for processing a corresponding designated task; each chip type corresponds to a resource prediction model; wherein the resource prediction model corresponding to any chip type is obtained by training using sample historical resource data of a designated sample task and a true value corresponding to the designated sample task, wherein the designated sample task is a sample task processed by a GPU chip of the chip type, wherein the sample historical resource data is data representing resource usage in a sample historical time period, wherein the true value corresponding to the designated sample task is data representing resource usage in a sample target time period, wherein the sample target time period is a time period after the historical sample time period;

[0017] The historical resource data of the designated task corresponding to the node is input into the resource prediction model to be utilized to obtain the reference resource data of the designated task corresponding to the node.

[0018] Optionally, the training method of the resource prediction model corresponding to each chip type includes:

[0019] Determine training samples; wherein each training sample is historical resource data of a sample task processed by a GPU chip of the chip type, and each training sample has a specified label, and the specified label is data characterizing resource usage of the sample task in a sample target time period;

[0020] Input the training samples into the resource prediction model to be trained to obtain the prediction results corresponding to the training samples;

[0021] Based on the prediction results, determining whether the model has converged;

[0022] If convergence occurs, the training ends; otherwise, the parameters of the resource prediction model are adjusted and the process returns to the step of determining the training sample.

[0023] Optionally, the to-be-used resources corresponding to each node include the to-be-used resources corresponding to each GPU chip configured for the node;

[0024] The calling of the target node's to-be-utilized resources to process the target task includes:

[0025] Invoke the to-be-used resources corresponding to a GPU chip configured on the target node to process the target task.

[0026] In a second aspect, an embodiment of the present invention provides a resource scheduling device, which is applied to a master control node in a graphics processor GPU cluster, and the device includes:

[0027] A prediction module, configured to predict reference resource data of designated tasks corresponding to respective nodes of the GPU cluster in response to receiving a target task to be processed; wherein the designated task corresponding to each node is the task currently being processed by the node; and the reference resource data of any designated task is used to characterize the resource usage of the designated task in a target time period when the designated task is executed by the corresponding node;

[0028] A determination module, configured to determine, for each of the nodes, based on reference resource data of a designated task corresponding to the node, an allocatable resource corresponding to the node in the target time period as a to-be-used resource corresponding to the node;

[0029] A selection module is used to select a target node that meets the resource requirements of the target task from each node based on the to-be-utilized resources corresponding to each node;

[0030] The calling module is used to call the resources to be used of the target node to process the target task.

[0031] Optionally, the prediction module includes:

[0032] An acquisition submodule is used to acquire, for each node in each node of the GPU cluster, historical resource data of a specified task corresponding to the node; wherein the historical resource data corresponding to any specified task represents resource usage in a historical time period before the target time period when the specified task is executed by the corresponding node;

[0033] The prediction submodule is used to predict the reference resource data of the specified task corresponding to the node by using the historical resource data of the specified task corresponding to the node.

[0034] Optionally, the prediction submodule is specifically used for:

[0035] Determine a resource prediction model corresponding to a target chip type as a resource prediction model to be used; wherein the target chip type is a chip type of a GPU chip configured by the node for processing a corresponding designated task; each chip type corresponds to a resource prediction model; wherein the resource prediction model corresponding to any chip type is obtained by training using sample historical resource data of a designated sample task and a true value corresponding to the designated sample task, wherein the designated sample task is a sample task processed by a GPU chip of the chip type, wherein the sample historical resource data is data representing resource usage in a sample historical time period, wherein the true value corresponding to the designated sample task is data representing resource usage in a sample target time period, wherein the sample target time period is a time period after the historical sample time period;

[0036] The historical resource data of the designated task corresponding to the node is input into the resource prediction model to be utilized to obtain the reference resource data of the designated task corresponding to the node.

[0037] Optionally, the training method of the resource prediction model corresponding to each chip type includes:

[0038] Determine training samples; wherein each training sample is historical resource data of a sample task processed by a GPU chip of the chip type, and each training sample has a specified label, and the specified label is data characterizing resource usage of the sample task in a sample target time period;

[0039] Input the training samples into the resource prediction model to be trained to obtain the prediction results corresponding to the training samples;

[0040] Based on the prediction results, determining whether the model has converged;

[0041] If convergence occurs, the training ends; otherwise, the parameters of the resource prediction model are adjusted and the process returns to the step of determining the training sample.

[0042] Optionally, the to-be-used resources corresponding to each node include the to-be-used resources corresponding to each GPU chip configured for the node;

[0043] The calling module is specifically used for:

[0044] Invoke the to-be-used resources corresponding to a GPU chip configured on the target node to process the target task.

[0045] In a third aspect, an embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0046] Memory, used to store computer programs;

[0047] The processor is used to implement any of the steps of the resource scheduling method described above when executing the program stored in the memory.

[0048] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above-mentioned resource scheduling methods are implemented.

[0049] Beneficial effects of the embodiments of the present invention:

[0050] The solution provided by the embodiment of the present invention predicts the reference resource data of the designated tasks corresponding to each node of the GPU cluster in response to the target task to be processed. The reference resource data corresponding to each node is the resource usage of the task currently processed by the node in the target time period. According to the reference resource data of the designated task corresponding to each node, the to-be-used resources corresponding to the node in the target time period can be determined, so that the target node that meets the resource requirements of the target task can be selected from each node, and then the to-be-used resources of the target node are called to process the target task, thereby realizing the dynamic scheduling of GPU resources. Compared with the prior art, in which tasks are processed by fixed nodes of static partitions, this solution can schedule GPU resources across nodes, thereby making full use of the GPU resources in the GPU resource pool. It can be seen that through this solution, the utilization rate of the resource pool can be improved.

[0051] Of course, it is not necessary to achieve all of the advantages described above at the same time to implement any product or method of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0053] Figure 1 A flow chart of a resource scheduling method provided by an embodiment of the present invention;

[0054] Figure 2 Another flow chart of the resource scheduling method provided by an embodiment of the present invention;

[0055] Figure 3 A flowchart of a method for training a resource prediction model provided by an embodiment of the present invention;

[0056] Figure 4 A flowchart for implementing resource scheduling in related technologies;

[0057] Figure 5 A flowchart of a specific example of the resource scheduling method provided by an embodiment of the present invention;

[0058] Figure 6 A schematic diagram of the structure of a resource scheduling device provided by an embodiment of the present invention;

[0059] Figure 7 A block diagram of an electronic device for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field based on this application belong to the scope of protection of the present invention.

[0061] In related technologies, static partitioning of GPU resources is adopted between different upper-layer applications, that is, statically allocating nodes to tasks corresponding to different upper-layer applications to achieve resource allocation and isolation between tasks corresponding to different upper-layer applications. However, the static partitioning method ignores the heterogeneous characteristics of nodes. After static partitioning, GPU resources cannot be scheduled across nodes, resulting in low resource pool utilization. When new applications are deployed, chips cannot be deployed based on performance advantages, and performance is poor in heterogeneous environments.

[0062] With the complexity of upper-level applications, the nodes in the GPU cluster gradually show heterogeneous characteristics, that is, the nodes are equipped with GPU chips of various types. In addition, there are more and more usage scenarios for artificial intelligence chips, and the applicability of artificial intelligence chips specially customized for artificial intelligence usage scenarios is significantly higher than that of general-purpose chips. Then, the processing tasks corresponding to different upper-level applications in the related technology can only be processed by the GPU resources of fixed nodes, which may cause some nodes to have excess GPU resources for the assigned processing tasks, resulting in low utilization of the resource pool. Therefore, how to deploy the tasks generated by the application to the appropriate chip and improve the overall utilization of the resource pool according to the unique computing power requirements of different applications becomes the key.

[0063] Based on the above content, in order to improve the utilization rate of the resource pool, the embodiments of the present invention provide a resource scheduling method, apparatus, device and storage medium.

[0064] Below, a resource scheduling method provided by an embodiment of the present invention is first introduced.

[0065] A resource scheduling method provided in an embodiment of the present invention is applied to a master control node under a graphics processing unit GPU cluster. Specifically, the execution subject of the resource scheduling method may be a resource scheduling device, which is a computer program running in the master control node under the graphics processing unit GPU cluster, and the computer program is used to schedule GPU resources when executed.

[0066] The resource scheduling method provided in the embodiment of the present invention may include the following steps:

[0067] In response to receiving the target task to be processed, predicting the reference resource data of the designated tasks corresponding to each node of the GPU cluster; wherein the designated task corresponding to each node is the task currently being processed by the node; the reference resource data of any designated task is used to characterize the resource usage in the target time period when the designated task is executed by the corresponding node;

[0068] For each of the nodes, based on the reference resource data of the designated task corresponding to the node, determining the allocatable resources corresponding to the node in the target time period as the to-be-used resources corresponding to the node;

[0069] Based on the to-be-used resources corresponding to each node, a target node that meets the resource requirements of the target task is selected from each node;

[0070] Invoke the target node's to-be-utilized resources to process the target task.

[0071] The solution provided by the embodiment of the present invention predicts the reference resource data of the designated tasks corresponding to each node of the GPU cluster in response to the target task to be processed. The reference resource data corresponding to each node is the resource usage of the task currently processed by the node in the target time period. According to the reference resource data of the designated task corresponding to each node, the to-be-used resources corresponding to the node in the target time period can be determined, so that the target node that meets the resource requirements of the target task can be selected from each node, and then the to-be-used resources of the target node are called to process the target task, thereby realizing the dynamic scheduling of GPU resources. Compared with the prior art, in which tasks are processed by fixed nodes of static partitions, this solution can schedule GPU resources across nodes, thereby making full use of the GPU resources in the GPU resource pool. It can be seen that through this solution, the utilization rate of the resource pool can be improved.

[0072] The resource scheduling method provided by the embodiment of the present invention is introduced below with reference to the accompanying drawings.

[0073] like Figure 1 As shown, the resource scheduling method provided in the embodiment of the present invention is applied to the master control node under the graphics processing unit GPU cluster. The method may include steps S101-S104:

[0074] S101, in response to receiving a target task to be processed, predicting reference resource data of designated tasks corresponding to each node of the GPU cluster; wherein the designated task corresponding to each node is the task currently being processed by the node; the reference resource data of any designated task is used to characterize the resource usage of the designated task in a target time period when the designated task is executed by the corresponding node;

[0075] In this embodiment, after receiving the target task to be processed, the reference resource data of the specified task corresponding to each node of the GPU cluster can be predicted, so that the predicted reference resource data can be used to provide a reference for resource scheduling for the target task, and finally the GPU resources in the scheduling resource pool can be used to process the target task to be processed.

[0076] It can be understood that, considering that when the target task is received, there may be idle GPU chips in the resource pool, and the idle GPU chips can be used to process the task. At this time, in order to improve the resource scheduling rate, in an optional implementation method, before predicting the reference resource data of the specified tasks corresponding to each node of the GPU cluster, the method may also include: detecting whether there are idle GPU chips in the resource pool, if not, executing the step of predicting the reference resource data of the specified tasks corresponding to each node of the GPU cluster; if so, the target task can be directly deployed on the GPU chip to use the GPU resources provided by the GPU chip to process the target task.

[0077] The tasks processed by the GPU cluster can be diverse, such as artificial intelligence training tasks, reasoning tasks, or computing tasks, etc. Moreover, the target task and the specified task can be the same or different tasks. For example, the target task can be a training task that relies on the VAE (Variational Autoencoder) model framework, or a training task that relies on the DCGAN (Deep Convolution Generative Adversarial Networks) model framework, while the specified task can be a computing task or a training task of other model frameworks, etc.

[0078] Exemplarily, the target time period may be the time period before the tasks currently being processed by each node in the GPU cluster are completed. Exemplarily, the reference resource data of the specified tasks corresponding to each node of the GPU cluster may be periodically predicted, and the target time period may be the time period of the next cycle of the current time when the target task to be processed is received. Exemplarily, the cycle may be five minutes, ten minutes, and so on. It should be noted that the target time period may be set by relevant staff based on experience, and the embodiments of the present invention are not limited thereto.

[0079] Exemplarily, the reference resource data may be the resource utilization rate of GPU resources, or the resource usage, etc. In addition, there may be multiple ways to predict the reference resource data of the designated tasks corresponding to each node. Exemplarily, the resource usage of the designated tasks corresponding to each node in the historical time period before the target time period may be used to predict the reference resource data of the designated tasks corresponding to the node. It should be noted that for the sake of clarity of the layout, the method for predicting the reference resource data of the designated tasks corresponding to each node will be introduced below, which will not be repeated here.

[0080] S102, for each of the nodes, based on the reference resource data of the designated task corresponding to the node, determining the allocatable resources corresponding to the node in the target time period as the to-be-used resources corresponding to the node;

[0081] It can be understood that since the reference resource data of the specified task corresponding to each node is the resource utilization status of each GPU chip configured on the node in processing the specified task in the target time period, in order to reasonably schedule GPU resources to process the target task, the allocatable resources corresponding to each node in the target time period can be first determined based on the reference resource data of the specified task corresponding to each node. The allocatable resources are the idle GPU resources of the node in the target time period, and the allocatable resources corresponding to the node in the target time period are used as the to-be-used resources corresponding to the node, so that the to-be-used resources of the specified node can be called subsequently to process the target task.

[0082] S103, based on the to-be-used resources corresponding to each node, selecting a target node that meets the resource requirements of the target task from the nodes;

[0083] In this embodiment, the resource requirement of the target task is the GPU resources required to process the target task. It is understandable that in order to process the target task, a node with GPU resources that meet the requirements for processing the target task can be selected from various nodes as the target node, that is, the resources to be used of the target node are not less than the GPU resources required for processing the target task, so that the resources to be used of the target node can be called to complete the processing of the target task later.

[0084] Exemplarily, a method of selecting a target node that meets the resource requirements of the target task from the various nodes may be to determine any node that meets the resource requirements as the target node, or to determine the node among the various nodes that meets the resource requirements and has the most corresponding resources to be used as the target node, and so on.

[0085] S104: Call the target node's to-be-used resources to process the target task.

[0086] In this embodiment, after the target node that meets the resource requirements of the target task is selected through step S103, the resources to be utilized in the target node can be called to process the target task. It can be understood that since the GPU chip configured by the target node provides GPU resources, when the resources to be utilized of the target node are called to process the target task, the target node processes the target task through a GPU chip deployed thereon. It should be noted that since the GPU chip is often set with a maximum task processing capacity, when a GPU chip configured by the target node is used to process the target task, the maximum task processing capacity of the chip can also be adjusted so that the chip can process the target task at the same time as processing the currently processed task.

[0087] The solution provided by the embodiment of the present invention predicts the reference resource data of the designated tasks corresponding to each node of the GPU cluster in response to the target task to be processed. The reference resource data corresponding to each node is the resource usage of the task currently processed by the node in the target time period. According to the reference resource data of the designated task corresponding to each node, the to-be-used resources corresponding to the node in the target time period can be determined, so that the target node that meets the resource requirements of the target task can be selected from each node, and then the to-be-used resources of the target node are called to process the target task, thereby realizing the dynamic scheduling of GPU resources. Compared with the prior art, in which tasks are processed by fixed nodes of static partitions, this solution can schedule GPU resources across nodes, thereby making full use of the GPU resources in the GPU resource pool. It can be seen that through this solution, the utilization rate of the resource pool can be improved.

[0088] Optionally, in another embodiment of the present invention, Figure 1 Based on the embodiment shown, Figure 2 As shown, the above step S101 predicts the reference resource data of the designated task corresponding to each node of the GPU cluster, which may include steps S1011-S1012:

[0089] S1011, for each node in the GPU cluster, obtaining historical resource data of a specified task corresponding to the node; wherein the historical resource data corresponding to any specified task represents resource usage in a historical time period before the target time period when the specified task is executed by the corresponding node;

[0090] In this embodiment, in order to predict the reference resource data of the designated tasks corresponding to each node, for each node, the historical resource data of the designated task corresponding to the node can be first obtained, so as to predict the resource usage of the designated task in the target time period by analyzing the historical resource data of the designated task. Exemplarily, the GPU resources used by each chip configured by the node to process the designated task can be monitored to obtain the historical resource data of the designated task corresponding to the node. Exemplarily, the historical resource data can be the utilization rate of the GPU resources corresponding to the designated task when the designated task is processed in the historical time period, or the usage of GPU resources, etc. The historical time period can be any time period before the target time period, for example, the historical time period can be the previous week, previous month, or previous cycle, etc., of the current time when the target task to be processed is received.

[0091] S1012: using the historical resource data of the designated task corresponding to the node, predict the reference resource data of the designated task corresponding to the node.

[0092] It can be understood that, since for each node, the process of the node using its configured GPU chip to process tasks is regular, after obtaining the historical resource data of the specified task corresponding to the node through step S1011, the historical resource data can be used to predict the reference resource data of the specified task corresponding to the node.

[0093] Optionally, in an implementation, using the historical resource data of the designated task corresponding to the node to predict the reference resource data of the designated task corresponding to the node may include steps A1-A2:

[0094] A1, determine the resource prediction model corresponding to the target chip type as the resource prediction model to be used; wherein the target chip type is the chip type of the GPU chip configured by the node for processing the corresponding specified task; each chip type corresponds to a resource prediction model; wherein the resource prediction model corresponding to any chip type is obtained by training using sample historical resource data of a specified sample task and a true value corresponding to the specified sample task, the specified sample task is a sample task processed by a GPU chip of the chip type, the sample historical resource data is data representing resource usage in a sample historical time period, the true value corresponding to the specified sample task is data representing resource usage in a sample target time period, and the sample target time period is a time period after the historical sample time period;

[0095] In this implementation, since the resource usage corresponding to each chip type when used to process tasks is different, the resource prediction model corresponding to the chip type can be determined for different chip types as the resource prediction model to be used. The resource prediction model corresponding to each chip type can be obtained by training the initial resource prediction model with the sample historical resource data of the specified sample task and the true value corresponding to the specified sample task. Exemplarily, the training process of the resource prediction model can be: inputting the sample historical resource data of the specified sample task into the initial resource prediction model to obtain the prediction result corresponding to the specified sample task; calculating the loss value between the true value corresponding to the specified sample task and the prediction result through a preset loss function, and adjusting the parameters of the initial resource prediction model by minimizing the loss value until the loss value of the resource prediction model reaches the preset requirements.

[0096] In addition, the initial resource prediction model may be a neural network model, such as a convolutional neural network model; or a regression model, such as an SVR (Support Vector Regression) model, etc. It should be noted that the embodiment of the present invention does not limit the specific type of the resource prediction model.

[0097] A2, inputting the historical resource data of the designated task corresponding to the node into the to-be-utilized resource prediction model to obtain the reference resource data of the designated task corresponding to the node.

[0098] It can be understood that after determining the resource prediction model corresponding to the target chip type through step A1, the historical resource data of the designated task corresponding to the node is input into the resource prediction model to be used, and the reference resource data of the designated task corresponding to the node can be predicted. It should be noted that since the heterogeneous node is configured with more than one GPU chip type, the designated task is processed by a GPU chip in the node. Therefore, the historical resource data of the processing task corresponding to each chip in the designated task can be input into the resource prediction model to be used corresponding to the chip type to obtain the reference resource data of the processing task corresponding to the chip. The reference resource data of the processing task corresponding to each chip is the reference resource data of the designated task corresponding to the node.

[0099] It can be seen that through this solution, by utilizing the historical resource data corresponding to the designated tasks corresponding to each node of the GPU cluster, the reference resource data of the designated tasks corresponding to each node can be predicted, and the correlation between resource usage and time can be fully explored, so as to accurately predict the reference resource data of the designated tasks corresponding to each node.

[0100] Alternatively, in another embodiment of the present invention, Figure 3As shown, the training method of the resource prediction model corresponding to each chip type may include steps S301-S304:

[0101] S301, determining training samples; wherein each training sample is historical resource data of a sample task processed by a GPU chip of the chip type, and each training sample has a specified label, which is data representing resource usage of the sample task in a sample target time period;

[0102] In this embodiment, for each chip type, the training sample is the historical resource data of the sample task processed by the GPU chip of the chip type, and each training sample has a specified label, which represents the data of the resource usage of the sample task in the sample target time period, that is, the true value of the resource usage of the sample task in the sample target time period. It can be understood that since the chips of each chip type have different processing capabilities, the corresponding resource usage is different when the task is processed on different types of chips. Therefore, in order to reasonably schedule the GPU resources provided by each GPU chip, a corresponding resource prediction model can be trained for each chip type, thereby training resource prediction models corresponding to different chip types.

[0103] S302, inputting the training sample into the resource prediction model to be trained to obtain the prediction result corresponding to the training sample;

[0104] In this embodiment, the resource prediction model to be trained can be a regression model, a neural network model, etc. By inputting the training sample into the resource prediction model to be trained, a prediction result of data representing the resource usage of the training sample in the sample target time period can be obtained, so that the resource prediction model can be trained later according to the difference between the prediction result and the corresponding true value. In addition, it should be noted that the resource prediction model to be trained corresponding to different chip types can be the same model, but because the training samples corresponding to different chip types are different, after the model training is completed, various resource prediction models for different chip types will be trained.

[0105] S303, judging whether the model has converged based on the prediction result;

[0106] It is understandable that, since each training sample has a specified label, and the specified label represents the data of resource usage of the sample task in the sample target time period, each training sample is input into the liveness detection model to be trained to obtain the prediction results corresponding to each training sample, and the total loss value of the model can be determined based on the difference between the prediction results corresponding to each training sample and the specified label. Thus, the total loss value of the model can be used to judge whether the model has converged. Exemplarily, it can be judged whether the model has converged by judging whether the total loss value of the model is less than a preset loss threshold, and the preset loss threshold can be set by relevant technical personnel based on experience. Exemplarily, the difference between the prediction result corresponding to each training sample and the specified label can be calculated by the cross entropy loss function, or the Euclidean distance. It should be noted that the embodiment of the present invention does not limit the calculation method of the difference.

[0107] S304, if converged, the training ends, otherwise adjust the parameters of the resource prediction model and return to the step of determining the training sample.

[0108] It is understandable that if the model converges based on step S303, the training can be terminated to obtain the resource prediction model corresponding to the chip type. If the model does not converge, the parameters of the resource prediction model are adjusted and the step of determining the training sample is returned until the model converges and the training is terminated.

[0109] It can be seen that through this solution, by training the corresponding resource prediction model for each chip type, a resource prediction model corresponding to the chip type can be trained according to the chip characteristics of the chip type, so that the resource prediction model corresponding to each chip type can be used subsequently to accurately predict the reference resource data of the tasks processed on the chip.

[0110] Optionally, in another embodiment of the present invention, the to-be-used resources corresponding to each node include the to-be-used resources corresponding to each GPU chip configured by the node;

[0111] Accordingly, in this embodiment, calling the target node's to-be-utilized resources to process the target task in the above step S104 may include:

[0112] Invoke the unused resources corresponding to a GPU chip configured by the target node to process the target task.

[0113] It is understandable that, due to the heterogeneity of nodes, the GPU chips configured by the nodes may be of various types. Therefore, in this embodiment, the unused resources corresponding to each node include the unused resources corresponding to each GPU chip configured by the node. Exemplarily, the method of calling the unused resources corresponding to a GPU chip configured by the target node may be: randomly calling the unused resources corresponding to a GPU chip configured by the target node, or calling the unused resources corresponding to the chip with the most unused resources among the GPU chips configured by the target node, and so on. It should be noted that the unused resources of any chip that meets the GPU resource requirements required for processing the target task can be used to process the target task, and the embodiment of the present invention does not limit the method of determining the GPU chip for processing the target task.

[0114] It can be seen that through this solution, the target tasks can be reasonably deployed according to the to-be-used resources corresponding to each GPU chip when the nodes are heterogeneous, so that the GPU resources in the resource pool can be fully utilized and the task processing efficiency can be improved when the nodes are heterogeneous.

[0115] In order to better understand the content of the embodiments of the present invention, Figure 4 and Figure 5 A specific example of an embodiment of the present invention is introduced.

[0116] Figure 4 The resource scheduling method for node heterogeneity in the prior art is presented, including the following steps:

[0117] S401, statically adding node type labels to heterogeneous GPU nodes in the GPU cluster;

[0118] S402, classifying tasks generated by upper layer applications of the GPU cluster;

[0119] S403, testing the time required for the newly added task to run when it exclusively occupies multiple types of heterogeneous GPU nodes in the GPU cluster;

[0120] S404, determining performance differences of heterogeneous GPU nodes;

[0121] S405, using the second price trading method to trade the performance differences of heterogeneous GPU nodes and deploying newly added tasks.

[0122] Due to the resource scheduling scheme in the prior art, when a node is occupied, the newly added application will wait in line, which prolongs the processing time of the task. Therefore, in order to improve the utilization rate of the resource pool without idle nodes, this scheme predicts the resource usage of the task by constructing a high-dimensional nonlinear model, so that when a new task (corresponding to the target task in the above text) is deployed in the resource pool, it can be deployed to the appropriate node according to the predicted value (corresponding to the reference resource data in the above text) of the resource usage of the existing task (corresponding to the specified task in the above text) in the future period of time (corresponding to the target time period in the above text).

[0123] Figure 5 A flowchart showing a specific example of an embodiment of the present invention includes the following steps:

[0124] S501, monitoring the resource usage of the task;

[0125] The resource usage of the task completion process when the task occupies the chip exclusively is monitored according to the time slice, and the monitoring data corresponding to the task deployed on different heterogeneous chips is counted according to time.

[0126] S502, determining a resource prediction model to be trained and training samples;

[0127] Obtain the monitoring data of resource usage obtained in step S501, perform statistics on each sample task, and organize it into a multi-dimensional data matrix. Specifically, the application access log can be combined to perform time domain transformation on each application request sequence and convert it into a data matrix consisting of GPU usage and time required to process the tasks generated by the application:

[0128] {(x1,E1),(x2,E2),...(x m ,E m )}

[0129] Where m represents the number of times monitoring data is collected within a certain time segment. E represents the current utilization rate of the GPU chip (corresponding to the resource usage mentioned above), and x represents time. Based on the idea of ​​fitting regression, an SVR model (corresponding to the resource prediction model mentioned above) is generated to summarize the changing characteristics of resource requests corresponding to different tasks. The formula of the model function F(X) corresponding to the SVR model is as follows:

[0130] F(X)=<ω,Φ(X)>+b

[0131] Among them, ω and b are the parameters of the separating hyperplane to be trained. X is the input data composed of the above data matrix, Φ(X) represents the nonlinear mapping of the input data X to the high-dimensional space, and <ω,Φ(X)> represents the dot product operation of ω and Φ(X) in the high-dimensional space. The model regression uses the minimum regularized risk function (corresponding to the loss function mentioned above):

[0132]

[0133] Where C is the regularization constant, l ε is an insensitive loss function, and i is a positive integer between 1 and m. The training goal of the model is to find the optimal ω and b to minimize the value of the risk function.

[0134] S503, solving the model parameters of the resource prediction model;

[0135] In order to reduce the impact of abnormal data in the above data matrix on the model function F(X), the above risk function can be optimized. The deviation interval is adjusted according to the deviation of the data matrix in the above steps so that all input data can be regressed. The function within a reasonable deviation interval is expressed as follows:

[0136]

[0137] Among them, ε represents any small number greater than 0, and the slack variable ξ is introduced i and Modify the regularized risk function and obtain the formula of the optimized risk function as follows:

[0138]

[0139]

[0140] Where C is the regularization constant and st is the mathematical symbol for "constrained on". Introducing slack variables can make the loss value of the model function F(X) be within ∈+ξ i and Since the above optimization process has a strong duality, by introducing Lagrange multipliers, it can be solved as the following constrained dual optimization problem:

[0141]

[0142] Among them, α i and η i is a non-negative Lagrange multiplier, and is its dual multiplier, is the Lagrangian function used to solve the dual optimization problem.

[0143] The parameters ω, b, ξ are i , Find the partial derivative, set it to 0, and use the KKT (Karush KuhnTucker, optimization) condition to get the final model:

[0144]

[0145] S504, introduce the Gaussian kernel function to improve the resource prediction model:

[0146]

[0147] Among them, δ is the kernel width parameter of the Gaussian kernel function, and the preset parameters C and δ in step S503 need to be further optimized to obtain the preset parameter values ​​when the resource pool utilization rate is the highest. By using the kernel technique to reduce the amount of calculation to obtain the best model, the model accuracy is improved.

[0148] S505, using the trained resource prediction model, predicting the resource usage of the specified sample task in the future time period, and deploying new tasks according to the prediction results;

[0149] According to the prediction results, the new task is deployed to the appropriate node, and the GPU resources provided by the GPU chip on the node are used to process the task. The deployment principle is to cross-deploy related nodes according to the complementary GPU resource occupancy of each node. Every time a new task enters the task deployment queue, the cnmem module in the .theanorc file is adjusted to adjust the number of program runs that can be satisfied on a GPU chip, so as to achieve dynamic adjustment of resource deployment. Tasks are preferentially deployed on the node with the most matching remaining GPU resources when GPU resources are satisfied.

[0150] It can be seen that through this solution, a method of hybrid deployment of tasks by feature classification is proposed to minimize the running time in view of the heterogeneous characteristics of nodes. By predicting the resource usage of tasks, deploying multiple tasks in combination, and running multiple programs on a GPU chip at the same time, the task waiting time and the overall running time of tasks in the resource pool are shortened. When a new task is deployed in the resource pool, this solution can predict the usage of GPU resources within a period of time and deploy the new task to a suitable node. In addition, this solution also improves the scalability of the resource pool to tasks, and can decouple and separate each GPU resource and schedule it in parallel. When the cluster environment is complex and heterogeneous, it can make full use of the GPU resources of the cluster and improve the execution efficiency of tasks. In addition, with the help of the idea of ​​fitting regression, a resource prediction model is generated, and tasks are deployed with the help of the prediction results, which gives play to the concurrency performance of the heterogeneous resource pool, so that multiple tasks can efficiently share the resources of the heterogeneous resource pool.

[0151] Corresponding to the above method embodiment, the present invention also provides a resource scheduling device, such as Figure 6 As shown, the device is applied to a master control node under a graphics processor GPU cluster, and includes:

[0152] The prediction module 610 is used to predict the reference resource data of the designated tasks corresponding to the nodes of the GPU cluster in response to receiving the target task to be processed; wherein the designated task corresponding to each node is the task currently being processed by the node; the reference resource data of any designated task is used to characterize the resource usage of the designated task in the target time period when it is executed by the corresponding node;

[0153] A determination module 620 is used to determine, for each of the nodes, based on the reference resource data of the designated task corresponding to the node, the allocatable resource corresponding to the node in the target time period as the to-be-used resource corresponding to the node;

[0154] A selection module 630 is used to select a target node that meets the resource requirements of the target task from each node based on the to-be-utilized resources corresponding to each node;

[0155] The calling module 640 is used to call the to-be-used resources of the target node to process the target task.

[0156] Optionally, the prediction module includes:

[0157] An acquisition submodule is used to acquire, for each node in each node of the GPU cluster, historical resource data of a specified task corresponding to the node; wherein the historical resource data corresponding to any specified task represents resource usage in a historical time period before the target time period when the specified task is executed by the corresponding node;

[0158] The prediction submodule is used to predict the reference resource data of the specified task corresponding to the node by using the historical resource data of the specified task corresponding to the node.

[0159] Optionally, the prediction submodule is specifically used for:

[0160] Determine a resource prediction model corresponding to a target chip type as a resource prediction model to be used; wherein the target chip type is a chip type of a GPU chip configured by the node for processing a corresponding designated task; each chip type corresponds to a resource prediction model; wherein the resource prediction model corresponding to any chip type is obtained by training using sample historical resource data of a designated sample task and a true value corresponding to the designated sample task, wherein the designated sample task is a sample task processed by a GPU chip of the chip type, wherein the sample historical resource data is data representing resource usage in a sample historical time period, wherein the true value corresponding to the designated sample task is data representing resource usage in a sample target time period, wherein the sample target time period is a time period after the historical sample time period;

[0161] The historical resource data of the designated task corresponding to the node is input into the resource prediction model to be utilized to obtain the reference resource data of the designated task corresponding to the node.

[0162] Optionally, the training method of the resource prediction model corresponding to each chip type includes:

[0163] Determine training samples; wherein each training sample is historical resource data of a sample task processed by a GPU chip of the chip type, and each training sample has a specified label, and the specified label is data characterizing resource usage of the sample task in a sample target time period;

[0164] Input the training samples into the resource prediction model to be trained to obtain the prediction results corresponding to the training samples;

[0165] Based on the prediction results, determining whether the model has converged;

[0166] If convergence occurs, the training ends; otherwise, the parameters of the resource prediction model are adjusted and the process returns to the step of determining the training sample.

[0167] Optionally, the to-be-used resources corresponding to each node include the to-be-used resources corresponding to each GPU chip configured for the node;

[0168] The calling module is specifically used for:

[0169] Invoke the to-be-used resources corresponding to a GPU chip configured on the target node to process the target task.

[0170] The embodiment of the present invention further provides an electronic device, such as Figure 7 As shown, it includes a processor 701, a communication interface 702, a memory 703 and a communication bus 704, wherein the processor 701, the communication interface 702, and the memory 703 communicate with each other through the communication bus 704.

[0171] Memory 703, used for storing computer programs;

[0172] The processor 701 is used to implement the steps of any resource scheduling method in the above embodiments when executing the program stored in the memory 703.

[0173] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0174] The communication interface is used for communication between the above electronic device and other devices.

[0175] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0176] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0177] In another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned resource scheduling methods are implemented.

[0178] In another embodiment of the present invention, a computer program product including instructions is provided, which, when executed on a computer, enables the computer to execute any resource scheduling method in the above embodiments.

[0179] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0180] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0181] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0182] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A resource scheduling method, characterized in that: Applied to a master control node in a graphics processing unit GPU cluster, the method comprises: In response to receiving a target task to be processed, for each node in the GPU cluster, historical resource data of a designated task corresponding to the node is obtained; wherein the designated task corresponding to each node is a task currently being processed by the node, and the historical resource data corresponding to any designated task represents the resource usage in a historical time period before the target time period when the designated task is executed by the corresponding node; a resource prediction model corresponding to a target chip type is determined as a resource prediction model to be used; wherein the target chip type is a chip type of a GPU chip configured by the node for processing the corresponding designated task; each chip type corresponds to a resource prediction model; wherein the resource prediction model corresponding to any chip type is a resource prediction model corresponding to a specified sample The sample historical resource data of the task, and the corresponding true value training of the specified sample task, the specified sample task is a sample task processed by the GPU chip of the chip type, the sample historical resource data is data representing the resource usage in the sample historical period, and the true value corresponding to the specified sample task is data representing the resource usage in the sample target time period, and the sample target time period is the time period after the sample historical period; the historical resource data of the specified task corresponding to the node is input into the to-be-used resource prediction model to obtain the reference resource data of the specified task corresponding to the node; the reference resource data of any specified task is used to represent the resource usage in the target time period when the specified task is executed by the corresponding node; For each of the nodes, based on the reference resource data of the designated task corresponding to the node, determining the allocatable resources corresponding to the node in the target time period as the to-be-used resources corresponding to the node; Based on the to-be-used resources corresponding to each node, a target node that meets the resource requirements of the target task is selected from each node; Invoke the target node's to-be-utilized resources to process the target task.

2. The method according to claim 1, characterized in that The training method of the resource prediction model corresponding to each chip type includes: Determine training samples; wherein each training sample is historical resource data of a sample task processed by a GPU chip of the chip type, and each training sample has a specified label, and the specified label is data characterizing resource usage of the sample task in a sample target time period; Input the training samples into the resource prediction model to be trained to obtain the prediction results corresponding to the training samples; Based on the prediction results, determining whether the model has converged; If convergence occurs, the training ends; otherwise, the parameters of the resource prediction model are adjusted and the process returns to the step of determining the training sample.

3. The method according to claim 1, characterized in that The unused resources corresponding to each node, including the unused resources corresponding to each GPU chip configured for the node; The calling of the target node's to-be-utilized resources to process the target task includes: Invoke the to-be-used resources corresponding to a GPU chip configured on the target node to process the target task.

4. A resource scheduling device, characterized in that: Applied to a master control node in a graphics processing unit GPU cluster, the device comprises: A prediction module is used to, in response to receiving a target task to be processed, obtain historical resource data of a designated task corresponding to each node in each node of the GPU cluster; wherein the designated task corresponding to each node is the task currently being processed by the node, and the historical resource data corresponding to any designated task represents the resource usage in the historical time period before the target time period when the designated task is executed by the corresponding node; determine a resource prediction model corresponding to a target chip type as a resource prediction model to be used; wherein the target chip type is the chip type of a GPU chip configured by the node for processing the corresponding designated task; each chip type corresponds to a resource prediction model; wherein the resource prediction model corresponding to any chip type is a resource prediction model to be used The sample historical resource data of the designated sample task and the corresponding true value training of the designated sample task are obtained, wherein the designated sample task is a sample task processed by the GPU chip of the chip type, the sample historical resource data is data representing the resource usage in the sample historical period, and the true value corresponding to the designated sample task is data representing the resource usage in the sample target time period, and the sample target time period is the time period after the sample historical period; the historical resource data of the designated task corresponding to the node is input into the to-be-used resource prediction model to obtain the reference resource data of the designated task corresponding to the node; the reference resource data of any designated task is used to represent the resource usage in the target time period when the designated task is executed by the corresponding node; A determination module, configured to determine, for each of the nodes, based on reference resource data of a designated task corresponding to the node, an allocatable resource corresponding to the node in the target time period as a to-be-used resource corresponding to the node; A selection module is used to select a target node that meets the resource requirements of the target task from each node based on the to-be-utilized resources corresponding to each node; The calling module is used to call the resources to be used of the target node to process the target task.

5. The device according to claim 4, characterized in that The training method of the resource prediction model corresponding to each chip type includes: Determine training samples; wherein each training sample is historical resource data of a sample task processed by a GPU chip of the chip type, and each training sample has a specified label, and the specified label is data characterizing resource usage of the sample task in a sample target time period; input the training sample into a resource prediction model to be trained to obtain a prediction result corresponding to the training sample; Based on the prediction results, determining whether the model has converged; If convergence occurs, the training ends; otherwise, the parameters of the resource prediction model are adjusted and the process returns to the step of determining the training sample.

6. The device according to claim 4, characterized in that The unused resources corresponding to each node, including the unused resources corresponding to each GPU chip configured for the node; The calling module is specifically used for: Invoke the to-be-used resources corresponding to a GPU chip configured on the target node to process the target task.

7. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-3 when executing a program stored in a memory.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 3 is implemented.

Images