DNN reasoning task batch scheduling method oriented to heterogeneous cluster
A heterogeneous cluster and batch scheduling technology, which is applied in the direction of reasoning methods, electrical digital data processing, multi-programming devices, etc., can solve problems such as research on the distribution characteristics of few tasks
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
Method used
Image
Examples
Embodiment 1
[0097] Large data centers usually have very large heterogeneous computing clusters that can accommodate hundreds to thousands of computing nodes. There are a lot of trained neural network models deployed. Outside the cluster, a wide range of user groups use various DNN applications to send DNN inference task requests to the data center for inference. These tasks correspond to different inference models and allowable inference delays. The DNN task inference scheduling in this scenario includes the following three processes: cluster DNN inference request collection, DNN inference task initialization, and computing node DNN inference. The DNN-BS scheduling optimization method mainly acts on how to schedule each task to each computing node with different computing power after these DNN inference tasks reach the cluster. DNN-BS can analyze the dynamic hardware processing capability of the current cluster heterogeneous computing node division, and find the most suitable task-compu...
Embodiment 2
[0099] With the development of technologies such as the Internet of Things, artificial intelligence, and smart embedded devices, it is no longer just deploying a few large data centers to handle all DNN inference tasks. Instead, multiple small processing clusters are deployed where the network edge devices are concentrated. In this case, local-scale DNN inference task scheduling is required on small processing clusters at the network edge.
[0100] For such scenarios, the DNN-BS scheduling method can analyze all DNN applications at the edge of the network, and first classify the DNN reasoning task. At the same time, all computing nodes of the edge small cluster are divided, and then the dynamic hardware processing capability matrix is obtained. When the local DNN inference task reaches the small processing cluster, the weight parameters in the optimization objective function can be adjusted according to the scheduling target, so as to continue the complete DNN inference tas...
Embodiment 3
[0102] Smart homes also face such problems. In a smart home there is usually a processing center that processes the data from all the sensors. These data arrive at the processing center and turn into DNN inference tasks. And often there is more than one hardware processing unit in the processing center. Then all types of data analysis inside the smart home can be attributed to a limited number of DNN inference task categories, which are given to multiple hardware processing units in the processing center for inference.
[0103] In this scenario, the DNN-BS scheduling optimization method can be deployed on the scheduling nodes of the home small processing center, and the dynamic hardware processing capability matrix can be obtained by obtaining the inference time of all types of DNN inference tasks in each hardware processing unit. Afterwards, all DNN inference tasks can be scheduled to the corresponding hardware processing units for inference through the remaining DNN-BS sch...
PUM
Login to View More Abstract
Description
Claims
Application Information
Login to View More