Task processing method, device and system

Abstract

The embodiment of the invention provides a task processing method, device and system. The method can comprise the steps that a graphics processing unit GPU operates a management task on a first thread, and the management task is kept in a runtime state; and the GPU schedules at least one second thread to execute at least one computing task through the management task on the first thread.

Description

technical field [0001] The present disclosure relates to machine learning technology, in particular to a task processing method, device and system. Background technique [0002] A large number of artificial intelligence-related tasks are running in the computing cluster of the data center, especially tasks that require high parallel computing capabilities such as deep learning. Correspondingly, a large number of general-purpose GPUs (Graphics Processing Unit, graphics processing unit) It is the main computing accelerator, but the GPU resources in the cluster are not fully utilized. [0003] Compared with CPUs, the rental price of GPUs in data centers is more expensive, and the low utilization rate further increases the price per unit of computing power. As a result, many users are unwilling to purchase GPU resources and prefer to use deep learning model reasoning, etc. Lightweight tasks run on the CPU. However, as more and more computing resources are required for these ta...

AI Technical Summary

Problems solved by technology

[0002] A large number of artificial intelligence-related tasks are running in the computing cluster of the data center, especially tasks that require high parallel computing capabilities such as deep learning. Correspondingly, a large number of general-purpose GPUs (Graphics Processing Unit, graphics processing unit) The main computing accelerator, but the GPU resources in the cluster are not fully utilized

[0003] Compared with CPUs, the rental price of GPUs in data centers is more expensive, and the low utilization rate further increases the price per unit of computing power. As a result, many users are unwilling to purchase GPU resources and prefer to use deep learning model reasoning, etc. Lightweight tasks run on the CPU

However, as more and more computing resources are required for these tasks, for example, the scale of deep learning models is increasing day by day, and operations on CPUs are gradually unable to meet users' low-latency requirements. The trend of using general-purpose computing accelerators such as GPUs is unavoidable.

Images