Parallel accelerating method and system in heterogeneous computation

Abstract

The invention discloses a parallel accelerating method and system in heterogeneous computation. The method comprises the following steps: determining a topological structure of data transmission between GPUs (Graphics Processing Unit) in advance according to the quantity of the GPUs; acquiring a current task by each GPU in the topological structure and computing data in the current task, so as to obtain a calculating result corresponding to the current task; sharing the calculating result which is obtained by each GPU and corresponds to the current task to the other GPUs in the topological structure; after obtaining the calculating results of the other GPUs in the topological structure by each GPU, starting to execute the next task. By adopting the parallel accelerating method and system in the heterogeneous computation, the parallel computation capability of the GPUs can be improved and the bandwidth requirements on nodes of each GPU are reduced.

Description

technical field [0001] The invention relates to the field of parallel computing, in particular to a parallel acceleration method and system in heterogeneous computing. Background technique [0002] GPU (Graphic Processing Unit, graphics processor), which is mainly used for various parallel calculations, such as the calculations required for drawing computer graphics, including vertex settings, light and shadow, pixel operations, etc. In the GPU, most of the transistors are used to implement calculations, and only a few transistors are used for logic control, cache, etc., so that the GPU has strong computing capabilities and is very suitable for processing simple logic but huge calculations. Tasks, such as deep learning tasks, commonly use GPU devices to calculate the parameters of each layer of neural network models. In order to cope with increasingly large training data or increasingly complex computing tasks, more and more GPUs need to work together to complete computing ...

AI Technical Summary

Problems solved by technology

The higher the bandwidth of the master node, the more slave nodes communicate with the master node at the same time, but in practical applications, due to hardware limitations, the bandwidth of the master node can only allow communication with a small number of slave nodes at the same time; if there are many slave nodes, then The longer the time for each GPU to complete a data update; if the bandwidth of the master node remains unchanged, as the size of the slave node increases, the time for each GPU data update will increase linearly, which seriously affects the acceleration efficiency of the GPU.

Images