GPU-based deep neural network model training method and apparatus, and computer device

Abstract

The invention relates to a GPU-based deep neural network model training method and device, computer equipment and a storage medium. The method comprises the steps: when a deep neural network model istrained for the first time, compressing output data of all hidden layers to a GPU main memory for storage, and obtaining the compressed output data and the main memory allowance of the GPU; when the main memory margin does not reach the preset margin threshold, determining a preliminary hidden layer according to the sparse degree value of the output data and the time proportion of the compressed output data occupying the GPU main memory; when the deep neural network model is iteratively trained, according to the preliminary hidden layer, compressing output data of the preliminary hidden layerto a GPU main memory for storage to obtain a preliminary margin of the GPU main memory until the preliminary margin reaches a preset margin threshold; and when the preliminary margin reaches a presetmargin threshold, determining that the output data needs to be compressed to a final hidden layer stored in a GPU main memory, and performing training to obtain a trained deep neural network model. Byadopting the method, the GPU resource utilization rate can be improved.

Description

technical field [0001] The present application relates to the field of computer technology, in particular to a GPU-based deep neural network model training method, device, computer equipment and storage medium. Background technique [0002] With the development of the Internet and artificial intelligence technology, intelligent tasks such as image recognition, speech recognition, and natural language processing are ubiquitous in life. As one of the most effective algorithms for realizing such intelligent tasks, the neural network has received extensive attention and application in academia and industry. The training of modern deep neural network (DNN, Deep Neural Network) usually relies on GPU (Graphics Processing Unit, Graphics Processing Unit) to train complex hundreds of layers deep network. [0003] In the training process of the current deep neural network, each hidden layer will generate corresponding intermediate output data, such as the feature map matrix, where som...

AI Technical Summary

Problems solved by technology

However, the above method occupies more GPU main memory resources, and there is a problem that the utilization rate of GPU resources is not high

Images