Adaptive neural network training method, electronic device, medium and program product

Abstract

The invention provides a self-adaptive neural network training method, electronic equipment, a medium and a program product, and the method comprises the steps: training a target neural network based on a self-adaptive parameter of a current training round, and the self-adaptive parameter is used for determining the training task amount of each training node for training the target neural network; and based on the training time of each training node in the current training round, adjusting the adaptive parameter of the current training round, and determining the adjusted adaptive parameter as the adaptive parameter of the next training round. According to the method, the electronic equipment, the medium and the program product provided by the invention, the training tasks do not need to be manually allocated to the training nodes with different performances, the labor cost can be reduced, the training efficiency of the deep learning task under the heterogeneous cluster can be improved, and meanwhile, the distributed training mode according to the allocation can be realized. Therefore, the training efficiency of the deep learning task under the heterogeneous cluster is further improved.

Description

technical field [0001] The invention relates to the technical field of deep learning, in particular to an adaptive neural network training method, electronic equipment, media and program products. Background technique [0002] With the rapid development of deep learning technology, deep learning has been widely used in image recognition, natural language processing, speech recognition, reinforcement learning and other fields. In order to achieve better results in practical applications, the deep learning structure represented by the neural network is becoming more and more complex, that is, the number of network layers and parameters of the neural network is increasing continuously, and at the same time, the data set for training the neural network is also increasing. Larger and larger, the training process of deep learning tasks requires a lot of computing resources. However, the computing power of a single machine is limited, which makes the training time too long. [00...

AI Technical Summary

Problems solved by technology

[0005] The present invention provides an adaptive neural network training method, electronic equipment, media and program products to solve the defect of low training efficiency of deep learning tasks under heterogeneous clusters in the prior art

Images