Deep learning model training method, working node and parameter server

Abstract

The embodiment of the invention provides a deep learning model training method, a working node and a parameter server. The deep learning model training method applied to the working node comprises thesteps that a first statistical parameter sent by a parameter server is received, and the first statistical parameter is determined by the parameter server according to historical training data of a target layer of a target model; when the target layer is trained based on target batch training samples, target statistical parameters of the target layer are obtained, and the target statistical parameters are statistical parameters of the target batch training samples; and an actual statistical parameter of the target layer is determined based on the first statistical parameter and the target statistical parameter, batch standardization is performed on the target batch training samples based on the actual statistical parameter, and the target statistical parameter is sent to the parameter server. According to the embodiment of the invention, the training efficiency of the deep learning model can be improved.

Description

technical field

[0001] The invention relates to the technical field of deep learning, in particular to a deep learning model training method, a working node and a parameter server. Background technique

[0002] With the development of information technology, the use of deep learning models for training to use the trained models to predict target data has been more and more widely used. In order to further improve the accuracy of the trained models, the training sample The number is also increasing, which results in the complexity of training and longer training time.

[0003] In related technologies, usually multiple working nodes can be used to train the same model. For example, different working nodes are responsible for training different training layers in the same model. At this time, the next training layer needs to wait for the training of the previous training layer to complete. Being able to perform a training process whose waiting time greatly increases the overal...

Images