Model training method and related device

Abstract

The embodiment of the invention discloses a model training method and a related device. The model training method comprises the following steps: performing parallel training on a network model throughN processing nodes; determining M processing nodes in the N processing nodes when the ith training iteration is finished, wherein M is less than N; and obtaining model parameters of a network model trained by the M processing nodes as to-be-fused parameters, and determining initial model parameters of a network model trained by a target processing node at the beginning of (i + 1) th training iteration according to the to-be-fused parameters, wherein the target processing node is a processing node except M processing nodes in the N processing nodes. As the M processing nodes are local processing nodes in the N processing nodes, the initial model parameters can reflect the training characteristics of the local processing nodes, and the diversity of the initial model parameters is enhanced,and the over-fitting problem of the network model when the training is finally completed is reduced, and the model quality is ensured on the premise of improving the training efficiency.

Description

technical field [0001] The present application relates to the field of data processing, in particular to a model training method and related devices. Background technique [0002] With the development of artificial intelligence technology, various services can be provided to users through neural network models, such as speech recognition, image recognition, search, etc. A high-quality neural network model can only be obtained after training with a large amount of training data. When the magnitude of the training data is large, the time required to complete the training is very considerable, and it is difficult to meet the growing service needs. [0003] To solve the problem of high training time consumption, some related technologies have proposed a solution for parallel training of multi-processing nodes. For data sets that include massive training data, the same initial model is trained in parallel through multiple processing nodes. The training process will include multi...

AI Technical Summary

Problems solved by technology

[0005] However, the network model determined according to this multi-processing node parallel training method may have over-fitting problems in some cases, resulting in a high error rate for the network model

Images