Data training method, device and system, computer equipment

Abstract

The invention discloses a data training method, device and system and computer equipment, and belongs to the field of artificial intelligence. The method comprises the following steps: acquiring the same pseudo-random number seed with other training hosts, wherein the other training hosts are training hosts except a first training host in the plurality of training hosts; Executing at least one training process, until the execution result of the training process meets a preset stop condition; The training process comprises the following steps: performing randomization processing on the file name set based on pseudo random seeds; obtaining an updated file name set, training a first data subset in a training data set on the basis of the updated file name set until the training hosts completetraining of the training data set at this time; executing the training process again, wherein the first data subset comprises data corresponding to the first training host. According to the invention,the problems of high data training network overhead and poor data training system stability in the prior art are solved. The method is used for data training.

Description

technical field [0001] The present invention relates to the field of artificial intelligence, in particular to a data training method, device and system, and computer equipment. Background technique [0002] Deep learning is a relatively effective artificial intelligence method. Deep learning can learn relevant knowledge from data by simulating the learning behavior of the human brain, and use the learned knowledge for subsequent predictions. At present, in large-scale deep learning training, due to the large amount of data in the training data set, in order to improve the efficiency of data training, multiple training hosts are usually used to train the data at the same time. [0003] In related technologies, during the initialization process of data training, each training host can obtain and store the same training data set (that is, the training data sets stored in each training host are the same), and the designated training host A based on the training data set genera...

AI Technical Summary

Problems solved by technology

[0006] The embodiment of the present invention provides a data training method, device and system, and computer equipment, which can solve the problems of relatively large network overhead for data training and poor stability of the data training system in the related art

Images