Training data transmission method, device and equipment, and medium

Abstract

The invention discloses a training data transmission method, which comprises the steps of: monitoring a current monitoring value used for reflecting a current training condition in a node, and judgingwhether the current monitoring value is greater than a preset value or not; if so, compressing obtained current training data into a preset type of compressed data; and establishing communication with the destination node, and sending the compressed data to the destination node. And after calculation is completed, decompression operation is carried out to continue the model training process. Therefore, by compressing the current training data into the compression data of the preset type, the data volume transmitted in the communication process is reduced while the model training precision isensured, the pressure of network resources is relieved, and the accuracy and high efficiency in the data transmission process are ensured. In addition, a training data transmission device, training data transmission equipment and a storage medium provided by the invention correspond to the method.

Description

technical field [0001] The present invention relates to the field of communication technology, in particular to a training data transmission method, device, equipment and medium. Background technique [0002] At present, deep learning models have been widely used in various fields, such as computer vision, recommendation system, natural language processing and other fields. In order to make the deep learning model get better training results, the model parameters have reached the order of 1 billion. In response to this situation, researchers use the distributed computing framework to distribute the training process of the deep learning model to multiple computing nodes to enable parallel computing, and then establish communication during the training process to realize gradient data in the backpropagation process. unanimous. The current distributed computing framework supports data types such as int32, float32, and float16 for computing. [0003] Because in the cloud comp...

AI Technical Summary

Problems solved by technology

[0004] However, with the increase in the number of layers of the deep learning model, the scale of the parameter volume is getting larger and larger, and the float16 type data also generates a large amount of training data in the large-scale training process, and there will still be an incommunicable bandwidth called model training. The bottleneck situation does not fundamentally solve the problem

Images