Machine learning method, master node, work nodes and system

Abstract

The embodiment of the present invention provides a machine learning method, a master node, work nodes and a distributed machine learning system which are used to reduce the synchronization overhead ofmachine learning. The master node of the distributed machine learning system starts a parameter training process, the work nodes that join the parameter training process are determined, and time information corresponding to the parameter training process is sent to the work nodes, wherein the time information includes an end time of the parameter training process. After the work nodes receive a notification that the master node determines that the work nodes join the parameter training process, the time information corresponding to the parameter training process sent by the master node is obtained. The parameter training is carried out in a time range indicated by the time information, and the master node updates global parameters based on obtained training sub results after the master node receives training sub results sent by work nodes which join the parameter training process.

Description

technical field [0001] The invention relates to the technical field of computer communication, in particular to a machine learning method, a master node, a working node and a distributed machine learning system. Background technique [0002] With the advent of the era of big data, big data processing technology has gradually developed. With the increase of input training data and data models, there are problems such as memory limitation and time limitation in machine learning training of a single node, so distributed machine learning came into being. The distributed machine learning system includes a master node and multiple working nodes. Its core goal is that the master node disassembles computing tasks into multiple small tasks, and distributes them to the processors of multiple working nodes for calculation. That is to say, different working nodes correspond to the same training model. After each working node is assigned different data for parameter training, the traini...

AI Technical Summary

Problems solved by technology

Some of the working nodes have strong processing capabilities and can complete their own training tasks in a short period of time, while some working nodes have weak processing capabilities and take a long time to complete the training tasks

In addition, once a working node has a large delay due to some reasons, the working node that completed the training first needs to wait for other working nodes to complete the training before proceeding to the next round of training, resulting in a long idle time and a large synchronization overhead

Images