Data processing method, related equipment and system

Abstract

The embodiment of the invention discloses a data processing method, related equipment and a system. The method is applied to a first intermediate node of a deep learning system. The deep learning system comprises an intermediate node, a child node set and a root node, wherein the child node set comprises at least one child node; the method comprises the steps that a first intermediate node receives first variable data of a target application sent by a next-hop node set, the target application corresponds to the first intermediate node, the first intermediate node is any intermediate node connected with a root node in a deep learning system, and the next-hop node set is a child node set or a next-hop intermediate node set; the first intermediate node performs aggregation processing according to the first variable data to obtain second variable data; and the first intermediate node sends the second variable data to the root node, and the second variable data are used for the root node toperform aggregation processing, so that the waste of computing resources on the child nodes can be reduced to a certain extent, and the computing efficiency is improved.

Description

technical field [0001] The present invention relates to the field of computer technology, in particular to a data processing method, related equipment and system. Background technique [0002] At present, machine learning has become a hot spot in the development of the information technology industry. Deep learning algorithms represented by deep neural networks have made great progress and been widely used in recent years. The data aggregation mode is a commonly used variable synchronization mode in the calculation of deep learning algorithms. [0003] Such as figure 1 As shown, after multiple sub-nodes (workers) perform a round of training calculations, each sub-node will generate a copy of variable data; the sub-nodes will transmit the variable data to the root node (paramenter server, ps), multiple copies of variable data in a After the root node gathers a new variable data, the child nodes perform the next round of training calculations based on the new variable data. ...

AI Technical Summary

Problems solved by technology

[0006] The technical problem to be solved in this application is to solve the problem of how to reduce the waste of computing resources on child nodes to a certain extent and improve computing efficiency

Images