Method and system for expanding stand-alone graph neural network training to distributed training, and medium

Abstract

The invention discloses a method for expanding stand-alone graph neural network training to distributed training. The method, by providing functions of graph division and data synchronization expansion, using a stand-alone graph neural network frame with automatic back propagation, and just adding a small number of data synchronization codes to original stand-alone graph neural network model codes, and then allocating a large graph to multiple servers, can realize distributed large graph training equivalent to stand-alone graph neural network training by the multiple servers under the condition of not modifying the stand-alone graph neural network framework or modifying calculation logic of an original stand-alone graph neural network model. In addition, the invention also discloses a system for expanding stand-alone graph neural network training to distributed training. In addition, the invention discloses a computer readable storage medium storing a computer program.

Description

technical field [0001] The present invention relates to the field of deep learning and graph neural network, in particular to a method for expanding single-machine graph neural network training to distributed training. Background technique [0002] Graph structure data can represent the correlation between data and can be used to describe many problems in real life. Graph neural network (such as GCN, GAT, GraphSage, etc.), a graph-based deep learning method, can be used to predict the type of nodes on the graph, predict the possibility of edges between nodes, etc., and has achieved great success in many fields. had a very good effect. [0003] Stand-alone graph neural network frameworks (for example, DGL and PyG) provide flexible and convenient programming interfaces and have good stand-alone training performance. However, in actual production applications, the scale of the graph is already very large, and the points and edges have reached hundreds of millions or even bill...

AI Technical Summary

Problems solved by technology

Although this method can also use the same computing logic as a single server to train large-scale graph data, so as to ensure the accuracy of model training and the convergence speed of parameters, it requires users to write forward propagation and reverse propagation of graph computing operations at the same time. The calculation logic of forward propagation, while most of the deep learning frameworks and stand-alone graph neural network frameworks currently adopt the method of automatic backpropagation, which does not require users to write backpropagation steps, because the calculation logic of backpropagation is relatively The calculation logic of direction propagation is more complicated, and it will become more complicated in distributed scenarios. The back propagation logic implemented by the user is prone to errors, and it is difficult for users to verify the correctness of the back propagation logic, so this This solution lacks flexibility, and it is not very convenient to quickly develop and implement the graph neural network model

Images