Large-scale graph deep learning calculation framework based on hierarchical optimization normal form

Abstract

The invention provides a large-scale graph deep learning calculation framework based on a hierarchical optimization normal form, and relates to the technical field of deep learning. Block-based data stream conversion is used for dividing vertex and edge data of a graph into small blocks; optimizing of the data flow diagram is used for generating a scheduling strategy; propagation of the kernel onthe GPU is used for efficient kernel propagation operation; parallel processing of multiple GPUs is used for parallel computing of the multiple GPUs in one server. Large-scale graph neural networks (GNNs) are supported, the model can be simply expressed, and a telescopic and efficient GPUs parallel processing engine is supported. In order to represent recursive calculations of each layer of GNN including graph propagation and deep neural network (DNN) calculations, a neural network (Separation-UDF _ Edge-Aggregation-UDF _ Vertex with Neural Networks, SUAU-NN) vertex program abstraction with the functions of dispersion, side application, aggregation and vertex application is adopted.

Description

technical field [0001] The invention belongs to the technical field of deep learning, and in particular relates to a large-scale graph deep learning computing framework based on a layered optimization paradigm. Background technique [0002] Deep learning in the form of deep neural networks has been successfully applied to many fields, such as speech, vision, natural language processing, etc. In these fields, the representation of the underlying data usually adopts a regular grid structure, which is conducive to hardware acceleration (such as GPU) when a large amount of data is parallelized. Driven by the importance of graph data in social networks, knowledge graphs, bioinformatics, and neuroscience, applying deep learning models to data with irregular graph structures is an emerging trend. Use advanced prediction results in target applications, such as classification, embedding, question answering systems. These graph-based neural networks typically apply neural network mo...

AI Technical Summary

Problems solved by technology

[0003] However, none of the existing solutions can support GNNs well. Existing graphics processing engines usually provide a Gather-Apply-Scatter (GAS) vertex program model, which is unable to express and Neural Network Framework Supporting Graph Structure

Deep learning frameworks are designed as data flow graphs to express neural networks, such as TensorFlow, PyTorch, MxNet, CNTK, etc., but cannot directly support graph propagation models

Furthermore, none of them provide the scalability needed to handle large-scale graphs, nor support efficient implementations of GPU-based graph propagation operators.

The current lack of support for these requirements severely limits the ability to fully exploit the potential of large-scale GNNs, while also presenting enormous challenges for combining DNNs with large graph structures at the system level.

Images