Online graph division method for heterogeneous graph data

Abstract

The invention discloses a heterogeneous graph data-oriented online graph division method. The method comprises the following steps of: 1) evaluating the calculation speed imbalance and the storage space imbalance of a graph calculation system; determining the calculation speed imbalance of the graph calculation system according to the node function time complexity T of different types of nodes in the heterogeneous graph calculation of the graph calculation system; according to the storage space Sv occupied by the data carried by the different types of nodes and the storage space Se occupied by the data carried by the different types of edges in the heterogeneous graph calculation performed by the graph calculation system, determining the storage space imbalance of the graph calculation system; and 2) according to the node function time complexity T corresponding to different types of nodes, the storage space Sv and the storage space Se corresponding to different types of edges, distributing the current to-be-processed heterogeneous graph data to different partitions. According to the method, the task allocation in the graph calculation is optimized, so that the load and memory use in the graph calculation process are more balanced, and the result of improving the running efficiency of the graph calculation is achieved.

Description

technical field

[0001] The invention belongs to the technical field of graph computing and real-time, and in particular relates to an online graph division method for heterogeneous graph data. Background technique

[0002] Graph is an important data structure with more powerful expression ability, and can integrate different sources and different types of data into the same graph for analysis, and obtain results that are difficult to find by independent analysis. Therefore, graph computing can be widely used. It is widely used in social networks, recommender systems, network security, text retrieval, and biomedical fields. Many problems can be efficiently solved with the help of graph-related algorithms under the support of graph theory. In recent years, with the rapid development of technologies such as big data, machine learning, and data mining, the scale of abstracted graphs in many fields has grown exponentially. In order to cope with the challenges brought by massive d...

Examples

Embodiment Construction

[0025] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments and the accompanying drawings. It should be understood that the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts shall fall within the protection scope of the present invention.

[0026] image 3 This is a flow chart of graph division and graph computation in the present invention. The present invention will be described below in conjunction with a heterogeneous graph containing three types of node functions and carrying different amounts of data on edges. The point data of the heterograph that has come:

[0027] vertexId vertexType vertexAttr1 vertexAttr2 vertexAttr... 1 3 0 0 ... 2 2 0 0 ... 3 2 0 0 ....

technical field

[0001] The invention belongs to the technical field of graph computing and real-time, and in particular relates to an online graph division method for heterogeneous graph data. Background technique

[0002] Graph is an important data structure with more powerful expression ability, and can integrate different sources and different types of data into the same graph for analysis, and obtain results that are difficult to find by independent analysis. Therefore, graph computing can be widely used. It is widely used in social networks, recommender systems, network security, text retrieval, and biomedical fields. Many problems can be efficiently solved with the help of graph-related algorithms under the support of graph theory. In recent years, with the rapid development of technologies such as big data, machine learning, and data mining, the scale of abstracted graphs in many fields has grown exponentially. In order to cope with the challenges brought by massive d...

Examples

Embodiment Construction

[0025] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments and the accompanying drawings. It should be understood that the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts shall fall within the protection scope of the present invention.

[0026] image 3 This is a flow chart of graph division and graph computation in the present invention. The present invention will be described below in conjunction with a heterogeneous graph containing three types of node functions and carrying different amounts of data on edges. The point data of the heterograph that has come:

[0027] vertexId vertexType vertexAttr1 vertexAttr2 vertexAttr... 1 3 0 0 ... 2 2 0 0 ... 3 2 0 0 ....

Images