Streaming graph calculation method and system based on state update propagation

Abstract

The invention relates to a streaming graph calculation method and system based on state update propagation. The streaming graph calculation model based on state update propagation can concurrently calculate the influence of incremental information on the basis of the original graph state without recalculating on the whole graph. At the same time, through the fine-grained distributed locking, the concurrent updating of the state is realized, and the correctness of the calculation results is ensured. The real-time performance and the accuracy of the invention are tested through a real data set,and the results show that the algorithm constructed by the flow chart calculation model based on the state update propagation can obtain more accurate calculation results, and the calculation deviation is less than 1%; 90% of the graph data update requests can be responded within 12 ms, which meets the real-time requirements; the collision probability of any two compute nodes is less than 3%, which can meet the requirement of high concurrency.

Description

technical field [0001] The invention relates to a method and system for calculating a flow graph, in particular to a method and system for calculating a flow graph based on state update propagation, which uses an incremental method for updating, and belongs to the field of software technology. Background technique [0002] Graph data is a type of data structure commonly used in computer science, which well expresses the relationship between data. A lot of data in the real world can be abstracted into graph data, such as the links between web pages, the interaction between social figures, and the transactions between buyers and sellers can all be abstracted into graphs formed by interrelated relationships. With the rapid development of the Internet, the total amount of graph data is also increasing dramatically. For example, as of the fourth quarter of 2016, Facebook had 1.86 billion active users, and each user had an average of 155 friends; the number of vertices in the gra...

AI Technical Summary

Problems solved by technology

For accurate calculations, the existing KineoGraph (Cheng R, Hong J, Kyrola A, et al. Kineograph: taking the pulse of a fast-changing and connected world[C] / / Proceedings of the 7th ACM european conference on Computer Systems. ACM, 2012:85-98.) and IncGraph proposed to use the incremental computing model for real-time computing, but this incremental update is performed serially, and the real-time performance is limited

SpecGraph (Jing Nianqiang, Xue Jilong, Qu Zhi, Yang Zhi, Dai Yafei. SpecGraph: A distributed real-time graph computing model based on concurrent updates [J]. Computer Research and Development, 2014, (S1): 155-160.) Although On the basis of the above-mentioned incremental model, a concurrent update model based on a speculative mechanism is proposed. However, this model assumes that the state of the vertex only depends on the information currently received by the vertex, and has nothing to do with the old state before the vertex. This assumption The applicability of the system is poor. In many algorithms, the state of the vertex is not only related to the message received by the vertex, but also related to the old state of the vertex. Therefore, the expressive ability of the model is limited.

[0007] In short, the existing technology has the problems of limited real-time performance, poor system applicability, and limited model expression ability.

Images