A data storage method and system suitable for social network graphs

Abstract

The invention relates to a data storage method and system suitable for a social network graph. The method includes the steps that graph structure information of the social network graph is read in a side list or an adjacency list mode; a peak is separated into a plurality of sub peaks, and each sub peak has all the peak data and data on corresponding sides of the peak; with regard to sides, which are not distributed in partitions, in the graph, firstly the sides are distributed to one partition, and at the moment sides spanning the partitions are not generated; if the distribution is not succeed, one side spanning the partitions is generated. The system comprises a plurality of management nodes, a plurality of work nodes, and ZooKeeper clusters. Only one management node is in the working state at a same moment, and each work node comprises a PHT index, a graph structure storage, a request processor and a database interface. The data storage method and system suitable for the social network graph can be increased to reflect the change of a social network graph structure, reduces communication cost among servers, increases overall throughput of the servers, and improves user experience.

Description

technical field [0001] The invention belongs to the field of cloud storage, specifically designing a data storage method and system suitable for social network graphs. Background technique [0002] With the continuous development of social networks such as Renren.com, Facebook, and Weibo, more and more UGC (User Generated Content) data are posted to social networks by users, and the data in social networks is growing explosively. The company presented serious challenges. [0003] In order to cope with the increasing data expansion, due to the needs of fault tolerance and load balancing, the existing website architecture uses traditional distributed technology, adopts horizontal expansion, and uses simple hash, consistent hash or DHT technology to distribute data in multiple servers. The advantage of this data placement method is that it is uniform, consistent, and easy to maintain, but it ignores the data access mode of the social network. [0004] Friend relationships or...

AI Technical Summary

Problems solved by technology

These algorithms are problematic: First, social network graphs are different from general graphs in that they have high average degree of vertices, close connections, and small diameters, which conform to Power-law characteristics. Research has proved that such graphs are difficult to segment, and the effect Similar to simple hashing, cannot be used in real-time systems

Secondly, in the production environment, the social network is changing frequently, and these algorithms have to re-segment the whole graph every time, which cannot meet the real-time requirements of the production environment.

Third, these algorithms are sensitive to input. Small changes in the graph structure will cause huge changes in the segmentation results. The resulting migration cost will greatly affect system performance, which is unbearable in real-time systems.

Images