Unlock instant, AI-driven research and patent intelligence for your innovation.

A large-scale graph data streaming partitioning method and system based on concurrency improvement

A data stream, large-scale technology, applied in the transmission system, electrical components, etc., to achieve the effect of improving the division efficiency

Active Publication Date: 2018-06-12
HUAZHONG UNIV OF SCI & TECH
View PDF5 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0010] Aiming at the above defects or improvement needs of the prior art, the present invention provides a method and system for stream division of large-scale graph data based on concurrency improvement, which solves the problems of existing stream streams by sending multiple vertices and their related information at one time. The graph partition method handles the problem of one vertex with a network round-trip propagation delay, reduces the impact of network delay on the system, and improves the efficiency of graph partitioning.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A large-scale graph data streaming partitioning method and system based on concurrency improvement
  • A large-scale graph data streaming partitioning method and system based on concurrency improvement
  • A large-scale graph data streaming partitioning method and system based on concurrency improvement

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0026] In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not constitute a conflict with each other.

[0027] figure 1 It is a flow chart of the large-scale graph data stream division method of the present invention, which specifically includes the following steps:

[0028] Step 1 Work node registration synchronization:

[0029] All working nodes send their SessionId (composed of IP and port number) to the proxy server, and the proxy server will number them as Ids according to the order in whic...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a method and system for stream division of large-scale graph data based on concurrency improvement, belonging to the field of computer storage. The invention includes: working node registration synchronization; proxy server sending vertex information; working node returning gradient information; proxy server sending optimal partition information; working node saving partition result. The invention solves the problem of processing one vertex at a time in a network time delay in the existing stream graph division method by sending multiple vertices and related information at one time, reduces the influence of the network time delay on the system, and improves the efficiency of graph division.

Description

technical field [0001] The invention belongs to the technical field of computer storage, and more specifically relates to a method and system for stream division of large-scale graph data based on concurrency improvement. Background technique [0002] Graph partitioning refers to splitting a large-scale graph data into several parts and dispersing them into a distributed system for processing. The graph partition algorithm generally has three goals. The first is to ensure that the amount of data contained in the divided partitions satisfies a certain balance; It means communication between hosts; the third is that the algorithm can efficiently complete the division. [0003] The static graph partitioning algorithm handles the partitioning of all points in the graph data according to the information of the entire static graph. When the scale of the graph data is small, the static graph partition algorithm can handle it effectively and obtain a smaller edge cutting rate. Ho...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): H04L29/08
CPCH04L67/10H04L67/563
Inventor 施展冯丹鲍匡迪郭鹏飞韩江黄力余静欧阳梦云
Owner HUAZHONG UNIV OF SCI & TECH