Block-based subgraph construction and distributed graph processing method

A construction method and distributed technology, applied in the field of computer big data processing, can solve problems such as insufficient utilization of network bandwidth, and achieve the effect of improving overall performance, low cutting rate and high performance

Active Publication Date: 2016-05-18
HUAZHONG UNIV OF SCI & TECH
View PDF2 Cites 23 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The main reason for the high communication overhead is that this fine-

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Block-based subgraph construction and distributed graph processing method
  • Block-based subgraph construction and distributed graph processing method
  • Block-based subgraph construction and distributed graph processing method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0034] In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not constitute a conflict with each other.

[0035] The block-based subgraph construction and distributed graph processing method provided by the present invention, its flow is as follows figure 1 As shown, including the data preprocessing steps and execution steps of subgraph construction;

[0036] Among them, the data preprocessing steps include the steps of graph segmentation, relabeling, generating data slices, partitioning and genera...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a block-based subgraph construction and distributed graph processing method. The block-based subgraph construction method comprises partitioning a graph, relabeling summits in the graph, associating intervals with shards, partitioning blocks and constructing subgraphs, and employs a heuristic and lightweight SGP (Streaming Graph Partition) method to partition a graph, and a user-defined heuristic function to designate the summits to P subgraphs in order; the method possesses the characteristics of high performance and low edge cutting rate, and considers subgraph work load balance. According to a distributed graph processing system, after partitioning, subgraphs perform communication by employing a whole memory block as a unit; compared with the fine grit communication mode of the prior art, the distributed graph processing system exchanges data based on a memory block, and can fully utilize network bandwidth and reduce communication time.

Description

technical field [0001] The invention belongs to the technical field of computer big data processing, and more specifically relates to a block-based subgraph construction and distributed graph processing method. Background technique [0002] Under the current big data background, there are more and more application requirements for analyzing, processing and mining large-scale graph data. In recent years, researchers have proposed some distributed graph processing frameworks, such as Google's Pregel, Carnegie Mellon University's DistributedGraphLab and PowerGraph, and Stanford University's GPS; Distributed to each computing node of the cluster; during the computing process, vertices interact through message passing or direct access to distributed shared memory. This kind of fine-grained interaction brings huge system communication overhead; for example, when GPS executes the PageRank graph algorithm, its communication time accounts for 95% of the entire running time. The mai...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06T7/00H04L29/08
CPCH04L67/10
Inventor 王芳程永利冯丹汪修能张永选戎佳磊蒋子威
Owner HUAZHONG UNIV OF SCI & TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap