Storage optimization-based distributed graph processing method

A technology for storage optimization and graph processing, which is applied in the field of graph computing, can solve problems such as load imbalance, speed up graph data processing, and overall performance degradation, and achieve the effect of eliminating working nodes, increasing speed, and improving performance

Active Publication Date: 2017-09-01
HUAZHONG UNIV OF SCI & TECH
View PDF5 Cites 21 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] Aiming at the above defects of the prior art, the present invention provides a storage-optimized distributed graph processing method, which performs partition storage and IO balance on graph data, realizes graph data

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Storage optimization-based distributed graph processing method
  • Storage optimization-based distributed graph processing method
  • Storage optimization-based distributed graph processing method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0034] In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not constitute a conflict with each other.

[0035] figure 2 The system architecture diagram of the distributed graph processing system optimized for the storage of the present invention, the system consists of two parts, the master control node and the work node, the master control node controls the execution of the entire graph processing system, and the work node completes the basic process of graph processing, A computational model t...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a storage optimization-based distributed graph processing method, and belongs to the field of graph calculation. The method comprises the following steps of: carrying out data partitioning in a data pre-processing stage; distributing graph partition data; starting data iteration processing; updating message transfer; making a work node extension decision; and ending the data processing. According to the method, a consistency hash algorithm is proposed to partition and store graph data, and an external storage mode-based distributed graph processing system is designed and realized; and by utilizing a dynamic storage optimization strategy, graph data processing load balance is realized according to partition storage of load adjustment graphs, so that the graph data processing speed is improved, the problem that the overall performance is reduced as hotspots are caused by load imbalance in the graph data processing process in the prior art is solved, and then the graph processing performance is improved.

Description

technical field [0001] The invention belongs to the field of graph computing, and more specifically relates to a storage-optimized distributed graph processing method. Background technique [0002] As a classic data structure, a graph expresses complex data relationships through points and edges. It has been widely used in various fields of society, including social data analysis and mining in the Internet field, protein interaction in the chemical field, and prediction of disease outbreak paths in the medical field. The citation relationship of literature in the academic field, etc., has derived many important algorithms, including PageRank, shortest path, connected branches, and extremely independent sets. Because graph data is of great significance and requires a lot of calculations, various graph processing systems have emerged. [0003] The first is the distributed memory mode graph processing system, including Pregel, GraphLab, etc. These systems first put all the inf...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F9/50G06F9/52G06F9/54G06F3/06
CPCG06F3/061G06F3/064G06F3/0644G06F3/067G06F9/5083G06F9/52G06F9/546
Inventor 施展冯丹单玉祥李君浩毛艳张芸怡方交凤
Owner HUAZHONG UNIV OF SCI & TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap