Graph big data streaming division method based on graph algorithm load

A data stream and graph algorithm technology, applied in the field of data processing, can solve the problems of not considering the algorithm, unsatisfactory balanced partition effect, not considering application tasks and task loads, etc., to achieve the effect of meeting the partition requirements

Active Publication Date: 2020-05-26
TONGJI UNIV
View PDF1 Cites 5 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] The purpose of the present invention is to overcome the problem that the above-mentioned prior art graph partition algorithm only considers information such as vertices and edges, does not consider the algorithm use...

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Graph big data streaming division method based on graph algorithm load
  • Graph big data streaming division method based on graph algorithm load
  • Graph big data streaming division method based on graph algorithm load

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0050] Such as figure 1 As shown, this embodiment is a streaming partitioning method for graph big data based on graph algorithm load. First, the graph data is loaded in a streaming manner; then, according to the parallel computer architecture and its performance parameters, graph algorithms, and partition requirements Factors, select the appropriate measure, and calculate the expected measure value under the measure; finally, according to the algorithm operation measurement mechanism of the following figure and sub-graph of the corresponding measure, according to different heuristic rules, implement arbitrary proportion stream partitioning, will reach The vertices of are placed into different subgraphs. This method includes the following three core contents:

[0051] (1) Define the problem of arbitrary ratio division of large graph data: Aiming at the problems of large overhead, poor flexibility, and narrow applicability in the current graph partition algorithm of equal scal...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a graph big data streaming division method based on graph algorithm load. The graph big data streaming division method comprises the following steps: a data loading and initializing step: loading graph big data and initializing sub-graphs; a measurement selection and calculation step: calculating an expected measurement value of each sub-graph according to a target of graph big data parallel processing; a heuristic rule selection step: selecting a heuristic rule from a pre-established heuristic rule set; and a sub-graph division step: calculating preset positions of vertexes of the graph big data through a graph big data processing system, dividing the sub-graphs, and calculating the preset positions based on expected measurement values and heuristic rules of the sub-graphs. Compared with the prior art, the method has the advantages that a non-equilibrium division mode is adopted, the division metric measure is selected according to a parallel computer system structure, graph algorithm execution behavior characteristics and the like, the heuristic rule is selected according to different division targets, and diversified graph division requirements and targets in various application fields can be met.

Description

technical field [0001] The invention relates to the field of data processing, in particular to a method for stream division of graph big data based on graph algorithm load. Background technique [0002] Graph data division is widely used in various industries, especially as the scale of graphs is getting larger and larger, which cannot be processed by a single machine and needs to be processed by multi-machine parallel computing. Graph division is becoming more and more important. The graph size reflects the number of vertices and edges included in the graph data. If the graph data includes a large number of vertices and edges, it is large-scale graph data. [0003] At present, the methods of graph big data partitioning mainly include balanced partitioning, unbalanced partitioning, streaming partitioning, etc. Balanced partitioning divides the graph evenly into subgraphs of the same or similar size according to the number of vertices and the number of connection edges. Un...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F16/906G06F16/901
CPCG06F16/906G06F16/9024
Inventor 曾国荪程腾腾丁春玲
Owner TONGJI UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap