A fault-tolerant method and system for graph computing based on message regeneration mechanism

A graph computing and mechanism technology, applied in the field of cloud computing, can solve the problems of subtasks without any dependence or communication, large network and disk overhead, and slow computing speed, so as to improve the efficiency of fault recovery, easy maintenance, and reduce the amount of data Effect

Inactive Publication Date: 2017-01-25
PEKING UNIV
View PDF5 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] 2. There is no dependency or communication between subtasks
Although this method is simple to implement, its network and disk overhead are very large, which seriously slows down the calculation speed, so this function is often turned off in practical applications

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A fault-tolerant method and system for graph computing based on message regeneration mechanism
  • A fault-tolerant method and system for graph computing based on message regeneration mechanism
  • A fault-tolerant method and system for graph computing based on message regeneration mechanism

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0032] The present invention will be further described below through specific embodiments and accompanying drawings.

[0033] Traditional snapshot data includes two parts: a) graph structure data snapshot; b) cached message data snapshot for the next Superstep. The present invention reduces the data volume of these two parts through two different methods, realizes the light weight of the snapshot data through the incremental snapshot of the graph structure data and the regeneration mechanism of the message data, achieves the purpose of rapid generation and recovery, and greatly shortens the snapshot time. generation and recovery times.

[0034] 1. Incremental graph structure data snapshot

[0035] For graph structure data, compared to the traditional method of snapshotting all graph structure data, the present invention uses incremental snapshots, that is, adopts a method based on log append to save graph structure changes between two snapshots. Formally, let Δ i，j is the c...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a diagram calculation fault-tolerant method and system based on an information regeneration mechanism. According to the method, in the diagram calculating process, change information of a diagram structure between two adjacent snapshots is stored for diagram structure data, a peak value set is stored for information data, a diagram calculation system is recovered to a Superstep corresponding to the effectively snapshots through the stored change information of the diagram structure and the stored peak value set when efficacy losing occurs, and then calculation of the next superstep is started. The system comprises a control node, multiple calculation nodes and a distributed file system, and in order to better adapt to the efficacy losing condition of the diagram calculation system, the method and the system can lighten snapshot data, greatly shorten the generation time and the recover time of the snapshots, and improve the fault tolerance capacity of the diagram calculation system.

Description

technical field [0001] The invention belongs to the field of cloud computing, and in particular relates to a high-efficiency fault-tolerant method and system based on a message regeneration mechanism for a graph computing system. Background technique [0002] Graph computing systems are increasingly being used by various enterprises for daily data analysis and computing. Many open source graph computing frameworks have emerged in the open source world, such as Apache Giraph and PowerGraph. Inspired by Pregel, these graph computing system frameworks all adopt vertex-based semantic models and BSP (Bulk Synchronous Parallel) computing models, read graph data from distributed storage systems, perform iterative calculations in memory, and finally output the calculation results to the database or file system. [0003] A graph computing task is divided into several supersteps. In each superstep, each vertex will process a message from a superstep on a neighboring vertex, and send...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06F11/14
Inventor 薛继龙曲直杨智代亚非
Owner PEKING UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap