Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Data access method and system in large graph calculation

A data access and graph computing technology, applied in the field of graph computing, can solve the problems of high I/O overhead and low overall performance of graph computing

Active Publication Date: 2020-01-14
TSINGHUA UNIV
View PDF3 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] In order to overcome the problem in the prior art that the I / O overhead of the external storage device is too large in the graph calculation process, resulting in low overall performance of the graph calculation, the present invention provides a data access method and system in the large graph calculation

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data access method and system in large graph calculation
  • Data access method and system in large graph calculation
  • Data access method and system in large graph calculation

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0043] The specific implementation manners of the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0044] It should be noted that graph calculation is graph algorithm, graph algorithm refers to a simple algorithm that uses a special line graph to obtain the answer. Undirected graphs, directed graphs, and networks can use many commonly used graph algorithms, including: various traversal algorithms, algorithms for finding the shortest path, and algorithms for finding the lowest-cost path in the network. Graph algorithms can be applied to many occasions, such as: optimizing pipelines, routing tables, express services, communication websites, etc.

[0045] Generally, the amount of graph data involved in graph algorithms is relatively large, especially for large graphs with...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a data access method and system in large graph calculation. The method comprises the steps of preprocessing a target graph data file; obtaining a compact graph data file corresponding to the target graph data file, recording active vertexes of the target graph data file in each iteration step through the index bitmap; determining useful edge data and useless edge data in each iteration step according to the active vertexes; determining all useful edge data blocks corresponding to each iteration step according to the useful edge data and the useless edge data; and generating an I / O request according to the initial position and the size of the useful edge data block, so as to directly accessing each piece of edge data in the corresponding useful edge data block from the compact graph data file according to the initial position and the size when the I / O request is processed. According to the method and the system, the addressing overhead of the external storage equipment and the I / O overhead of the external storage equipment are comprehensively considered, the I / O of the external storage equipment of useless side data is reduced to a certain extent, the access sequence of the side data is ensured, and the overall performance of large graph calculation is effectively improved.

Description

technical field [0001] The present invention relates to the technical field of graph computing, and more specifically, to a method and system for accessing data in large graph computing. Background technique [0002] With the rapid development of social network, biological information network and information technology, the graph data with these information objects is increasing day by day. Large-scale computing systems based on external storage devices often use relatively cheap external storage devices to expand the scale of large-scale computing problems due to the limited memory capacity of computers when dealing with large-scale data computing problems. In this way, for such large-scale computing systems based on external storage devices, the I / O of external storage devices is often the performance bottleneck. In addition, the graph algorithm is an iterative algorithm, and in each iteration, not all edge data must be used, so how to effectively reduce the external stor...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F3/06
CPCG06F3/061G06F3/0626G06F3/0659G06F3/067
Inventor 张广艳郑纬民
Owner TSINGHUA UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com