Data compression method, decompression method, system and related device for graph data

A compression method and technology for graph data, applied in file systems, file system functions, special data processing applications, etc., can solve problems such as large decoding logic, time-consuming, and difficulty in copying multiple node information in parallel, and increase processing bandwidth. Effect

Active Publication Date: 2022-08-02
INSPUR BEIJING ELECTRONICS INFORMATION IND
View PDF8 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Since the length of column nodes is randomly distributed, it will be very difficult to replicate multiple node information in parallel on hardware
For example, in a graph data with a parallel processing capacity of 128 nodes, the input row nodes correspond to several column nodes, and copying at this time requires huge decoding logic and time-consuming

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data compression method, decompression method, system and related device for graph data
  • Data compression method, decompression method, system and related device for graph data
  • Data compression method, decompression method, system and related device for graph data

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0038] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments It is a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of this application.

[0039] Please refer to figure 1 , figure 1 A flowchart of a compression method for a graph computing system provided by an embodiment of the present application, the method includes

[0040] S101: Obtain original graph data and determine the node pair of each edge in the original graph data;

[0041] The original graph data exists i...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The present application provides a method for compressing graph data, including: obtaining original graph data and determining a node pair of each edge in the original graph data; grouping the same source nodes in the node pair according to preset grouping rules; A source node; store the corresponding FIFO according to the group type of each group; add labels to the source node data of the source node and the destination node data of the destination node; poll and store the FIFO to obtain compressed graph data. The present application enables source nodes with different numbers of corresponding destination nodes to adopt different compression rates, only one source node is reserved in the same group, and there is no need to store duplicate source nodes. It can support large-scale parallel decompression of graph data and greatly increase the bandwidth of graph data processing. The present application also provides a data compression system for a graph computing system, a computer-readable storage medium, and a data compression terminal, which have the above beneficial effects.

Description

technical field [0001] The present application relates to the field of data storage, and in particular, to a compression method, decompression method, compression system and related devices for graph data. Background technique [0002] In the era of big data, a large amount of data interacts with each other to form a large-scale graph, which is widely used in various algorithms such as PageRank. At present, the scale of graphs often reaches the order of tens of millions to hundreds of millions of nodes, and the edge (connection between nodes and nodes) information of the graph reaches the scale of billions. Usually, Internet graph data is in a sparse matrix format. The storage methods that can be used include COO and CSR. COO adopts the triple format of <source node, destination node, weight>. Considering that a node is represented by 4 bytes, then a Edges require 12 bytes of storage. CSR considers that nodes in the same row have the same address and only stores the ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/17
CPCG06F16/1744
Inventor 梅国强郝锐王江为阚宏伟
Owner INSPUR BEIJING ELECTRONICS INFORMATION IND
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap