Segmented difference compression and inert decompression method for large-scale graph iterative computation

Abstract

The invention discloses a segmented difference compression and inert decompression method for large-scale graph iterative computation, and relates to the technical field of large-scale graph data compression in high-frequency iterative computation. The method comprises a segmented difference compression process based on cluster distribution characteristics and an on-demand decompression process based on an inert decompression mechanism. On the basis of the characteristic that outgoing edges of a graph have cluster distribution, an outgoing edge sequence of an adjacency list is segmented according to clusters, and on-demand decompression inert decompression technology is designed on the basis of segmentation difference compression, so the decompression problem can be flexibly solved. For message sending of a specific vertex, the decompression process can be completed only by finding the dictionary value corresponding to the segment and the corresponding difference value, and the side table does not need to be completely decompressed; when the method is used for dynamically changing graph data, the changing vertex data can be directly updated, and the whole graph data does not need to be reordered and compressed.

Description

technical field [0001] The invention relates to the technical field of large-scale graph data compression in high-frequency iterative calculations, and in particular to a segment difference compression and lazy decompression method for large-scale graph iterative calculations. Background technique [0002] As the most commonly used data structure in computer science, graph is especially suitable for expressing different entities (vertices) and their relationships (edges) in the real world. The resulting complex network topology results in graph-related queries usually requiring iterative calculations, that is, looping or recursively processing vertices along edges until the convergence condition is met. Graph iterative analysis calculations have been widely used in various fields of national economy and people's livelihood, such as military positioning and urban planning (shortest path calculation, diameter estimation, BFS traversal), social network analysis (connected domai...

AI Technical Summary

Problems solved by technology

For example, the single-source shortest path algorithm updates its own value only when the value of the received message is smaller than the value of the current vertex, becomes an active vertex and sends a message to notify its out-degree neighbors to continue updating the shortest distance; otherwise, the vertex does not It is necessary to access the outbound edge, and the outbound edge data edges decompressed when calling the compute function are useless, resulting in invalid decompression and a waste of computing resources

Images