Parallel computing method and application of graph roaming based on pruning

Abstract

The invention discloses a picture roaming parallel computing method based on pruning and application, and relates to the technical field of large-scale data parallel computing and processing. The method comprises the steps that efficient picture roaming based on pruning is achieved in a shared memory system, boundary points in pictures are recognized according to topology features in the preprocessing stage before computing, and corresponding pruning computing operation is carried out in the computing process on the basis. According to the picture roaming parallel computing method based on pruning and application, by pruning the boundary points, a vertex can be spread in a higher updating speed, the convergence rate of an algorithm can be higher, and thus the number of iteration rounds is reduced; when tasks are partitioned in parallel computing, task partitioning is more balanced by pruning the boundary points, and thus the purposes of reducing computing expenses and improving computing performance are achieved.

Description

technical field [0001] The invention relates to the technical field of large-scale data parallel computing and processing, in particular to a pruning-based graph roaming parallel computing method and application. Background technique [0002] In recent years, with the continuous development of the Internet, more and more information and data need to be stored and calculated using the graph structure, for example, friend relationships in online social networks, citation relationships in paper libraries, website links recorded in search engines Relationships and knowledge graphs in Wikipedia, etc. These graph calculations often have huge data scales, with the number of nodes and edges reaching GB / TB / ZB levels, or even larger. Large-scale graph computing has difficulties such as low computing density and random access to data. Parallel computing is a common technical means for large-scale graph computing problems. [0003] A graph is a data structure, and a graph G usually c...

AI Technical Summary

Problems solved by technology

On the one hand, although the distributed system has stronger computing power and storage capacity, for a distributed system with a large number of nodes, the low computing density and the characteristics of random data access lead to a large communication overhead during the computing process. The performance improvement brought by technology does not match the increase in system cost, and the computing power cannot be fully utilized; on the other hand, Pregel directly calculates the input graph without a preprocessing mechanism, and there will be some unnecessary calculations in the actual calculation process. computing overhead

Images