On-chip sorting method based on on-chip communication mechanism and application

Abstract

The invention provides a graph calculation method based on extensible heterogeneous many-core parallel calculation, a heterogeneous many-core calculation system and a computer readable medium. The graph calculation method comprises the following steps: obtaining data of a graph to be calculated, wherein the graph comprises a plurality of vertexes and edges; in the processing of the side message, the message generation part and the message rearrangement part distribute the message to a target node by adopting a bucket distribution step; the slave cores are divided into two classes, namely producers and consumers according to rows, the producers obtain data from the memory and carry out data processing at the current stage, if the data are generated, the data are put into a sending buffer corresponding to the consumer which should process the message according to a target address, and the data are transmitted to the corresponding consumer through the RMA when the buffer is full; the consumer receives the message transmitted by the RMA and carries out subsequent operation needing mutual exclusion; consumers of the same number of each core group are responsible for the same output buckets, and counting of bucket tail positions is achieved through taking and adding atoms on a main memory among the core groups. According to the scheme, slave core acceleration is realized, and the processing efficiency is improved.

Description

technical field [0001] The present invention generally relates to three types of vertex degree-aware 1.5-dimensional graph partitioning methods and applications, and more particularly relates to large-scale graph computing methods, distributed parallel computing systems, and computer-readable media. Background technique [0002] Graph computing frameworks are a class of general-purpose programming frameworks used to support graph computing applications. On China's new generation of Shenwei supercomputer, a new generation of "Shentu" super large-scale graph computing framework is provided to support large-scale graph computing applications with the largest scale of the whole machine, tens of trillions of vertices, and three trillion edges. [0003] Graph computing applications are a class of data-intensive applications that rely on data consisting of vertices and edges connecting two vertices to perform computations. Typical applications include PageRank for web page importa...

AI Technical Summary

Problems solved by technology

For large-scale graph computing, the load of graph data is seriously unbalanced, which is manifested in the severe imbalance of the edges of different vertices, and the degree difference of different vertices is very large. At this time, both one-dimensional and two-dimensional partitions will face scalability problems. One-dimensional vertices The partition method will make too many heavy vertices deploy near-global agents, and the 2D vertex partition method will make too many vertices deploy agents on rows and columns

Images