Super computer benchmark test acceleration method based on connected component generation optimization

Abstract

The invention discloses a super computer benchmark test acceleration method based on connected component generation optimization, and aims to minimize a communication path, maximize a memory access bandwidth utilization rate and accelerate a super computer big data benchmark test. According to the technical scheme, by means of the characteristic that a graph generated by Graph500 contains a plurality of connected components, the connected components are quickly found in the graph, the connected components are stored through a two-dimensional vector, path compression is conducted on the set membership of vertexes in the connected components, two connected components with different root vertexes are combined. And the vertexes of the same connected component are divided to physical nodes with shorter communication paths in the super computer, so that the communication overhead is small and the operation speed is high during graph traversal access. All connected components in the graph can be effectively and quickly stored, the merging speed is furthest improved, the query speed of the root vertex is accelerated, the occupation overhead of a stack in a memory is reduced, and the test speed of the big data processing capability of a super computer is improved.

Description

technical field [0001] The invention relates to a supercomputer big data benchmark test acceleration method, in particular to a method for benchmark test acceleration based on two-dimensional vector and path compressed connected component generation optimization. Background technique [0002] As a common data structure, a graph can be used to abstractly express various complex relationships among real things. For example, social networks, the World Wide Web, etc. can all be represented by graphs. Graph computing is the processing and calculation of graph data, which plays an important role in many scenarios in real life. In recent years, the scale of graph data has continued to grow. According to relevant reports, in the third quarter of 2020, Facebook’s daily active users were 1.82 billion, and Tencent’s WeChat and WeChat monthly active users reached 1.21 billion, abstracting the relationship between users and them is the points and edges in the graph, then the scale of t...

AI Technical Summary

Problems solved by technology

In the traditional union search algorithm, only the father vertices of some vertices are changed during the union (merging), resulting in multiple calls to the function of finding the father vertices (including function recursion) in the find (find) phase. In large-scale graph calculations, The overhead of function recursion is undoubtedly huge

Therefore, such a union search algorithm cannot be directly used for supercomputer big data benchmark acceleration

[0016] Although there are also optimization algorithms for path compression to optimize and search algorithms, for example, the 1985 SIAM Journal on Computing (SICOMP) paper "On the expected performance of path compression algorithms" (on the expected performance of path compression algorithms) analyzed path compression The complexity makes it unnecessary to call the function of finding the parent vertex multiple times when doing the search, reducing function recursion. However, the current path compression optimization algorithm does not consider the coupling degree with the data structure, resulting in a mediocre optimization effect.

Therefore, the union search algorithm with path compression cannot be directly used for supercomputer big data benchmark test acceleration

Images