Vertex reordering-based testing method for supercomputer big data processing capability

A supercomputer and big data processing technology, which is applied in the direction of electric digital data processing, calculation, and other database retrieval, etc., to achieve the effect of improving the hit rate, increasing the hit rate, and reducing the number of memory accesses

Active Publication Date: 2022-02-01
NAT UNIV OF DEFENSE TECH
View PDF2 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0010] The technical problem to be solved by the present invention is: the preprocessing before the BFS traversal is not included in the time-consuming and the high probability of edge relationship between the vertex with high degree and the root node is high, and a supercomputer based on vertex reordering is proposed Big data processing ability test method, reducing the number of memory accesses, avoiding invalid memory accesses to the greatest extent, speeding up the traversal of graphs compared with Graph500, and improving the speed of supercomputer big data processing ability testing

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Vertex reordering-based testing method for supercomputer big data processing capability
  • Vertex reordering-based testing method for supercomputer big data processing capability
  • Vertex reordering-based testing method for supercomputer big data processing capability

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0073] image 3 For the overall flow chart of the present invention, the steps of the present invention are as follows:

[0074] The first step, graph generation. Use the Kronecker graph generator to generate a random graph structure G=(V,E), V is a set of vertices, which contains N V vertices, N V is a positive integer; E is an edge set;

[0075] The second step is to construct an adjacency matrix A that stores the graph G. A ij = 0 means there is no edge between vertex i and vertex j, A ij =1 means there is an edge between vertex i and vertex j, 0≤i≤N V -1, 0≤j≤N V -1, i and j are both positive integers;

[0076] The third step is to preprocess V based on the degree of the vertex. The specific method is as follows:

[0077] 3.1. Traverse each vertex in V and record the degree of each vertex to obtain the set D of vertex degrees.

[0078] The i-th element in D deg(v i ) represents the vertex v i degrees, that is, deg(v i ) vertices and vertex v i There is an edg...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a method for testing the large data processing capability of a supercomputer based on vertex reordering, and aims to improve the testing speed of the large data processing capability of the supercomputer. The technical solution is to generate a graph, construct the adjacency matrix of the graph, sort the vertices in the graph based on the degree of the vertices, and use the sorted vertex set to perform a BFS search on the graph. When traversing the same layer, the vertices with high degrees have a high probability of edge association The characteristics of traversal search the child nodes of the nodes in the vertex set of the current layer, and the vertices with high degree of inspection are prioritized to minimize invalid memory access traversal. By adopting the invention, the hit rate of edge relations between nodes can be improved, the number of invalid memory accesses can be reduced, unnecessary memory accesses can be avoided to the greatest extent, the traversal of graphs can be accelerated, and the test speed of supercomputer big data processing capabilities can be improved.

Description

technical field [0001] The invention relates to a method for testing the large data processing capability of a supercomputer, in particular to a method for testing the large data processing capability of a supercomputer based on vertex reordering. Background technique [0002] Graph structure is one of the most important data structures in big data applications, and has been widely used in many fields, such as social media, bioinformatics, astrophysics, artificial intelligence, data mining, etc. The common feature of these applications is the large amount of data and complex structure, which can often reach billions of edges and trillions of nodes, which leads to higher demands on data storage and computing power. Supercomputers are mainly used for numerical calculations, and most HPC benchmarks use computing power as a measure, such as HPL used by Top 500; in the era of big data where data-intensive applications are widely emerging, Graph500, as an important supplement to T...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06F11/34G06F16/901
CPCG06F11/3409G06F11/3419
Inventor 甘新标曾瑞庚吴涛杨志辉孙泽文刘杰龚春叶李胜国杨博徐涵晏益慧
Owner NAT UNIV OF DEFENSE TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap