Super computer big data processing capability testing method based on vertex reordering

A technology of supercomputer and big data processing, which is applied in the fields of electrical digital data processing, computing, and other database retrieval, etc.

Active Publication Date: 2019-04-19
NAT UNIV OF DEFENSE TECH
View PDF2 Cites 5 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0010] The technical problem to be solved by the present invention is: the preprocessing before the BFS traversal is not included in the time-consuming and the high probability of edge relationship between the vertex with high degree and the root node is high, and a supercomputer based on vert...

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Super computer big data processing capability testing method based on vertex reordering
  • Super computer big data processing capability testing method based on vertex reordering
  • Super computer big data processing capability testing method based on vertex reordering

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0073] image 3 For the overall flow chart of the present invention, the steps of the present invention are as follows:

[0074] The first step, graph generation. Use the Kronecker graph generator to generate a random graph structure G=(V,E), V is a set of vertices, which contains N V vertices, N V is a positive integer; E is an edge set;

[0075] The second step is to construct an adjacency matrix A that stores the graph G. A ij = 0 means there is no edge between vertex i and vertex j, A ij =1 means there is an edge between vertex i and vertex j, 0≤i≤N V -1, 0≤j≤N V -1, i and j are both positive integers;

[0076] The third step is to preprocess V based on the degree of the vertex. The specific method is as follows:

[0077] 3.1. Traverse each vertex in V and record the degree of each vertex to obtain the set D of vertex degrees.

[0078] The i-th element in D deg(v i ) represents the vertex v i degrees, that is, deg(v i ) vertices and vertex v i There is an edg...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a super computer big data processing capability test method based on vertex reordering, and aims to improve the super computer big data processing capability test speed. technical scheme comprises the following steps of generating an adjoining matrix for a map and a construction map, sorting the vertexes in the graph based on the vertex degrees; carrying out BFS search on the graph by utilizing the sequenced vertex set, wherein the characteristic that the probability of edge association of vertexes with high degrees is also high during the same layer of traversal is utilized, transversing and searching child nodes of nodes in the current layer of vertex set, and transversing and checking the vertexes with high degrees preferentially, so that invalid memory access traversal is reduced to the maximum extent. By adopting the method, the hit rate of the edge relation between the nodes can be improved, invalid memory access times are reduced, unnecessary memory access is avoided to the maximum extent, graph traversal is accelerated, and the testing speed of the big data processing capacity of the super computer is increased.

Description

technical field [0001] The invention relates to a method for testing the large data processing capability of a supercomputer, in particular to a method for testing the large data processing capability of a supercomputer based on vertex reordering. Background technique [0002] Graph structure is one of the most important data structures in big data applications, and has been widely used in many fields, such as social media, bioinformatics, astrophysics, artificial intelligence, data mining, etc. The common feature of these applications is the large amount of data and complex structure, which can often reach billions of edges and trillions of nodes, which leads to higher demands on data storage and computing power. Supercomputers are mainly used for numerical calculations, and most HPC benchmarks use computing power as a measure, such as HPL used by Top 500; in the era of big data where data-intensive applications are widely emerging, Graph500, as an important supplement to T...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F11/34G06F16/901
CPCG06F11/3409G06F11/3419
Inventor 甘新标曾瑞庚吴涛杨志辉孙泽文刘杰龚春叶李胜国杨博徐涵晏益慧
Owner NAT UNIV OF DEFENSE TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap