Big data processing capacity testing method based on vertex reordering and priority caching

A technology for big data processing and capability testing, applied in special data processing applications, other database retrieval, other database indexes, etc., and can solve problems such as limited performance improvement, non-adjacent logic, and long memory access paths.

Inactive Publication Date: 2020-11-03
NAT UNIV OF DEFENSE TECH
View PDF3 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

This ordered global memory access method is more image 3 The unordered random global memory access shown has made great progress, and the performance has been significantly improved. However, this method is still a global me...

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Big data processing capacity testing method based on vertex reordering and priority caching
  • Big data processing capacity testing method based on vertex reordering and priority caching
  • Big data processing capacity testing method based on vertex reordering and priority caching

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0100] Such as Figure 5 Shown, the present invention comprises the following steps:

[0101] The first step, graph generation. Use the Kronecker graph generator to generate a random graph structure G=(V,E), V is a set of vertices, which contains N V vertices, N V is a positive integer; E is a set of edges; E contains N E side, N E is a positive integer; N V and N E All are positive integers.

[0102] The second step is to construct an adjacency matrix A that stores the graph G. A ij = 0 means there is no edge between vertex i and vertex j, A ij =1 means that there is an edge between vertex i and vertex j, both i and j are positive integers and 0≤i≤N V -1, 0≤j≤N V -1;

[0103] The third step is to sort the vertices of V based on the degree of vertices and cache the key vertices first. The specific method is as follows:

[0104] 3.1. Traverse each vertex in V, record the degree of each vertex, and get the vertex degree set D, the i-th element in D deg(v i ) repres...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a big data processing capacity testing method based on vertex reordering and priority caching, and aims to reduce the number of memory access times, minimize a communication path, avoid invalid memory access and improve the big data processing capacity testing speed of a super computer. The technical scheme is that the method comprises the following five steps: generating agraph G, constructing an adjacent matrix A for storing the graph G, sorting vertexes of the graph G, preferentially caching key vertexes, carrying out BFS search on the graph G by utilizing a sortedvertex set and the preferentially cached key vertexes, and calculating an evaluation value of graph test performance. By adopting the method, the hit rate of the edge relationship between the nodes can be improved, the invalid memory access frequency is reduced, unnecessary memory access is avoided to the greatest extent, the traversal of the graph is accelerated, the big data processing capability test speed of the super computer is improved, and the power consumption can be reduced.

Description

technical field [0001] The invention relates to a method for testing the large data processing capability of a supercomputer, in particular to a method for testing the large data processing capability based on vertex reordering and priority caching. Background technique [0002] Graph structure is one of the most important data structures in big data applications, and has been widely used in many fields, such as social media, bioinformatics, astrophysics, artificial intelligence, data mining, etc. The common feature of these applications is the large amount of data and complex structure, which can often reach trillions of vertices and edges, resulting in higher requirements for data storage and computing power. Supercomputers are mainly used for numerical calculations. Most HPC benchmarks use computing power as a measure, such as HPL used by Top 500; in the era of big data where data-intensive applications are widely emerging, Graph 500 is an important supplement to Top500. ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F16/901G06F16/903
CPCG06F16/9024G06F16/90348Y02D10/00
Inventor 甘新标龚春叶杨博陈旭光肖调杰谭雯
Owner NAT UNIV OF DEFENSE TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap