A memory access optimization method and system for graph processing based on activeness layout

Abstract

The invention discloses a graph processing memory access optimization method and system based on activity layout, belonging to the technical field of computer data processing, including: after obtaining the data of the target graph to be processed, sorting the vertices according to the descending order of betweenness centrality, and Divide the sorting results into active vertex sets and inactive vertex sets according to the active percentage; organize and store the active vertex sets; after organizing and storing the inactive vertex sets, divide them into multiple vertex blocks of size C , and the corresponding outbound edge is divided into multiple edge blocks, each edge block includes one or more outbound edges of consecutively numbered vertices, and each edge block is not less than C, and the state tables TActvB and TActeB are maintained at the same time to Filter out useless data in the inactive vertex set; TActvB is used to record the activity of each vertex block, and TActeB is used to record the activity of each edge block and the offset of the next edge block. The invention can reduce the memory access waiting time in the execution process of the graph application and improve the graph processing efficiency.

Description

technical field [0001] The invention belongs to the technical field of computer data processing, and more specifically relates to a method and system for optimizing memory access for graph processing based on activity layout. Background technique [0002] As an important data structure in computer science, graphs can clearly describe the relationship between various things, so the method of using graph models to analyze complex data is becoming more and more popular. Graphs are widely used in many fields of social life and help solve complex problems in these fields, such as the optimal route in the field of transportation, the decomposition of protein molecular structures in the field of biomedicine, community discovery in the field of social networks, and the current hot topic Knowledge graph, data mining, semantic network, link prediction, etc. With the advent of the era of big data, the scale of graph data generated in these fields is also growing rapidly. For example,...

AI Technical Summary

Problems solved by technology

GraphChi organizes the data reasonably according to the characteristics of the processor accessing data in order to take advantage of the locality of the active data. The optimized data will be more suitable for the memory access behavior of the processor. Due to the load pressure from the future, there is a lot of data that does not participate in the calculation during the graph processing. These data not only bring useless calculations, but also bring a lot of redundant I / O, which still affects the efficiency of graph processing.

In general, there is still a large room for optimization in memory access methods for graph processing

Images