Heterogeneous server structure-oriented graph calculation load balancing method

Abstract

The invention discloses a heterogeneous server structure-oriented graph calculation load balancing method. The method includes: calculating a variation coefficient of all processing times in a currentsuper step by a main control node according to the times used by processing graph calculation loads by all calculation nodes in the super step; if variation coefficients in two consecutive super steps are greater than a preset threshold value, determining that the graph calculation loads of all the calculation nodes in running are not balanced, and the graph calculation loads on the calculation nodes need to be re-assigned after the current super step ends; and otherwise, continuing to execute a next super step by all the calculation nodes. According to the method of the invention, vertex migration is utilized to balance processing times of the calculation nodes in each super step, thus synchronization overheads of all the calculation nodes in each super step are effectively reduced, a running time of a graph calculation task is shortened, and a resource utilization rate of each calculation node in running is increased.

Description

technical field [0001] The invention belongs to the technical field of graph computing, and more specifically relates to a graph computing load balancing method oriented to a heterogeneous server structure. Background technique [0002] Distributed graph computing technology is widely used in many fields such as social network analysis, natural language processing, and web page information retrieval. Its composition system usually includes three parts: graph algorithm, graph data set, and graph computing system. Most graph computing systems use the BSP (Bulk Synchronization Barrier) computing model. Under this model, the graph computing process is usually divided into several iterations, and each iteration is also called a super step. In each super step, the active vertex updates the vertex value according to the message received in the last super step, and sends the updated value to the corresponding neighbor vertex in the form of a message. A synchronization barrier is se...

AI Technical Summary

Problems solved by technology

[0005] In view of the above defects or improvement needs of the prior art, the present invention provides a graph computing load balancing method oriented to heterogeneous server structures, which is used to balance the processing time of all computing nodes in each superstep in a distributed graph computing system, Thereby reducing the synchronization overhead of computing nodes in each superstep, shortening the running time of graph computing tasks, and solving the unbalanced load of computing nodes and asynchronous processing time when processing graph computing tasks under the existing heterogeneous server structure question

Images