Message passing method and device in distributed graph calculation process

A technology of message passing and graph computing, which is applied in the directions of multi-program device, computing, and inter-program communication, and can solve the problems of large storage cost and huge communication cost, etc.

Active Publication Date: 2020-05-19
四川蜀天梦图数据科技有限公司 +1
View PDF10 Cites 8 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Therefore, these cross-node message transfers will generate huge communication costs
At the same time, each

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Message passing method and device in distributed graph calculation process
  • Message passing method and device in distributed graph calculation process
  • Message passing method and device in distributed graph calculation process

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0061] Embodiment 1 of the present invention provides a message delivery method in the process of distributed graph calculation. The vertices of the graph are distributed and stored on different computer nodes. It is characterized in that the identifier Nj of the sending destination node and the identifier number of the source vertex Vi, combined as the primary key (Nj, Vi) of the message record; wherein, the message record generated by the corresponding source vertex is: [(Nj, Vi), Mi]; where j is the number identification number of the corresponding destination node, i is The numbered identification number of the corresponding source vertex, such as image 3 and Figure 4 As shown, delivery methods include:

[0062] In step 201, when the source vertex generates a message M1, the source vertex generates a message group to be sent according to the graph relationship where the source vertex is located; wherein, the message group to be sent consists of one or more [(Nj, V1), M1...

Embodiment 2

[0086] An embodiment of the present invention provides a graph data access method based on a group association table, which is used to support the message passing method in the distributed graph computing process described in Embodiment 1, such as Image 6 , including the following steps:

[0087] Step 301, using the attribute table to store the attribute data of the graph, and using the group association table to store the topology data of the graph; wherein, the topology data includes adjacent vertices and associated edge information of each vertex in the graph.

[0088] Graph data can be divided into attribute data and topological data. The attribute data and the topological data are stored separately using different data storage structures. Among them, the attribute data of points and edges in the graph are stored in the attribute table, which can be stored through the Key - Value can be stored in the form of a link or linked list. The topology data of the graph is stored...

Embodiment 3

[0118] like Figure 11 As shown in FIG. 2 , it is a schematic structural diagram of a message passing device in a distributed graph computing process according to an embodiment of the present invention. The message passing device in the distributed graph calculation process of this embodiment includes one or more processors 21 and memory 22 . in, Figure 11 A processor 21 is taken as an example.

[0119] Processor 21 and memory 22 can be connected by bus or other means, Figure 11 Take connection via bus as an example.

[0120] The memory 22, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs and non-volatile computer-executable programs, such as the message passing method in the distributed graph computing process in Embodiment 1 . The processor 21 executes the message passing method in the distributed graph computing process by running the non-volatile software programs and instructions stored in the memory 22 .

[0...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to the technical field of distributed graph calculation, and provides a message transmission method and device in a distributed graph calculation process. The method comprises the steps that when a source vertex generates a message M1, the source vertex generates a to-be-sent message group according to a graph relationship where the corresponding source vertex is located; thesource node sends the to-be-sent message to a corresponding destination node according to the destination node identifier in each to-be-sent message group; after receiving the message sent by the source node, each destination node caches the message according to a message format; and each destination node traverses one or more vertexes stored in the destination node, determines the destination vertex belonging to the same graph relationship with the source vertex V1 in the message, and adjusts the corresponding destination vertex according to the content of the corresponding message M1. According to the invention, the number of messages which need to be transmitted across nodes and stored is irrelevant to the number of destination vertexes and is only positively relevant to the number ofdestination nodes, so that the communication cost of message transmission and the storage cost of message caching are greatly reduced.

Description

【Technical field】 [0001] The present invention relates to the technical field of distributed graph computing, in particular to a method and device for message transmission in the process of distributed graph computing. 【Background technique】 [0002] Large-scale parallel graph computing is usually implemented based on the BSP (Bulk Synchronous Parallel) model. The BSP graph computing model is implemented with vertices as the center and parallel computing based on message passing. A BSP graph calculation consists of a series of iterative processes, and each iteration includes the following three sub-steps: [0003] 1. Local calculation; [0004] 2. Message exchange between adjacent vertices; [0005] 3. Waiting synchronously. After all vertices complete an iteration, they enter the next iteration together. [0006] BSP graph calculation iterative process, such as figure 1 shown. [0007] In a distributed environment, the vertices of the graph are distributed and stored...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F16/903G06F9/54G06F16/901
CPCG06F9/546G06F16/9024G06F16/903
Inventor 李海波吕继云李专李鹏
Owner 四川蜀天梦图数据科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap