Social network stream data oriented maximal clique enumeration method based on MPICH parallel computing

Abstract

This invention discloses a maximal clique enumeration method based on MPICH parallel computation for social network streaming data. This method utilizes a structural inversion clique tree (ICTree) to store maximal cliques in the graph. Parallel programming of the algorithm is achieved through the high-performance and widely portable message passing interface MPICH. As the graph structure changes under the streaming data, multiple processes compute in parallel and update the inversion clique tree (ICTree) structure, enumerating maximal cliques in the graph and all maximal cliques in the subgraph formed by nodes within a certain index interval in real time. This invention does not distinguish between edge addition and subtraction changes in the graph during maximal clique enumeration, enabling maximal clique enumeration under mixed edge addition / subtraction updates and large-batch updates. Furthermore, the use of a predefined number of processes for parallel computation solves the real-time processing problem caused by large data volumes, improving the time efficiency of maximal clique enumeration.

Description

Technical Field

[0001] This invention relates to the fields of graph computing and parallel computing, and in particular to a maximal clique enumeration method based on MPICH parallel computing for social network streaming data. Background Technology

[0002] Maximal clique enumeration (MCE) aims to find the set of all maximal cliques in a graph, i.e., the set of one of the densest subgraphs. It has wide applications in social networks, academic networks, and other fields. However, with the rapid development of society and science and technology, people's ways of acquiring data are becoming increasingly diverse, and real-world data networks are also changing rapidly. Maximal clique enumeration needs to adapt to the ever-changing graph networks and incrementally update the set of maximal cliques in the graph in real time.

[0003] The existing dynamic maximal clique enumeration methods can either only handle adding / deleting one edge at a time [1], or only handle adding or deleting a group of edges at the same time [2], or use a pseudo-hybrid method to handle the added and deleted edges separately and then summarize them [3], or handle a group of added and deleted edges, but the computational performance is poor, and with the increase of data volume and the decentralization of computing power, the centralized computing of a single server is currently difficult to meet the needs of distributed parallel graph computing.

[0004] Therefore, how to make full use of computing resources to achieve parallelization of the calculation of the maximal clique difference for mixed addition and subtraction of edges, and further improve the efficiency of maximal clique enumeration, are technical issues that urgently need to be addressed. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies by providing a maximal clique enumeration method based on MPICH parallel computing for social network streaming data.

[0006] The objective of this invention is achieved through the following technical solution: a maximal clique enumeration method based on MPICH parallel computation for social network streaming data, comprising the following steps:

[0007] A maximal clique enumeration method based on MPICH parallel computation for social network streaming data includes the following steps:

[0008] (1) Use MPICH programming to initialize the number of available slave processes, design node distribution rules in the undirected graph of the social network, and initialize each data structure; the data structures include, but are not limited to, the reverse clique tree, the v-header candidate clique, and the large and small neighbor list, i.e., the neighbor list; the distribution rule is to distribute the nodes to the process whose node number is the remainder of the total number of processes plus 1.

[0009] (2) Since the initial graph data is empty, the main process starts to read the streaming data in real time, obtains a set of edges that need to be updated, and updates the neighbor list of the original graph. Based on the principle of backing up each v-header maximal clique in the v-header candidate clique corresponding to the head node that needs to be updated to a time point that is not affected by the updated edges, the back-up timestamps of all v-header maximal cliques of the central node are determined, and the reverse clique tree ICTree structure is updated to have no link correspondence with all v-header maximal cliques of the central node; the head node is the central node.

[0010] (3) The main process sends the neighbor list information related to the central node and the v-header maximum clique back-off timestamp of the central node to the slave process according to the predefined node distribution rules;

[0011] (4) After receiving the information, each slave process performs parallel calculations, updates the v-header maximal cliques that have changed in the v-header candidate cliques, and sends the updated v-header maximal cliques of the central node back to the master process.

[0012] (5) After receiving the updated v-header maximal clique information of the central node, the main process traverses the received v-header maximal cliques to update the ICTree and marks the corresponding relationships. Based on the updated ICTree structure, it enumerates all maximal cliques in the updated graph or any sequence interval.

[0013] (6) Read in data in real time, repeat steps (2) to (5), and update all maximal cliques in the enumeration dynamic graph or any sequence interval in real time.

[0014] Furthermore, step (1) is implemented by the following sub-steps:

[0015] (1.1) The rule for determining the order of graph nodes is as follows: The user nodes in the initial graph are sorted according to their order of appearance in the social network: {v1, v2, ..., v...} n};

[0016] (1.2) Using the MPICH programming method, initialize the number of available slave processes and design the distribution rules for nodes in the graph;

[0017] (1.3) Initialize each data structure so that all data are empty.

[0018] Furthermore, step (2) is implemented through the following sub-steps:

[0019] (2.1) The main process reads in the data, obtains a set of edges that need to be updated, that is, adds / deletes edges, adds or removes some nodes' large or small neighbor nodes, and updates the neighbor list;

[0020] (2.2) Based on the changes in the neighbor list, obtain relevant statistical information, determine the head node of the v-header candidate group that needs to be updated, and the corresponding time point that needs to be rolled back;

[0021] (2.3) Delete the parts of ICTree that are linked to all v-header maximal cliques of the central node, and update the ICTree structure.

[0022] Specifically, in step (2), the streaming data refers to the data that changes incrementally over time, the original image refers to the image that was obtained before reading data for a certain period of time, and the initial image data is empty because no data was obtained before the initial data reading.

[0023] Furthermore, step (4) is implemented through the following sub-steps:

[0024] (4.1) After receiving the central node and its corresponding statistical information, the process determines the update range of each v-header maximal clique that needs to be updated based on the generation timestamp of the v-header maximal clique of each central node. For each v-header maximal clique that needs to be updated, the part that needs to be updated in the v-header candidate clique is deleted.

[0025] (4.2) Each slave process traverses the major neighbor nodes of the central node and re-uses the CMC algorithm to update the v-header candidate group;

[0026] (4.3) Send the updated v-header max clique of the central node to the main process.

[0027] Furthermore, step (5) is implemented through the following sub-steps:

[0028] (5.1) After receiving the updated information of the v-header maximal clique of the central node, the main process updates the ICTree and marks the corresponding relationship by traversing the received v-header maximal clique, that is, establishes the corresponding index link.

[0029] (5.2) Based on the final updated ICTree structure, enumerate all maximal cliques in the graph or any index interval.

[0030] The beneficial effects of this invention are:

[0031] This invention proposes a real-time maximal clique enumeration method for social network stream data. By dynamically reading social network stream data and enumerating maximal cliques in real time based on edge changes (but not distinguishing between edge addition and deletion), it achieves efficient dynamic community mining. This invention also proposes a maximal clique enumeration method based on MPICH parallel computing. By simultaneously analyzing and calculating v-header candidate cliques corresponding to different nodes through multiple processes, the computational pressure on the Master node is reduced, while improving the time efficiency of maximal clique enumeration. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the implementation of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0033] Figure 1 This is a flowchart illustrating the overall process of the method of the present invention.

[0034] Figure 2 Here is an example diagram of time t1 and the updated diagram of the next time step, as described in this invention; (a) is the diagram obtained at time t1; (b) is the updated diagram of the next time step.

[0035] Figure 3 The following diagram illustrates the data initialization process for the time-t1 graph as described in this invention: (a) is a list of large and small neighbor nodes of the user node; (b) is a graph of all v-header maximal cliques obtained in each slave process; and (c) is the final result obtained in the main process.

[0036] Figure 4 The diagrams illustrate the process of reading new data into the t1 graph and then performing parallel computation to update the enumerated maximal clique in real time. (a) is the result diagram after the main process ICTree is rolled back and updated. (b) is the range diagram that needs to be reconstructed, which is enclosed by a rectangle from the process. (c) is the ICTree graph updated by the main process. Detailed Implementation

[0037] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are for illustrative purposes only and are not intended to limit the scope of the invention.

[0038] like Figure 1 As shown, a maximal clique enumeration method based on MPICH parallel computation for social network streaming data includes the following steps:

[0039] (1) Use MPICH programming to initialize the number of available slave processes, design node distribution rules in the undirected graph of the social network, and initialize each data structure; the data structures include, but are not limited to, the reverse clique tree, the v-header candidate clique, and the large and small neighbor list, i.e., the neighbor list; the distribution rule is to distribute the nodes to the process whose node number is the remainder of the total number of processes plus 1.

[0040] (2) Since the initial graph data is empty, the main process starts to read the streaming data in real time, obtains a set of edges that need to be updated, and updates the neighbor list of the original graph. Based on the principle of backing up each v-header maximal clique in the v-header candidate clique corresponding to the head node that needs to be updated to a time point that is not affected by the updated edges, the back-up timestamps of all v-header maximal cliques of the central node are determined, and the reverse clique tree ICTree structure is updated to have no link correspondence with all v-header maximal cliques of the central node; the head node is the central node.

[0041] (3) The main process sends the neighbor list information related to the central node and the v-header maximum clique back-off timestamp of the central node to the slave process according to the predefined node distribution rules;

[0042] (4) After receiving the information, each slave process performs parallel calculations, updates the v-header maximal cliques that have changed in the v-header candidate cliques, and sends the updated v-header maximal cliques of the central node back to the master process.

[0043] (5) After receiving the updated v-header maximal clique information of the central node, the main process traverses the received v-header maximal cliques to update the ICTree and marks the corresponding relationships. Based on the updated ICTree structure, it enumerates all maximal cliques in the updated graph or any sequence interval.

[0044] (6) Read in data in real time, repeat steps (2) to (5), and update all maximal cliques in the enumeration dynamic graph or any sequence interval in real time.

[0045] In the data initialization phase for the initial graph, the first step is to define the node numbering rule in the graph as follows: user nodes are sorted according to the order in which they appear in the social network, such as... Figure 2 (a) is the graph obtained at time t1. After sorting, the graph contains nodes {v1,v2,v3,v4,v5,v6,v7}.

[0046] Using the MPICH programming method, initialize the total number of slave processes. In this example, the number of slave processes is set to 2. Design the node distribution rules in the graph. In this example, the node distribution rule is set as follows: distribute nodes to the process whose node index is the remainder of the total number of processes (2) plus 1. For example, node v2 is assigned to slave process 1 (2%2+1=1). Initialize the data structures in the master process and slave processes, where all data is empty.

[0047] The data processing and ICTree fallback update node updates the neighbor list based on a received set of mixed (added / deleted) edges H; wherein, Indicates at node and When adding / removing an edge between nodes, the default value is that the index of the left node is less than the index of the right node. x1, y1, x t y t The node number is represented by t, and the number of edges updated is t. Edges represent interactions between users, including forwarding, liking, following, unfollowing, etc.

[0048] For ease of explanation, let's assume the method has already read data once ( Figure 2 (a) Then, through steps two to five, the data of the main process and the slave process are calculated and updated based on the read data. The update results are as follows: Figure 3 As shown, where Figure 3 (a) is a list of the major and minor neighbor nodes of the user node. Figure 3 (b) represents all v-header maximal cliques obtained from each slave process. Figure 3 (c) The final result obtained in the main process, with all head nodes on the left and the corresponding ICTree on the right. Each head node has one or more v-header maximal cliques that are linked to the ICTree, and the corresponding child nodes are marked in light color on the ICTree.

[0049] After receiving a set of streaming data, the generated graph is as follows: Figure 2 As shown in (b), Figure 2 (a) add edge e + (v3,v5), the updated neighbor list is as follows Figure 4 As shown in (a), node v3's major neighbor increases by v5, and node v5's minor neighbor increases by v3, i.e. in and These represent the list of major neighbors of node v3 and the list of minor neighbors of node v5, respectively. and These represent changes in the large neighbor list for v3 and changes in the small neighbor list for v5, respectively, without distinguishing between increases and decreases. Based on the changes in the neighbor list, determine the candidate clusters that need to be updated. Let... express If we consider the node with the largest index among the changing nodes in the list of smaller neighbors, where k = 1, 2, ..., t, then the list of head nodes (center nodes) of the candidate clique that needs to be updated is:

[0050]

[0051] For each candidate clique index corresponding to a head node in S, determine the time point at which it needs to be backtracked, i.e. make Then v i -The timestamp for the large header group that needs to be rolled back is T. j y j Represents a node The serial number. Among them, This means that the following conditions are met: ① It is v i Large neighbor node; ②v i Less than The node with the largest index among the changing nodes in the list of smaller neighbor nodes. k The smallest node in the set of these two conditions. For example... Figure 4 (a) As shown in the above figure, the list of central nodes is as follows: Furthermore, both the v2-header maximal clique and the v3-header maximal clique require a rollback timestamp of T5.

[0052] The process involves deleting nodes on the ICTree that are linked to the central node. The deletion process is as follows: First, locate the corresponding linked nodes on the ICTree through the central node. Starting from that node, delete each node sequentially, taking its parent node as the current node. This process continues until the current node has other child nodes or is marked, at which point the loop stops. For example... Figure 4 (a) The figure below shows the result after the ICTRee rollback update.

[0053] During the data distribution phase, according to a predefined node distribution mechanism, the central node and its corresponding timestamp requiring rollback, the list of major neighbor nodes after the rollback timestamp of the central node, and the list of minor neighbor nodes corresponding to the major neighbor nodes are distributed to each slave process. Figure 4As shown, the central node {v2}, the corresponding timestamp that needs to be rolled back {v2:T5}, the list of major neighbor nodes after the rollback timestamp of the central node {v2:{v5,v6,v7}}, and the list of minor neighbor nodes corresponding to the major neighbor nodes {v5:{v2,v3,v4},v6:{v2,v3,v4,v5},v7:{v2,v6}} are sent to process 1.

[0054] During the parallel computation phase of each process, after each slave process receives information, it determines the specific reconstruction range of each v-header maximal clique based on the generation timestamp of the v-header maximal clique of each central node and the time point that needs to be rolled back. At this point, two cases need to be considered: ① If a certain v... i -header max clique generation timestamp at time T j Previously, the scope of its reconstruction was all maximal cliques not less than v. j ② If a certain v i -header max clique generation timestamp at time T j Or, subsequently, the scope of its reconstruction is the entire set of that maximal clique. For example... Figure 4 As shown in (b), the area enclosed by the rectangle represents the region that needs to be reconstructed. Based on the reconstruction region of each v-header maximal clique that needs to be updated, all parts of the v-header maximal clique that need to be reconstructed are deleted.

[0055] Update the v-header maximal clique of the central node, that is, update the v-header maximal clique of each slave process. Assume its rollback timestamp is T. j Then add nodes by traversing. In this way, the CMC algorithm is reused to evaluate v in the candidate group. i The v-header maximal clique is updated. The updated v-header maximal clique of the central node is then sent to the main process.

[0056] During the information collection and maximal clique enumeration phase, after the main process receives the v-header maximal clique data from the central node sent by each process, it iterates through the received v-header maximal cliques, updates the ICTree using the method for constructing the ICTree, and marks the corresponding link relationship between the head node and the nodes in the ICTree, such as... Figure 4 (c) shows the updated ICTree. Based on the updated ICTree structure, all maximal cliques in the updated graph (or any index interval) are enumerated.

[0057] The continuous reading and processing of streaming data stage involves repeatedly performing steps two through five, including data processing, ICTree rollback updates, data distribution, parallel computation of each process, information collection, and maximal clique enumeration. This achieves the goal of real-time dynamic maximal clique enumeration for streaming data, further enabling real-time and efficient dynamic community mining.

Claims

1. A maximal clique enumeration method based on MPICH parallel computation for social network streaming data, characterized in that, Includes the following steps: (1) Initialize the number of available slave processes using MPICH programming, design the node distribution rules in the undirected graph of the social network, and initialize each data structure; the data structures include, but are not limited to, the inversion clique tree, the v-header candidate clique, and the large and small neighbor list, i.e., the neighbor list; the distribution rule is to distribute the nodes to the process whose node index is the remainder of the total number of processes plus 1; specifically implemented by the following sub-steps: (1.1) The rule for determining the order of graph nodes is: sort the user nodes in the initial graph according to the order in which they appear in the social network; (1.2) Using the MPICH programming method, initialize the number of available slave processes and design the distribution rules for nodes in the graph; (1.3) Initialize all data structures so that all data is empty; (2) Since the initial graph data is empty, the main process starts to read the streaming data in real time, obtains a set of edges that need to be updated, and updates the neighbor list of the original graph. Based on the principle of backing up each -header maximal clique in the -header candidate clique corresponding to the head node that needs to be updated to a time point that is not affected by the updated edges, the back-up timestamps of all -header maximal cliques of the central node are determined, and the reverse clique tree ICTree structure is updated to have no link correspondence with all v-header maximal cliques of the central node; the head node is the central node; specifically implemented through the following sub-steps: (2.1) The main process reads in the data, obtains a set of edges that need to be updated, that is, adds / deletes edges, adds or removes some nodes' large or small neighbor nodes, and updates the neighbor list; (2.2) Based on the changes in the neighbor list, obtain relevant statistical information, determine the head node of the candidate group of -header that needs to be updated, and the corresponding time point that needs to be rolled back; (2.3) Delete the parts of the ICTRee that are linked to all v-header maximal cliques of the central node, and update the ICTRee structure; (3) The main process sends the neighbor list information related to the central node and the v-header maximum clique backoff timestamp of the central node to the slave process according to the predefined node distribution rules; (4) After receiving the information, each slave process performs parallel computation, updates the -header maximal cliques that have changed in the -header candidate cliques, and sends the updated v-header maximal cliques of the central node back to the master process. (5) After receiving the updated information of the v-header maximal clique of the central node, the main process traverses the received -header maximal cliques to update the ICTree and marks the corresponding relationship. Based on the updated ICTree structure, it enumerates all maximal cliques in the updated graph or any sequence interval. (6) Read in data in real time, repeat steps (2) to (5), and update all maximal cliques in the enumeration dynamic graph or any sequence interval in real time.

2. The maximal clique enumeration method based on MPICH parallel computation for social network streaming data as described in claim 1, characterized in that, In step (2), the streaming data refers to the data that changes incrementally over time, the original image refers to the image that was obtained before reading data for a certain period of time, and the initial image data is empty because no data was obtained before the initial data reading.

3. The maximal clique enumeration method based on MPICH parallel computation for social network streaming data according to claim 1, characterized in that, Step (4) is achieved through the following sub-steps: (4.1) After receiving the central node and its corresponding statistical information, the process determines the update range of each -header maximal clique that needs to be updated based on the generation timestamp of the -header maximal clique of each central node. For each -header maximal clique that needs to be updated, the part that needs to be updated in the -header candidate clique is deleted. (4.2) Each slave process traverses the major neighbor nodes of the central node and re-uses the CMC algorithm to update the -header candidate group; (4.3) Send the updated v-header max clique of the central node to the main process.

4. The maximal clique enumeration method based on MPICH parallel computation for social network streaming data according to claim 1, characterized in that, Step (5) is achieved through the following sub-steps: (5.1) After receiving the updated v-header maximal clique information of the central node, the main process updates the ICTree and marks the corresponding relationship by traversing the received v-header maximal clique, that is, establishes the corresponding index link. (5.2) Based on the final updated ICTree structure, enumerate all maximal cliques in the graph or any index interval.

Images