Distributed graph analysis method and system

Abstract

The embodiment of the invention provides a distributed graph analysis method and system. The method comprises the following steps: a center node performs community detection on a target relation network diagram, and divides the target relation network diagram into a plurality of first sub-diagrams; each first sub-graph corresponds to one community; the center node distributes the plurality of first sub-graphs to a plurality of working nodes; the plurality of working nodes execute a target operation in parallel, and the target operation comprises calculating a local minimum spanning tree of a single first sub-graph so as to obtain a local processing result; wherein the local minimum spanning tree corresponding to each first sub-graph forms a global minimum spanning tree of the target relation network graph; and the central node summarizes local processing results of the plurality of working nodes and executes global graph processing. The method can adapt to a graph analysis scene and has high efficiency.

Description

Technical Field

[0001] One or more embodiments of this specification relate to the field of computers, and more particularly, to a distributed graph analysis method and system. Background Art

[0002] With the rapid growth of data in fields such as social networks, internet data, the Internet of Things, and bioinformatics, there is a growing need to process extremely large-scale graph data with over 10 billion nodes. These graphs are massive and complex in structure, making traditional graph analysis solutions difficult to run efficiently at this scale, with significantly reduced iteration performance. This graph data may also contain private user data.

[0003] Existing technologies offer more efficient algorithms for general scenarios, such as distributed computing, that optimize computing resources and improve decision-making efficiency. By decomposing and distributing computing tasks across multiple nodes, algorithm scalability and processing efficiency can be significantly improved. However, in graph analysis scenarios, simple distributed strategies are ineffective in improving efficiency. Therefore, a more efficient distributed graph analysis solution is needed. Summary of the Invention

[0004] One or more embodiments of this specification describe a distributed graph analysis method and system that can be adapted to graph analysis scenarios and has high efficiency.

[0005] In a first aspect, a distributed graph analysis method is provided, the method comprising:

[0006] The central node performs community detection on the target relationship network graph, and divides the target relationship network graph into a plurality of first subgraphs; each first subgraph corresponds to a community;

[0007] The central node distributes the multiple first subgraphs to multiple working nodes;

[0008] The multiple working nodes execute the target operation in parallel, the target operation including calculating a local minimum spanning tree of a single first subgraph, thereby obtaining a local processing result; wherein the local minimum spanning trees corresponding to each first subgraph form a global minimum spanning tree of the target relationship network graph;

[0009] The central node aggregates the local processing results of the multiple working nodes and performs global graph processing.

[0010] In a possible implementation, the central node performs community detection on the target relationship network graph, including:

[0011] Obtaining an initial community to which any node in the target relationship network graph belongs;

[0012] Determine whether to merge adjacent communities based on whether the connection density between nodes in each community is improved after the community is merged;

[0013] When the community to which each node belongs no longer changes, the connecting edges between nodes in the same community are taken as a first subgraph.

[0014] In a possible implementation, calculating the local minimum spanning tree of the single first subgraph includes:

[0015] According to the n nodes included in the single first subgraph, construct a construction graph consisting of the n nodes and without any connecting edges;

[0016] Continuously extract the connecting edge with the smallest weight from the connecting edges included in the first subgraph. If the two nodes of the connecting edge come from different connected components, add this edge to the construction graph until all nodes are on the same connected component, and use the construction graph as the local minimum spanning tree corresponding to the first subgraph.

[0017] In a possible implementation, calculating the local minimum spanning tree of the single first subgraph includes:

[0018] Selecting a node from each node included in the single first subgraph as the starting point of the local minimum spanning tree corresponding to the first subgraph;

[0019] The selected nodes are added to the first set, and the other nodes are added to the second set;

[0020] Iteratively executing the process of selecting a connecting edge with the smallest weight from edges directly connecting nodes in the first set and nodes in the second set, and adding the selected connecting edge and the nodes in the second set connected thereto into the local minimum spanning tree.

[0021] In a possible implementation, the target operation includes:

[0022] Performing subgraph extraction processing on the local minimum spanning tree to obtain a second subgraph as a local processing result;

[0023] The global graph processing includes:

[0024] Aggregate the multiple second subgraphs, and perform graph analysis based on the aggregation results.

[0025] In a possible implementation, the local processing result is the local minimum spanning tree, and the global graph processing includes:

[0026] Merging the local minimum spanning trees to obtain a global minimum spanning tree of the target relationship network graph;

[0027] A subgraph extraction process is performed on the global minimum spanning tree to obtain a subgraph extraction result.

[0028] Furthermore, the subgraph extraction process includes:

[0029] Clustering is performed on each node of the minimum spanning tree to be analyzed to obtain several clusters;

[0030] Nodes and connecting edges of the same cluster are extracted from the minimum spanning tree to be analyzed as subgraph extraction results.

[0031] Furthermore, the subgraph extraction process includes:

[0032] For the minimum spanning tree to be analyzed, the graph structure of the preset structural pattern is found through subgraph matching;

[0033] The found graph structure is used as the subgraph extraction result.

[0034] In a possible implementation, the target relationship network graph includes multiple nodes representing users, and there is a connection edge between two nodes with a preset association relationship; the global graph processing is used to detect user connection structures with preset risk types.

[0035] In a second aspect, a distributed graph analysis system is provided, the system comprising: a central node and multiple working nodes;

[0036] The central node is used to perform community detection on the target relationship network graph, divide the target relationship network graph into multiple first subgraphs; each first subgraph corresponds to a community; and distribute the multiple first subgraphs to multiple working nodes;

[0037] The multiple working nodes are used to perform a target operation in parallel, wherein the target operation includes calculating a local minimum spanning tree of a single first subgraph to obtain a local processing result; wherein the local minimum spanning trees corresponding to the respective first subgraphs form a global minimum spanning tree of the target relationship network graph;

[0038] The central node is also used to aggregate the local processing results of the multiple working nodes and perform global graph processing.

[0039] In a third aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed in a computer, the computer is caused to execute the method of the first aspect.

[0040] In a fourth aspect, a computing device is provided, comprising a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method of the first aspect is implemented.

[0041] Through the method and device provided in the embodiment of this specification, the central node first performs community detection on the target relationship network graph, and divides the target relationship network graph into multiple first subgraphs; each first subgraph corresponds to a community; then the central node distributes the multiple first subgraphs to multiple working nodes; then the multiple working nodes perform the target operation in parallel, and the target operation includes calculating the local minimum spanning tree of a single first subgraph, thereby obtaining a local processing result; wherein the local minimum spanning trees corresponding to each first subgraph form the global minimum spanning tree of the target relationship network graph; finally, the central node summarizes the local processing results of the multiple working nodes and performs global graph processing. As can be seen from the above, in the embodiment of this specification, the central node divides the target relationship network graph into multiple subgraphs, each community corresponds to a subgraph, and since the subgraph corresponds to the community, the nodes and edges have a closer connection in the local area. On this basis, multiple working nodes respectively calculate the minimum spanning tree corresponding to the subgraph. This process can not only effectively prune and delete unnecessary edges, but also significantly reduce the size of the subgraph, thereby improving the subsequent local processing results or the iterative performance of the graph-related algorithm in the global graph processing. In summary, this distributed processing method fully utilizes computing resources, is adaptable to graph analysis scenarios, and is highly efficient. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0043] Figure 1 A schematic diagram of an implementation scenario of an embodiment disclosed in this specification;

[0044] Figure 2 A flow chart of a distributed graph analysis method according to one embodiment is shown;

[0045] Figure 3 A schematic diagram of a stand-alone MST process according to one embodiment is shown;

[0046] Figure 4 FIG. 1 shows a schematic diagram of distributed MST processing according to one embodiment;

[0047] Figure 5 A schematic block diagram of a distributed graph analysis system according to one embodiment is shown. DETAILED DESCRIPTION

[0048] The solution provided in this specification is described below in conjunction with the accompanying drawings.

[0049] Figure 1 This is a schematic diagram of an implementation scenario of an embodiment disclosed in this specification. This implementation scenario involves distributed graph analysis. It is understandable that distributed computing decomposes and distributes computing tasks to multiple nodes, which can significantly improve the scalability and processing efficiency of the algorithm. The computing tasks of distributed graph analysis are specifically graph analysis tasks, which often involve multiple rounds of iterative processing. For example, common graph analysis tasks include subgraph analysis, etc. The application of subgraph analysis algorithms in financial risk control is mainly reflected in the identification and evaluation of complex relationships related to financial transactions, customer relationships, social networks, etc. Subgraph analysis is a graph theory method that can extract specific substructures from large-scale graph data, thereby revealing potential risk factors. For example, subgraph analysis can be used for fraud detection. In financial transactions, fraudulent behavior often manifests itself as a complex transaction network. By constructing a transaction graph, representing nodes such as customers, accounts, and transactions as vertices in the graph, and transaction relationships as edges, subgraph analysis can help identify suspicious small subgraph structures. For example, small subgraph structures correspond to aggregated behavior, where multiple accounts transfer funds to the same account within a short period of time. This pattern may indicate fraudulent activity. Alternatively, small subgraph structures correspond to abnormal connections, where the connection patterns between some nodes are abnormal, such as some accounts frequently conducting high-value transactions with new accounts, which may represent collusion in fraud.

[0050] like Figure 1 As shown, the scenario of distributed graph analysis involves multiple participants, including a central node and multiple worker nodes. Each participant can be implemented as any device, platform, server or device cluster with computing and processing capabilities. The central node divides the large-scale graph data into multiple subgraphs, and assigns the multiple subgraphs to multiple worker nodes. Each worker node independently and in parallel performs graph analysis processing on the subgraphs it obtains, and obtains local processing results for each subgraph. Subsequently, the central node can aggregate the local processing results of multiple worker nodes and perform global graph processing, which helps improve efficiency. For example, the central node divides the large-scale graph data into N+1 subgraphs, respectively denoted as subgraph 0, ..., subgraph N, and assigns subgraph 0 to worker node 0, ..., and subgraph N to worker node N. Worker node 0 independently performs graph analysis processing on subgraph 0 it obtains, and obtains local processing result 0 for subgraph 0, ..., and work node N independently performs graph analysis processing on subgraph N it obtains, and obtains local processing result N for subgraph N.

[0051] In this embodiment, a central node partitions the input graph based on communities, generating subgraphs corresponding to these communities. Worker nodes then generate minimum spanning trees for each subgraph, pruning the subgraph, removing unnecessary edges, and significantly reducing its size. Community-based minimum spanning trees help reduce unimportant nodes, accelerating the iteration performance of graph-related algorithms. Parallel processing is achieved through a distributed computing platform comprised of multiple worker nodes, further improving the efficiency of large-scale graph processing.

[0052] The following is a brief explanation of community partitioning and minimum spanning tree.

[0053] Community detection is an important concept in network analysis. It is mainly used to identify subsets with high internal connections in social networks or other types of graphs (such as biological networks and information networks). These subsets are called communities. The connection density between nodes in each community is high, while the connection density with nodes in other communities is low. Community partitioning can help understand the structure and function of the network and discover certain important patterns or characteristics in the network. Common community partitioning algorithms include methods based on modularity optimization, spectral clustering, label propagation algorithms, etc. Community partitioning has a wide range of applications in social network analysis, recommendation systems, information dissemination and other fields.

[0054] The minimum spanning tree (MST) is a key concept in graph theory, specifically for connected undirected graphs. Given a weighted graph, a minimum spanning tree is a subgraph containing all vertices in the graph with the following properties: This subgraph is a spanning tree, meaning it has no cycles and all vertices are connected. The sum of all edge weights is minimized. Minimum spanning trees can be used to solve many practical problems, such as network design, circuit layout, and cluster analysis. Common algorithms for computing minimum spanning trees include Prim's algorithm and Kruskal's algorithm.

[0055] The embodiments of this specification achieve efficient local computing by dividing a large graph network into multiple relatively independent subgraphs corresponding to communities. A preliminary minimum spanning tree is established within each community through parallel processing, and finally these local results are integrated into a global minimum spanning tree through an efficient aggregation mechanism. This community-based distributed algorithm not only effectively reduces the burden of single-point computing, but also improves processing speed and scalability. With the rapid development of modern computing resources, especially the rise of cloud computing and edge computing, this algorithm can be widely applied to fields such as social network analysis, intelligent traffic management, and large-scale sensor networks.

[0056] Figure 2A flow chart of a distributed graph analysis method according to an embodiment is shown. The method can be based on Figure 1 The implementation scenario shown in Figure 1 is jointly executed by a central node and multiple working nodes. Figure 2 As shown, the distributed graph analysis method in this embodiment includes the following steps: Step 21, the central node performs community detection on the target relationship network graph, and divides the target relationship network graph into multiple first subgraphs; each first subgraph corresponds to a community; Step 22, the central node distributes the multiple first subgraphs to multiple working nodes; Step 23, multiple working nodes perform target operations in parallel, and the target operations include calculating the local minimum spanning tree of a single first subgraph, thereby obtaining a local processing result; wherein the local minimum spanning trees corresponding to each first subgraph form the global minimum spanning tree of the target relationship network graph; Step 24, the central node summarizes the local processing results of the multiple working nodes and performs global graph processing. The specific execution method of each of the above steps is described below.

[0057] First, in step 21, the central node performs community detection on the target relationship network graph, dividing it into multiple first subgraphs; each first subgraph corresponds to a community. It can be understood that a community is a subset of the target relationship network graph that has a high degree of internal connectivity. Community detection, also known as community partitioning, has no single method; any currently available community partitioning method can be used.

[0058] In one example, the central node performs community detection on the target relationship network graph, including:

[0059] Obtaining an initial community to which any node in the target relationship network graph belongs;

[0060] Determine whether to merge adjacent communities based on whether the connection density between nodes in each community is improved after the community is merged;

[0061] When the community to which each node belongs no longer changes, the connecting edges between nodes in the same community are taken as a first subgraph.

[0062] In this example, the decision to merge adjacent communities can be repeatedly performed, that is, multiple rounds of iterations can be performed until the community to which each node belongs no longer changes, at which point the multiple rounds of iterations are terminated. The target network diagram can be initialized so that each node corresponds to one community. In other words, the number of nodes in the target network diagram is the same as the number of initial communities.

[0063] In the embodiments of this specification, the nodes and edges included in the target relationship network diagram have a broad meaning. Typically, the target relationship network diagram includes multiple nodes representing users, and an edge connects two nodes with a preset relationship. The preset relationship may include, but is not limited to, transaction relationships.

[0064] Then, in step 22, the central node distributes the multiple first subgraphs to multiple worker nodes. It is understood that the number of first subgraphs can be the same as or different from the number of worker nodes. For example, N first subgraphs can be distributed to N worker nodes, with each worker node receiving one first subgraph; or 2N first subgraphs can be distributed to N worker nodes, with each worker node receiving two first subgraphs.

[0065] In the embodiments of this specification, the subgraphs can be distributed in various ways. They can be distributed evenly, with each worker node receiving the same number of first subgraphs; or they can be distributed based on the processing capacity of each worker node, with the first subgraphs preferentially allocated to worker nodes with strong processing capacity.

[0066] Next, in step 23, multiple working nodes execute the target operation in parallel. The target operation includes calculating a local minimum spanning tree for a single first subgraph, thereby obtaining a local processing result. The local minimum spanning trees corresponding to each first subgraph form a global minimum spanning tree for the target relationship network graph. It is understood that since the first subgraph contains fewer nodes than the target relationship network graph, calculating the minimum spanning tree for the first subgraph significantly reduces the computational complexity compared to calculating the minimum spanning tree for the target relationship network graph.

[0067] In one example, calculating the local minimum spanning tree of the single first subgraph includes:

[0068] According to the n nodes included in the single first subgraph, construct a construction graph consisting of the n nodes and without any connecting edges;

[0069] Continuously extract the connecting edge with the smallest weight from the connecting edges included in the first subgraph. If the two nodes of the connecting edge come from different connected components, add this edge to the construction graph until all nodes are on the same connected component, and use the construction graph as the local minimum spanning tree corresponding to the first subgraph.

[0070] In this example, each time, the edge with the smallest weight that does not form a cycle is selected from all the edges that have not been selected in the first subgraph. The process is completed when n-1 edges are selected. The spanning tree formed by these n-1 edges is the minimum spanning tree corresponding to the first subgraph.

[0071] In another example, calculating the local minimum spanning tree of the single first subgraph includes:

[0072] Selecting a node from each node included in the single first subgraph as the starting point of the local minimum spanning tree corresponding to the first subgraph;

[0073] The selected nodes are added to the first set, and the other nodes are added to the second set;

[0074] Iteratively executing the process of selecting a connecting edge with the smallest weight from edges directly connecting nodes in the first set and nodes in the second set, and adding the selected connecting edge and the nodes in the second set connected thereto into the local minimum spanning tree.

[0075] In this example, each time an edge is selected, it is selected from the edges that are directly connected to the nodes in the two sets, and the selected edges will not form a loop.

[0076] In one example, the target operation includes:

[0077] A subgraph extraction process is performed on the local minimum spanning tree to obtain a second subgraph as a local processing result.

[0078] In this example, multiple worker nodes independently compute local minimum spanning trees, and subsequently perform subsequent algorithms, such as subgraph extraction, on each local minimum spanning tree in parallel. This significantly opens up room for optimization and improves the efficiency of subsequent algorithms.

[0079] In another example, the local processing result is the local minimum spanning tree.

[0080] In this example, the working node no longer performs subsequent processing on the local minimum spanning tree, but directly uses the local minimum spanning tree as the local processing result. Since the process of generating each local minimum spanning tree is a parallel algorithm, the efficiency is much higher than directly generating the global minimum spanning tree of the target relationship network graph.

[0081] Furthermore, the subgraph extraction process includes:

[0082] Clustering is performed on each node of the minimum spanning tree to be analyzed to obtain several clusters;

[0083] Nodes and connecting edges of the same cluster are extracted from the minimum spanning tree to be analyzed as subgraph extraction results.

[0084] In this example, a graph clustering algorithm can be used to group the nodes in the graph to identify potential abnormal subgraphs.

[0085] Furthermore, the subgraph extraction process includes:

[0086] For the minimum spanning tree to be analyzed, the graph structure of the preset structural pattern is found through subgraph matching;

[0087] The found graph structure is used as the subgraph extraction result.

[0088] In this example, the subgraph isomorphism detection algorithm can be used to detect the occurrence of specific structural patterns in the large graph.

[0089] Finally, in step 24, the central node aggregates the local processing results of the multiple working nodes and performs global graph processing. It is understood that the above-mentioned global graph processing may include merging the multiple local processing results to obtain a merged result, and further graph analysis processing on the merged result.

[0090] In one example, the target operation includes:

[0091] Performing subgraph extraction processing on the local minimum spanning tree to obtain a second subgraph as a local processing result;

[0092] The global graph processing includes:

[0093] Aggregate the multiple second subgraphs, and perform graph analysis based on the aggregation results.

[0094] In this example, multiple working nodes perform subgraph extraction processing on the local minimum spanning tree in parallel, which can further improve efficiency.

[0095] In another example, the local processing result is the local minimum spanning tree, and the global graph processing includes:

[0096] Merging the local minimum spanning trees to obtain a global minimum spanning tree of the target relationship network graph;

[0097] A subgraph extraction process is performed on the global minimum spanning tree to obtain a subgraph extraction result.

[0098] In this example, the central node performs subgraph extraction on the global minimum spanning tree. Since the process of generating each local minimum spanning tree is a parallel algorithm, the efficiency is much higher than directly generating the global minimum spanning tree of the target relationship network graph.

[0099] In one example, the target relationship network graph includes multiple nodes representing users, and there is a connection edge between two nodes with a preset association relationship; the global graph processing is used to detect user connection structures with preset risk types.

[0100] In this example, the efficiency of detecting user connection structures with preset risk types can be improved.

[0101] In addition, global graph processing can further identify abnormal nodes in the above user connection structure.

[0102] Figure 3 FIG. 1 shows a schematic diagram of a single-machine MST process according to an embodiment. Figure 3 , the traditional MST algorithm can only run on a single machine for a given graph network. Given a graph network, the single-machine MST will generate edges serially and sequentially, and the generation order is: [(0,1), (0,3), (0,2), (0,4), (4,7), (5,9), (5,6), (5,8), (6,7)]. Among them, (0,1) represents the connection edge between node 0 and node 1, (0,3) represents the connection edge between node 0 and node 3, (0,2) represents the connection edge between node 0 and node 2, (0,4) represents the connection edge between node 0 and node 4, (4,7) represents the connection edge between node 4 and node 7, (5,9) represents the connection edge between node 5 and node 9, (5,6) represents the connection edge between node 5 and node 6, (5,8) represents the connection edge between node 5 and node 8, and (6,7) represents the connection edge between node 6 and node 7.

[0103] However, as graphs grow in size, with industrial graphs now reaching tens of billions or even trillions of edges, generating the corresponding minimum spanning tree using a single-machine MST algorithm faces severe resource constraints and performance bottlenecks. This specification proposes a large-scale distributed community-based minimum spanning tree algorithm. First, a community algorithm is used to segment the graph network into communities. Then, a parallel distributed MST algorithm is applied to each segmented community. This solves the problem that MST can only be computed on a single machine, significantly improving computational efficiency.

[0104] Figure 4 FIG. 4 shows a schematic diagram of distributed MST processing according to an embodiment. Figure 4 , first partition the graph network into communities, then assign a working node to each community, and each working node performs its own MST processing. In this way, each working node is executed in parallel, which greatly improves the operating efficiency. Suppose the graph network is divided into two communities, and two working nodes are used to perform MST calculations on each community to obtain the MST results. Because each working node is executed independently and in parallel, the speed is increased by 2 times. The order in which working node 0 generates edges is: [(0,1), (0,3), (0,2), (0,4)], and the local minimum spanning tree 0 is obtained; the order in which working node 1 generates edges is: [(5,9), (5,6), (5,8), (6,7)], and the local minimum spanning tree 1 is obtained. The local minimum spanning tree 0 and the local minimum spanning tree 1 are connected by the edge (4,7) to obtain the global minimum spanning tree, which is the same as the global minimum spanning tree. Figure 3 The results are consistent.

[0105] According to the method provided in the embodiment of this specification, the central node first performs community detection on the target relationship network graph, and divides the target relationship network graph into multiple first subgraphs; each first subgraph corresponds to a community; then the central node distributes the multiple first subgraphs to multiple working nodes; then the multiple working nodes perform the target operation in parallel, and the target operation includes calculating the local minimum spanning tree of a single first subgraph, thereby obtaining a local processing result; wherein the local minimum spanning trees corresponding to each first subgraph form the global minimum spanning tree of the target relationship network graph; finally, the central node summarizes the local processing results of the multiple working nodes and performs global graph processing. As can be seen from the above, in the embodiment of this specification, the central node divides the target relationship network graph into multiple subgraphs, each community corresponds to a subgraph, and since the subgraph corresponds to the community, the nodes and edges have a closer connection in the local area. On this basis, multiple working nodes respectively calculate the minimum spanning tree corresponding to the subgraph. This process can not only effectively prune and delete unnecessary edges, but also significantly reduce the size of the subgraph, thereby improving the subsequent local processing results or the iterative performance of the graph-related algorithm in the global graph processing. In summary, this distributed processing method fully utilizes computing resources, is adaptable to graph analysis scenarios, and is highly efficient.

[0106] According to another embodiment, a distributed graph analysis system is provided. Figure 2 The method provided by the illustrated embodiment. Figure 5 FIG. 1 shows a schematic block diagram of a distributed graph analysis system according to an embodiment. Figure 5 As shown, the system 500 includes: a central node 51 and a plurality of working nodes 52;

[0107] The central node 51 is used to perform community detection on the target relationship network graph, divide the target relationship network graph into multiple first subgraphs; each first subgraph corresponds to a community; and distribute the multiple first subgraphs to multiple working nodes 52;

[0108] The plurality of working nodes 52 are configured to execute a target operation in parallel, wherein the target operation includes calculating a local minimum spanning tree of a single first subgraph, thereby obtaining a local processing result; wherein the local minimum spanning trees corresponding to the respective first subgraphs form a global minimum spanning tree of the target relationship network graph;

[0109] The central node 51 is also used to aggregate the local processing results of the multiple working nodes 52 and perform global graph processing.

[0110] Optionally, as an embodiment, the central node 51 is specifically configured to:

[0111] Obtaining an initial community to which any node in the target relationship network graph belongs;

[0112] Determine whether to merge adjacent communities based on whether the connection density between nodes in each community is improved after the community is merged;

[0113] When the community to which each node belongs no longer changes, the connecting edges between nodes in the same community are taken as a first subgraph.

[0114] Optionally, as an embodiment, the working node 52 is specifically configured to:

[0115] According to the n nodes included in the single first subgraph, construct a construction graph consisting of the n nodes and without any connecting edges;

[0116] Continuously extract the connecting edge with the smallest weight from the connecting edges included in the first subgraph. If the two nodes of the connecting edge come from different connected components, add this edge to the construction graph until all nodes are on the same connected component, and use the construction graph as the local minimum spanning tree corresponding to the first subgraph.

[0117] Optionally, as an embodiment, the working node 52 is specifically configured to:

[0118] Selecting a node from each node included in the single first subgraph as the starting point of the local minimum spanning tree corresponding to the first subgraph;

[0119] The selected nodes are added to the first set, and the other nodes are added to the second set;

[0120] Iteratively executing the process of selecting a connecting edge with the smallest weight from edges directly connecting nodes in the first set and nodes in the second set, and adding the selected connecting edge and the nodes in the second set connected thereto into the local minimum spanning tree.

[0121] Optionally, as an embodiment, the target operation includes:

[0122] Performing subgraph extraction processing on the local minimum spanning tree to obtain a second subgraph as a local processing result;

[0123] The global graph processing includes:

[0124] Aggregate the multiple second subgraphs, and perform graph analysis based on the aggregation results.

[0125] Optionally, as an embodiment, the local processing result is the local minimum spanning tree, and the global graph processing includes:

[0126] Merging the local minimum spanning trees to obtain a global minimum spanning tree of the target relationship network graph;

[0127] A subgraph extraction process is performed on the global minimum spanning tree to obtain a subgraph extraction result.

[0128] Furthermore, the subgraph extraction process includes:

[0129] Clustering is performed on each node of the minimum spanning tree to be analyzed to obtain several clusters;

[0130] Nodes and connecting edges of the same cluster are extracted from the minimum spanning tree to be analyzed as subgraph extraction results.

[0131] Furthermore, the subgraph extraction process includes:

[0132] For the minimum spanning tree to be analyzed, the graph structure of the preset structural pattern is found through subgraph matching;

[0133] The found graph structure is used as the subgraph extraction result.

[0134] Optionally, as an embodiment, the target relationship network graph includes multiple nodes representing users, and there is a connection edge between two nodes with a preset association relationship; the global graph processing is used to detect user connection structures with preset risk types.

[0135] Through the system provided by the embodiment of this specification, the central node 51 first performs community detection on the target relationship network graph, and divides the target relationship network graph into multiple first subgraphs; each first subgraph corresponds to a community; then the central node 51 distributes the multiple first subgraphs to multiple working nodes 52; then the multiple working nodes 52 perform the target operation in parallel, and the target operation includes calculating the local minimum spanning tree of a single first subgraph, thereby obtaining a local processing result; wherein the local minimum spanning tree corresponding to each first subgraph forms the global minimum spanning tree of the target relationship network graph; finally, the central node 51 summarizes the local processing results of the multiple working nodes 52 and performs global graph processing. As can be seen from the above, in the embodiment of this specification, the central node divides the target relationship network graph into multiple subgraphs, each community corresponds to a subgraph, and since the subgraph corresponds to the community, the nodes and edges have a closer connection in the local area. On this basis, multiple working nodes respectively calculate the minimum spanning tree corresponding to the subgraph. This process can not only effectively prune and delete unnecessary edges, but also significantly reduce the size of the subgraph, thereby improving the subsequent local processing results or the iterative performance of the graph-related algorithm in the global graph processing. In summary, this distributed processing method fully utilizes computing resources, is adaptable to graph analysis scenarios, and is highly efficient.

[0136] According to another embodiment, there is also provided a computer readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute a combination of Figure 2 The method described.

[0137] According to another embodiment, a computing device is provided, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the system realizes the combination of Figure 2 The method described.

[0138] Those skilled in the art will appreciate that, in one or more of the above examples, the functions described herein may be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions may be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium.

[0139] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made on the basis of the technical solution of the present invention should be included in the scope of protection of the present invention.

Claims

1. A distributed graph analysis method, the method comprising: The central node performs community detection on the target relationship network graph, and divides the target relationship network graph into a plurality of first subgraphs; Each first subgraph corresponds to a community; The central node distributes the multiple first subgraphs to multiple working nodes; The multiple working nodes perform the target operation in parallel, and the target operation includes calculating the local minimum spanning tree of a single first subgraph, thereby obtaining a local processing result; wherein the local minimum spanning trees corresponding to each first subgraph form a global minimum spanning tree of the target relationship network graph; The central node aggregates the local processing results of the multiple working nodes and performs global graph processing.

2. The method of claim 1, wherein: The central node performs community detection on the target relationship network graph, including: Obtaining an initial community to which any node in the target relationship network graph belongs; Determine whether to merge adjacent communities based on whether the connection density between nodes in each community is improved after the community is merged; When the community to which each node belongs no longer changes, the connecting edges between nodes in the same community are taken as a first subgraph.

3. The method of claim 1, wherein: The step of calculating a local minimum spanning tree of a single first subgraph includes: According to the n nodes included in the single first subgraph, construct a construction graph consisting of the n nodes and not containing any connecting edges; Continuously extract the connecting edge with the smallest weight from the connecting edges included in the first subgraph. If the two nodes of the connecting edge come from different connected components, add this edge to the construction graph until all nodes are on the same connected component, and use the construction graph as the local minimum spanning tree corresponding to the first subgraph.

4. The method of claim 1, wherein: The step of calculating a local minimum spanning tree of a single first subgraph includes: Selecting a node from each node included in the single first subgraph as the starting point of the local minimum spanning tree corresponding to the first subgraph; The selected nodes are added to the first set, and the other nodes are added to the second set; Iterative execution selects a connecting edge with the smallest weight from the edges directly connecting the nodes in the first set and the nodes in the second set, and the selected connecting edge and the nodes in the second set connected to it are added to the local minimum spanning tree.

5. The method of claim 1, wherein: The target operations include: Perform subgraph extraction processing on the local minimum spanning tree to obtain a second subgraph as a local processing result; The global graph processing includes: Aggregate the multiple second subgraphs, and perform graph analysis based on the aggregation results.

6. The method of claim 1, wherein: The local processing result is the local minimum spanning tree, and the global graph processing includes: Merging the local minimum spanning trees to obtain a global minimum spanning tree of the target relationship network graph; A subgraph extraction process is performed on the global minimum spanning tree to obtain a subgraph extraction result.

7. The method according to claim 5 or 6, wherein: The sub-graph extraction process comprises: Clustering is performed on each node of the minimum spanning tree to be analyzed to obtain several clusters; Each node and connecting edge of the same cluster is extracted from the minimum spanning tree to be analyzed as a subgraph extraction result.

8. The method according to claim 5 or 6, wherein: The sub-graph extraction process comprises: For the minimum spanning tree to be analyzed, the graph structure of the preset structural pattern is searched through subgraph matching; The found graph structure is used as the subgraph extraction result.

9. The method of claim 1, wherein: The target relationship network graph includes multiple nodes representing users, and there is a connection edge between two nodes with a preset association relationship; the global graph processing is used to detect a user connection structure with a preset risk type.

10. A distributed graph analysis system, the system comprising: Central node and multiple working nodes; The central node is used to perform community detection on the target relationship network graph and divide the target relationship network graph into multiple first subgraphs; Each first subgraph corresponds to a community; distributing the multiple first subgraphs to multiple working nodes; The multiple working nodes are used to perform the target operation in parallel, and the target operation includes calculating the local minimum spanning tree of a single first subgraph to obtain a local processing result; wherein the local minimum spanning trees corresponding to each first subgraph form a global minimum spanning tree of the target relationship network graph; The central node is also used to aggregate the local processing results of the multiple working nodes and perform global graph processing.

11. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method according to any one of claims 1 to 9.

12. A computing device comprising a memory and a processor, wherein the memory stores executable codes, and when the processor executes the executable codes, the method according to any one of claims 1 to 9 is implemented.

Images