Distributed graph analysis method and system

Abstract

This specification provides a distributed graph analysis method and system. The method includes: a central node performing community detection on a target relational network graph, dividing the target relational network graph into multiple first subgraphs; each first subgraph corresponds to a community; the central node distributes the multiple first subgraphs to multiple worker nodes; the multiple worker nodes execute target operations in parallel, the target operations including calculating the local minimum spanning tree of a single first subgraph to obtain local processing results; wherein the local minimum spanning trees corresponding to each first subgraph form the global minimum spanning tree of the target relational network graph; the central node summarizes the local processing results of the multiple worker nodes and performs global graph processing. This method is adaptable to graph analysis scenarios and has high efficiency.

Description

Technical Field

[0001] This specification relates to one or more embodiments in the field of computer science, and more particularly to distributed graph analysis methods and systems. Background Technology

[0002] With the rapid growth of data in fields such as social networks, internet data, the Internet of Things, and bioinformatics, there is a growing demand for processing ultra-large-scale graph data with over 10 billion nodes. This graph data is massive in scale and complex in structure, making it difficult for traditional graph analysis solutions to operate efficiently at this scale, resulting in a significant decrease in iterative performance. Furthermore, the graph data may contain users' private data.

[0003] In existing technologies, for general scenarios, there are some more efficient algorithms that can optimize computing resources and improve decision-making efficiency, such as distributed computing. By decomposing and distributing computing tasks across multiple nodes, the scalability and processing efficiency of the algorithm can be significantly improved. However, when facing graph analysis scenarios, simple distributed strategies are not very effective in improving efficiency. Therefore, there is a need to provide more efficient distributed graph analysis solutions. Summary of the Invention

[0004] This specification describes one or more embodiments of a distributed graph analysis method and system that is adaptable to graph analysis scenarios and has high efficiency.

[0005] Firstly, a distributed graph analysis method is provided, which includes:

[0006] The central node performs community detection on the target relationship network graph and divides the target relationship network graph into multiple first subgraphs; each first subgraph corresponds to a community.

[0007] The central node distributes the multiple first subgraphs to multiple worker nodes;

[0008] The multiple working nodes execute the target operation in parallel. The target operation includes calculating the local minimum spanning tree of a single first subgraph to obtain a local processing result. The local minimum spanning trees corresponding to each first subgraph form the global minimum spanning tree of the target relational network graph.

[0009] The central node aggregates the local processing results of the multiple working nodes and performs global graph processing.

[0010] In one possible implementation, the central node performs community detection on the target relational network graph, including:

[0011] Obtain the initial community to which any node in the target relational network graph belongs;

[0012] Whether to merge adjacent communities depends on whether the connection density between nodes in each community increases after the community merger.

[0013] Once the community to which each node belongs no longer changes, nodes within the same community and the connecting edges between nodes are treated as a first subgraph.

[0014] In one possible implementation, computing the local minimum spanning tree of a single first subgraph includes:

[0015] Based on the n nodes included in a single first subgraph, construct a constructed graph consisting of the n nodes and containing no connecting edges;

[0016] Continuously extract the connecting edge with the smallest weight from the connecting edges included in the first subgraph. If the two nodes of the connecting edge come from different connected components, add this edge to the constructed graph until all nodes are on the same connected component. Then, use the constructed graph as the local minimum spanning tree corresponding to the first subgraph.

[0017] In one possible implementation, computing the local minimum spanning tree of a single first subgraph includes:

[0018] Select one node from the nodes included in a single first subgraph as the starting point of the local minimum spanning tree corresponding to that first subgraph;

[0019] The selected nodes are added to the first set, and the other nodes are added to the second set.

[0020] Iteratively, the edge with the smallest weight is selected from the edges that directly connect the nodes in the first set and the nodes in the second set. The selected edge and the node in the second set that it connects are added to the local minimum spanning tree.

[0021] In one possible implementation, the target operation includes:

[0022] The local minimum spanning tree is processed by subgraph extraction, and the second subgraph is obtained as the local processing result.

[0023] The global graph processing includes:

[0024] Summarize multiple second subgraphs and perform graph analysis based on the summarization results.

[0025] In one possible implementation, the local processing result is the local minimum spanning tree, and the global graph processing includes:

[0026] The local minimum spanning trees are merged to obtain the global minimum spanning tree of the target relational network graph;

[0027] Subgraph extraction is performed on the global minimum spanning tree to obtain the subgraph extraction result.

[0028] Furthermore, the subgraph extraction process includes:

[0029] Clustering is performed on each node of the minimum spanning tree to be analyzed to obtain several clusters;

[0030] Extract the nodes and connecting edges of the same cluster from the minimum spanning tree to be analyzed, and use them as the subgraph extraction results.

[0031] Furthermore, the subgraph extraction process includes:

[0032] For the minimum spanning tree to be analyzed, the graph structure with the preset structure pattern is found by subgraph matching;

[0033] The found graph structure is used as the subgraph extraction result.

[0034] In one possible implementation, the target relationship network graph includes multiple nodes representing users, and there are connecting edges between two nodes with a preset association relationship; the global graph processing is used to detect user connection structures with preset risk types.

[0035] Secondly, a distributed graph analysis system is provided, which includes a central node and multiple worker nodes.

[0036] The central node is used to perform community detection on the target relationship network graph, dividing the target relationship network graph into multiple first subgraphs; each first subgraph corresponds to a community; and the multiple first subgraphs are distributed to multiple working nodes.

[0037] The multiple working nodes are used to execute target operations in parallel. The target operations include calculating the local minimum spanning tree of a single first subgraph to obtain a local processing result; wherein the local minimum spanning trees corresponding to each first subgraph form the global minimum spanning tree of the target relational network graph.

[0038] The central node is also used to summarize the local processing results of the multiple working nodes and perform global graph processing.

[0039] Thirdly, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method of the first aspect.

[0040] Fourthly, a computing device is provided, including a memory and a processor, wherein the memory stores executable code, and the processor executes the executable code to implement the method of the first aspect.

[0041] The method and apparatus provided in the embodiments of this specification first perform community detection on the target relationship network graph by a central node, dividing the target relationship network graph into multiple first subgraphs; each first subgraph corresponds to a community; then, the central node distributes the multiple first subgraphs to multiple worker nodes; next, the multiple worker nodes execute target operations in parallel, the target operations including calculating the local minimum spanning tree of a single first subgraph to obtain local processing results; wherein, the local minimum spanning trees corresponding to each first subgraph form the global minimum spanning tree of the target relationship network graph; finally, the central node summarizes the local processing results of the multiple worker nodes and performs global graph processing. As can be seen from the above, in the embodiments of this specification, the central node divides the target relationship network graph into multiple subgraphs, with each community corresponding to one subgraph. Since the subgraphs correspond to communities, nodes and edges have closer connections within the local area. Based on this, the multiple worker nodes calculate the minimum spanning tree corresponding to each subgraph. This process not only effectively prunes and deletes unnecessary edges but also significantly reduces the size of the subgraphs, thereby improving the iterative performance of graph-related algorithms in obtaining subsequent local processing results or global graph processing. In summary, this distributed processing method makes full use of computing resources, is suitable for graph analysis scenarios, and is highly efficient. Attached Figure Description

[0042] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 This is a schematic diagram illustrating an implementation scenario of one embodiment disclosed in this specification;

[0044] Figure 2 A flowchart of a distributed graph analysis method according to one embodiment is shown;

[0045] Figure 3 A schematic diagram of a standalone MST process according to one embodiment is shown;

[0046] Figure 4 This diagram illustrates a distributed MST process according to one embodiment.

[0047] Figure 5 A schematic block diagram of a distributed graph analysis system according to one embodiment is shown. Detailed Implementation

[0048] The solution provided in this specification will now be described with reference to the accompanying drawings.

[0049] Figure 1 This is a schematic diagram illustrating an implementation scenario of one embodiment disclosed in this specification. This implementation scenario involves distributed graph analysis. It is understood that distributed computing decomposes and distributes computational tasks across multiple nodes, significantly improving the scalability and processing efficiency of the algorithm. The computational task of distributed graph analysis is specifically a graph analysis task, often involving multiple rounds of iterative processing. For example, common graph analysis tasks include subgraph analysis. The application of subgraph analysis algorithms in financial risk control mainly lies in identifying and assessing complex relationships related to financial transactions, customer relationships, and social networks. Subgraph analysis is a graph theory method that can extract specific substructures from large-scale graph data, thereby revealing potential risk factors. For example, subgraph analysis can be used for fraud detection. In financial transactions, fraudulent behavior often manifests as complex transaction networks. By constructing a transaction graph, representing nodes such as customers, accounts, and transactions as vertices in the graph, and transaction relationships as edges, subgraph analysis can help identify suspicious small subgraph structures. For example, a small subgraph structure corresponds to aggregation behavior, where multiple accounts transfer funds to the same account within a short period of time, which may indicate fraudulent activity; or, a small subgraph structure corresponds to abnormal connections, where the connection patterns between some nodes are abnormal, such as some accounts frequently engaging in high-value transactions with new accounts, which may represent collusive fraud.

[0050] like Figure 1 As shown, distributed graph analysis involves multiple participants, including a central node and multiple worker nodes. Each participant can be implemented as any device, platform, server, or device cluster with computing and processing capabilities. The central node divides the large-scale graph data into multiple subgraphs and assigns these subgraphs to multiple worker nodes. Each worker node independently performs graph analysis processing on its assigned subgraph in parallel, obtaining local processing results for each subgraph. The central node can then aggregate the local processing results from multiple worker nodes and perform global graph processing, which helps improve efficiency. For example, the central node divides the large-scale graph data into N+1 subgraphs, denoted as subgraph 0, ..., subgraph N. Subgraph 0 is assigned to worker node 0, ..., and subgraph N is assigned to worker node N. Worker node 0 independently performs graph analysis processing on its assigned subgraph 0, obtaining local processing result 0, ..., and worker node N independently performs graph analysis processing on its assigned subgraph N, obtaining local processing result N for subgraph N.

[0051] In this embodiment, the central node partitions the input graph according to communities, obtaining subgraphs corresponding to each community. Then, worker nodes generate minimum spanning trees for each individual subgraph, thereby pruning the subgraph, removing unnecessary edges, and significantly reducing the subgraph size. Community-based minimum spanning trees help reduce unimportant nodes, thus accelerating the iterative performance of graph-related algorithms. Parallel processing is achieved through a distributed computing platform including multiple worker nodes, further improving the efficiency of large-scale graph processing.

[0052] The following is a brief explanation of community partitioning and minimum spanning tree.

[0053] Community detection is an important concept in network analysis, primarily used to identify highly interconnected subsets within social networks or other types of graphs (such as biological networks and information networks). These subsets are called communities. Nodes within each community have a high connection density, while connections between nodes in other communities are relatively low. Community detection helps in understanding the structure and function of networks and uncovering important patterns or features. Common community detection algorithms include modularity optimization-based methods, spectral clustering, and label propagation algorithms. Community detection has wide applications in social network analysis, recommender systems, and information propagation.

[0054] The minimum spanning tree (MST) is an important concept in graph theory, relating to connected undirected graphs. Given a weighted graph, the minimum spanning tree is a subgraph containing all vertices of the graph and possessing the following properties: this subgraph is a spanning tree, meaning it has no cycles and all vertices are connected; and the sum of the weights of all edges is minimized. Minimum spanning trees can be used to solve many practical problems, such as network design, circuit placement, and cluster analysis. Commonly used algorithms for computing minimum spanning trees include Prim's algorithm and Kruskal's algorithm.

[0055] This specification describes an embodiment that achieves efficient local computation by dividing a large graph network into multiple relatively independent subgraphs corresponding to communities. A preliminary minimum spanning tree is built within each community through parallel processing, and finally, these local results are integrated into a global minimum spanning tree through an efficient aggregation mechanism. This community-based distributed algorithm not only effectively reduces the burden of single-point computation but also improves processing speed and scalability. With the rapid development of modern computing resources, especially the rise of cloud computing and edge computing, this algorithm can be widely applied to fields such as social network analysis, intelligent traffic management, and large-scale sensor networks.

[0056] Figure 2This diagram illustrates a distributed graph analysis method according to one embodiment, which can be based on... Figure 1 The implementation scenario shown is executed jointly by a central node and multiple worker nodes. For example... Figure 2 As shown, the distributed graph analysis method in this embodiment includes the following steps: Step 21, the central node performs community detection on the target relationship network graph, dividing the target relationship network graph into multiple first subgraphs; each first subgraph corresponds to a community; Step 22, the central node distributes the multiple first subgraphs to multiple worker nodes; Step 23, the multiple worker nodes execute the target operation in parallel, the target operation including calculating the local minimum spanning tree of a single first subgraph to obtain the local processing result; wherein, the local minimum spanning trees corresponding to each first subgraph form the global minimum spanning tree of the target relationship network graph; Step 24, the central node summarizes the local processing results of the multiple worker nodes and performs global graph processing. The specific execution method of each of the above steps is described below.

[0057] First, in step 21, the central node performs community detection on the target relationship network graph, dividing the target relationship network graph into multiple first subgraphs; each first subgraph corresponds to a community. It can be understood that a community is a subset of the target relationship network graph with high internal connectivity. Community detection, or community partitioning, can be performed in various ways, using any existing community partitioning method.

[0058] In one example, the central node performs community detection on the target relational network graph, including:

[0059] Obtain the initial community to which any node in the target relational network graph belongs;

[0060] Whether to merge adjacent communities depends on whether the connection density between nodes in each community increases after the community merger.

[0061] Once the community to which each node belongs no longer changes, nodes within the same community and the connecting edges between nodes are treated as a first subgraph.

[0062] In this example, the process of repeatedly determining whether to merge adjacent communities can be repeated, essentially performing multiple iterations, until the community to which each node belongs no longer changes, at which point the iterations stop. Specifically, the target relationship network graph can be initialized so that each node corresponds to one community; that is, the number of nodes in the target relationship network graph is the same as the initial number of communities.

[0063] In the embodiments of this specification, the meanings of nodes and connecting edges in the target relationship network graph are broad. Typically, the target relationship network graph includes multiple nodes representing users, and two nodes with a preset association relationship are connected by an edge. The aforementioned preset association relationship may include, but is not limited to, transaction relationships, etc.

[0064] Then, in step 22, the central node distributes the multiple first subgraphs to multiple worker nodes. It is understood that the number of first subgraphs can be the same as or different from the number of worker nodes. For example, N first subgraphs are distributed to N worker nodes, with each worker node receiving one first subgraph; or, 2N first subgraphs are distributed to N worker nodes, with each worker node receiving two first subgraphs.

[0065] In the embodiments described in this specification, the subgraph distribution method can be varied. It can be distributed evenly, with each worker node receiving the same number of first subgraphs; or it can be distributed according to the processing capacity of each worker node, prioritizing the allocation of first subgraphs to worker nodes with higher processing capacity.

[0066] Next, in step 23, multiple worker nodes execute the target operation in parallel. The target operation includes calculating the local minimum spanning tree of a single first subgraph to obtain a local processing result. The local minimum spanning trees corresponding to each first subgraph form the global minimum spanning tree of the target relational network graph. It is understood that the first subgraph contains fewer nodes than the target relational network graph; therefore, calculating the minimum spanning tree for the first subgraph significantly reduces computational complexity compared to calculating the minimum spanning tree for the target relational network graph.

[0067] In one example, computing the local minimum spanning tree of a single first subgraph includes:

[0068] Based on the n nodes included in a single first subgraph, construct a constructed graph consisting of the n nodes and containing no connecting edges;

[0069] Continuously extract the connecting edge with the smallest weight from the connecting edges included in the first subgraph. If the two nodes of the connecting edge come from different connected components, add this edge to the constructed graph until all nodes are on the same connected component. Then, use the constructed graph as the local minimum spanning tree corresponding to the first subgraph.

[0070] In this example, each time, the edge with the smallest weight that will not form a cycle is selected from all the edges that have not yet been selected in the first subgraph. Once n-1 edges have been selected, the process is complete. The spanning tree formed by these n-1 edges is the minimum spanning tree corresponding to the first subgraph.

[0071] In another example, computing the local minimum spanning tree of a single first subgraph includes:

[0072] Select one node from the nodes included in a single first subgraph as the starting point of the local minimum spanning tree corresponding to that first subgraph;

[0073] The selected nodes are added to the first set, and the other nodes are added to the second set.

[0074] Iteratively, the edge with the smallest weight is selected from the edges that directly connect the nodes in the first set and the nodes in the second set. The selected edge and the node in the second set that it connects are added to the local minimum spanning tree.

[0075] In this example, each time an edge is selected, it is chosen from the edges that directly connect the nodes in the two sets, and the selected edges will not form a cycle.

[0076] In one example, the target operation includes:

[0077] The local minimum spanning tree is processed by subgraph extraction, and the resulting second subgraph is used as the local processing result.

[0078] In this example, multiple worker nodes independently compute their local minimum spanning trees, meaning that subsequent algorithms, such as subgraph extraction, are performed on each local minimum spanning tree in parallel. This greatly expands the optimization space for subsequent operations and improves the efficiency of subsequent algorithms.

[0079] In another example, the result of the local processing is the local minimum spanning tree.

[0080] In this example, the worker node no longer processes the local minimum spanning tree, but directly uses the local minimum spanning tree as the local processing result. Since the process of generating each local minimum spanning tree is a parallel algorithm, it is much more efficient than directly generating the global minimum spanning tree of the target relational network graph.

[0081] Furthermore, the subgraph extraction process includes:

[0082] Clustering is performed on each node of the minimum spanning tree to be analyzed to obtain several clusters;

[0083] Extract the nodes and connecting edges of the same cluster from the minimum spanning tree to be analyzed, and use them as the subgraph extraction results.

[0084] In this example, a graph clustering algorithm can be used to group the nodes in the graph to identify potential anomalous subgraphs.

[0085] Furthermore, the subgraph extraction process includes:

[0086] For the minimum spanning tree to be analyzed, the graph structure with the preset structure pattern is found by subgraph matching;

[0087] The found graph structure is used as the subgraph extraction result.

[0088] In this example, a subgraph isomorphism detection algorithm can be used to detect the occurrence of a specific structural pattern in a large graph.

[0089] Finally, in step 24, the central node summarizes the local processing results of the multiple working nodes and performs global graph processing. It can be understood that the above global graph processing may include merging multiple local processing results to obtain a merged result, as well as further graph analysis processing on the merged result.

[0090] In one example, the target operation includes:

[0091] The local minimum spanning tree is processed by subgraph extraction, and the second subgraph is obtained as the local processing result.

[0092] The global graph processing includes:

[0093] Summarize multiple second subgraphs and perform graph analysis based on the summarization results.

[0094] In this example, multiple worker nodes perform subgraph extraction on their local minimum spanning tree in parallel, which can further improve efficiency.

[0095] In another example, the local processing result is the local minimum spanning tree, and the global graph processing includes:

[0096] The local minimum spanning trees are merged to obtain the global minimum spanning tree of the target relational network graph;

[0097] Subgraph extraction is performed on the global minimum spanning tree to obtain the subgraph extraction result.

[0098] In this example, the central node performs subgraph extraction on the global minimum spanning tree. Since the process of generating each local minimum spanning tree is a parallel algorithm, it is much more efficient than directly generating the global minimum spanning tree of the target relational network graph.

[0099] In one example, the target relationship network graph includes multiple nodes representing users, and there are connecting edges between two nodes with a preset association relationship; the global graph processing is used to detect user connection structures with preset risk types.

[0100] This example can improve the efficiency of detecting user connection structures with preset risk types.

[0101] Furthermore, global graph processing can further identify abnormal nodes in the aforementioned user connection structure.

[0102] Figure 3 A schematic diagram of a standalone MST process according to one embodiment is shown. (Refer to...) Figure 3 The traditional MST algorithm, for a given graph network, can only run on a single machine. Given a graph network, a single-machine MST will generate edges serially and sequentially, in the following order: [(0,1),(0,3),(0,2),(0,4),(4,7),(5,9),(5,6),(5,8),(6,7)]. Here, (0,1) represents the edge connecting node 0 and node 1, (0,3) represents the edge connecting node 0 and node 3, (0,2) represents the edge connecting node 0 and node 2, (0,4) represents the edge connecting node 0 and node 4, (4,7) represents the edge connecting node 4 and node 7, (5,9) represents the edge connecting node 5 and node 9, (5,6) represents the edge connecting node 5 and node 6, (5,8) represents the edge connecting node 5 and node 8, and (6,7) represents the edge connecting node 6 and node 7.

[0103] However, as graphs grow increasingly large, with current industrial graphs reaching tens or even trillions of edges, generating the corresponding minimum spanning tree using a single-machine MST (Minimum Spanning Tree) approach faces severe resource constraints and performance bottlenecks. This specification proposes a large-scale distributed, community-based minimum spanning tree algorithm. First, a community algorithm is used to partition the graph network into communities. Then, a parallel, distributed MST algorithm is performed on each partitioned community, thus solving the problem that MST can only be computed on a single machine and significantly improving computational efficiency.

[0104] Figure 4 A schematic diagram of distributed MST processing according to one embodiment is shown. (Refer to...) Figure 4 The graph network is first partitioned into communities, and then a worker node is assigned to each community. Each worker node performs its own Minimum Spanning Tree (MST) calculation. This allows each worker node to execute in parallel, greatly improving efficiency. For example, if the graph network is divided into two communities, each worker node performs its own MST calculation for each community, yielding the MST result. Because each worker node executes independently and in parallel, the speed is increased by a factor of two. Worker node 0 generates edges in the order [(0,1),(0,3),(0,2),(0,4)], resulting in Local Minimum Spanning Tree 0; worker node 1 generates edges in the order [(5,9),(5,6),(5,8),(6,7)], resulting in Local Minimum Spanning Tree 1. Local Minimum Spanning Tree 0 and Local Minimum Spanning Tree 1 are connected by edge (4,7) to obtain the global minimum spanning tree. This global minimum spanning tree is then connected to... Figure 3 The results were consistent.

[0105] The method provided in the embodiments of this specification first involves a central node performing community detection on the target relationship network graph, dividing the target relationship network graph into multiple first subgraphs; each first subgraph corresponds to a community; then, the central node distributes the multiple first subgraphs to multiple worker nodes; next, the multiple worker nodes execute target operations in parallel, the target operations including calculating the local minimum spanning tree of a single first subgraph to obtain local processing results; wherein, the local minimum spanning trees corresponding to each first subgraph form the global minimum spanning tree of the target relationship network graph; finally, the central node summarizes the local processing results of the multiple worker nodes and performs global graph processing. As can be seen from the above, in the embodiments of this specification, the central node divides the target relationship network graph into multiple subgraphs, with each community corresponding to one subgraph. Since the subgraphs correspond to communities, nodes and edges have closer connections within the local area. Based on this, multiple worker nodes calculate the minimum spanning tree corresponding to each subgraph. This process not only effectively prunes and deletes unnecessary edges but also significantly reduces the size of the subgraphs, thereby improving the iterative performance of graph-related algorithms in obtaining subsequent local processing results or global graph processing. In summary, this distributed processing method makes full use of computing resources, is suitable for graph analysis scenarios, and is highly efficient.

[0106] According to another embodiment, a distributed graph analysis system is also provided, which is used to perform the functions described herein. Figure 2 The method provided in the illustrated embodiment. Figure 5 A schematic block diagram of a distributed graph analysis system according to one embodiment is shown. Figure 5 As shown, the system 500 includes: a central node 51 and multiple working nodes 52;

[0107] The central node 51 is used to perform community detection on the target relationship network graph, dividing the target relationship network graph into multiple first subgraphs; each first subgraph corresponds to a community; and the multiple first subgraphs are distributed to multiple worker nodes 52.

[0108] The plurality of working nodes 52 are used to execute target operations in parallel. The target operations include calculating the local minimum spanning tree of a single first subgraph to obtain a local processing result; wherein the local minimum spanning trees corresponding to each first subgraph form the global minimum spanning tree of the target relation network graph.

[0109] The central node 51 is also used to summarize the local processing results of the multiple working nodes 52 and perform global graph processing.

[0110] Optionally, as an embodiment, the central node 51 is specifically used for:

[0111] Obtain the initial community to which any node in the target relational network graph belongs;

[0112] Whether to merge adjacent communities depends on whether the connection density between nodes in each community increases after the community merger.

[0113] Once the community to which each node belongs no longer changes, nodes within the same community and the connecting edges between nodes are treated as a first subgraph.

[0114] Optionally, as an embodiment, the working node 52 is specifically used for:

[0115] Based on the n nodes included in a single first subgraph, construct a constructed graph consisting of the n nodes and containing no connecting edges;

[0116] Continuously extract the connecting edge with the smallest weight from the connecting edges included in the first subgraph. If the two nodes of the connecting edge come from different connected components, add this edge to the constructed graph until all nodes are on the same connected component. Then, use the constructed graph as the local minimum spanning tree corresponding to the first subgraph.

[0117] Optionally, as an embodiment, the working node 52 is specifically used for:

[0118] Select one node from the nodes included in a single first subgraph as the starting point of the local minimum spanning tree corresponding to that first subgraph;

[0119] The selected nodes are added to the first set, and the other nodes are added to the second set.

[0120] Iteratively, the edge with the smallest weight is selected from the edges that directly connect the nodes in the first set and the nodes in the second set. The selected edge and the node in the second set that it connects are added to the local minimum spanning tree.

[0121] Optionally, as an embodiment, the target operation includes:

[0122] The local minimum spanning tree is processed by subgraph extraction, and the second subgraph is obtained as the local processing result.

[0123] The global graph processing includes:

[0124] Summarize multiple second subgraphs and perform graph analysis based on the summarization results.

[0125] Optionally, as an embodiment, the local processing result is the local minimum spanning tree, and the global graph processing includes:

[0126] The local minimum spanning trees are merged to obtain the global minimum spanning tree of the target relational network graph;

[0127] Subgraph extraction is performed on the global minimum spanning tree to obtain the subgraph extraction result.

[0128] Furthermore, the subgraph extraction process includes:

[0129] Clustering is performed on each node of the minimum spanning tree to be analyzed to obtain several clusters;

[0130] Extract the nodes and connecting edges of the same cluster from the minimum spanning tree to be analyzed, and use them as the subgraph extraction results.

[0131] Furthermore, the subgraph extraction process includes:

[0132] For the minimum spanning tree to be analyzed, the graph structure with the preset structure pattern is found by subgraph matching;

[0133] The found graph structure is used as the subgraph extraction result.

[0134] Optionally, as an embodiment, the target relationship network graph includes multiple nodes representing users, and there are connecting edges between two nodes with a preset association relationship; the global graph processing is used to detect user connection structures with preset risk types.

[0135] The system provided in this embodiment first performs community detection on the target relationship network graph by a central node 51, dividing the target relationship network graph into multiple first subgraphs; each first subgraph corresponds to a community. Then, the central node 51 distributes the multiple first subgraphs to multiple worker nodes 52. Next, the multiple worker nodes 52 execute target operations in parallel. These target operations include calculating the local minimum spanning tree of a single first subgraph to obtain local processing results. The local minimum spanning trees corresponding to each first subgraph form the global minimum spanning tree of the target relationship network graph. Finally, the central node 51 summarizes the local processing results of the multiple worker nodes 52 and performs global graph processing. As can be seen from the above, in this embodiment, the central node divides the target relationship network graph into multiple subgraphs, with each community corresponding to one subgraph. Since the subgraphs correspond to communities, nodes and edges have closer connections within the local area. Based on this, the multiple worker nodes calculate the minimum spanning tree corresponding to each subgraph. This process not only effectively prunes and deletes unnecessary edges but also significantly reduces the size of the subgraphs, thereby improving the iterative performance of graph-related algorithms in obtaining subsequent local processing results or global graph processing. In summary, this distributed processing method makes full use of computing resources, is suitable for graph analysis scenarios, and is highly efficient.

[0136] According to another embodiment, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed in a computer, causes the computer to perform a combination Figure 2 The method described.

[0137] According to another embodiment, a computing device is also provided, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, it implements a combination... Figure 2 The method described.

[0138] Those skilled in the art will recognize that, in one or more of the examples above, the functions described in this invention can be implemented using hardware, software, firmware, or any combination thereof. When implemented in software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium.

[0139] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solution of the present invention should be included within the scope of protection of the present invention.

Claims

1. A distributed graph analysis method, the method comprising: The central node performs community detection on the target relationship network graph and divides the target relationship network graph into multiple first subgraphs; Each first subgraph corresponds to a community; The central node distributes the multiple first subgraphs to multiple worker nodes; The multiple working nodes execute the target operation in parallel. The target operation includes calculating the local minimum spanning tree of a single first subgraph to obtain a local processing result. The local minimum spanning trees corresponding to each first subgraph form the global minimum spanning tree of the target relational network graph. The central node summarizes the local processing results of the multiple working nodes and performs global graph processing; The central node performs community detection on the target relationship network graph, including: Obtain the initial community to which any node in the target relational network graph belongs; Whether to merge adjacent communities depends on whether the connection density between nodes in each community increases after the community merger. Once the community to which each node belongs no longer changes, nodes within the same community and the connecting edges between nodes are treated as a first subgraph.

2. The method as described in claim 1, wherein, The calculation of the local minimum spanning tree of a single first subgraph includes: Based on the n nodes included in a single first subgraph, construct a constructed graph consisting of the n nodes and containing no connecting edges; Continuously extract the connecting edge with the smallest weight from the connecting edges included in the first subgraph. If the two nodes of the connecting edge come from different connected components, add this edge to the constructed graph until all nodes are on the same connected component. Then, use the constructed graph as the local minimum spanning tree corresponding to the first subgraph.

3. The method as described in claim 1, wherein, The calculation of the local minimum spanning tree of a single first subgraph includes: Select one node from the nodes included in a single first subgraph as the starting point of the local minimum spanning tree corresponding to that first subgraph; The selected nodes are added to the first set, and the other nodes are added to the second set. Iteratively, the edge with the smallest weight is selected from the edges that directly connect the nodes in the first set and the nodes in the second set. The selected edge and the node in the second set that it connects are added to the local minimum spanning tree.

4. The method of claim 1, wherein, The target operation includes: The local minimum spanning tree is processed by subgraph extraction, and the second subgraph is obtained as the local processing result. The global graph processing includes: Summarize multiple second subgraphs and perform graph analysis based on the summarization results.

5. The method of claim 1, wherein, The local processing result is the local minimum spanning tree, and the global graph processing includes: The local minimum spanning trees are merged to obtain the global minimum spanning tree of the target relational network graph; Subgraph extraction is performed on the global minimum spanning tree to obtain the subgraph extraction result.

6. The method as described in claim 4 or 5, wherein, The subgraph extraction process includes: Clustering is performed on each node of the minimum spanning tree to be analyzed to obtain several clusters; Extract the nodes and connecting edges of the same cluster from the minimum spanning tree to be analyzed, and use them as the subgraph extraction results.

7. The method as described in claim 4 or 5, wherein, The subgraph extraction process includes: For the minimum spanning tree to be analyzed, the graph structure with the preset structure pattern is found by subgraph matching; The found graph structure is used as the subgraph extraction result.

8. The method of claim 1, wherein, The target relationship network graph includes multiple nodes representing users, and there are connecting edges between two nodes with a preset association relationship; the global graph processing is used to detect user connection structures with preset risk types.

9. A distributed graph analysis system, the system comprising: A central node and multiple worker nodes; The central node is used to perform community detection on the target relationship network graph and divide the target relationship network graph into multiple first subgraphs. Each first subgraph corresponds to a community; the multiple first subgraphs are distributed to multiple worker nodes; The multiple working nodes are used to execute target operations in parallel. The target operations include calculating the local minimum spanning tree of a single first subgraph to obtain a local processing result; wherein the local minimum spanning trees corresponding to each first subgraph form the global minimum spanning tree of the target relational network graph. The central node is also used to summarize the local processing results of the multiple working nodes and perform global graph processing; The central node performs community detection on the target relationship network graph, including: Obtain the initial community to which any node in the target relational network graph belongs; Whether to merge adjacent communities depends on whether the connection density between nodes in each community increases after the community merger. Once the community to which each node belongs no longer changes, nodes within the same community and the connecting edges between nodes are treated as a first subgraph.

10. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method of any one of claims 1-8.

11. A computing device comprising a memory and a processor, wherein the memory stores executable code, and the processor, when executing the executable code, implements the method of any one of claims 1-8.

Images