Community detection method and system based on probability distribution hierarchical graph clustering network

Abstract

The invention provides a community detection method and system based on a probability distribution hierarchical graph clustering network, and the method comprises the steps: carrying out the preprocessing of to-be-processed graph data based on a graph convolution network, extracting the node features of all nodes in the to-be-processed graph data, and determining the similarity between all nodes based on the node features; determining the probability distribution of each node based on the similarity, clustering each node to a corresponding affiliation community based on the probability distribution, and constructing a multi-level community tree diagram based on a clustering result; and carrying out refined division on communities of the multi-level community tree diagram based on the model loss function, and completing community detection on each node in the to-be-processed diagram data based on a refined division result. According to the community detection method and device, accurate and reliable community detection is carried out on each node, the accuracy of community detection is guaranteed, the community detection effect is greatly improved, and great convenience is provided for understanding of functions and characteristics of a network.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a community detection method and system based on a probability distribution hierarchical graph clustering network. Background Art

[0002] At present, with the development of network technology, various complex networks emerge in an endless stream, such as social networks, communication networks, biological networks, etc. In these networks, community structure is a common and important feature, which reflects the closely connected substructure between nodes in the network. Accurately detecting the community structure in the network is of great significance for understanding the functions and characteristics of the network;

[0003] However, existing community detection methods are insufficient in accuracy and efficiency when facing complex and changing network environments. At the same time, there may be uncertainty and random factors in the network, which makes it impossible to accurately analyze the community structure at all levels in the network, greatly affecting the detection effect of communities in various networks.

[0004] Therefore, in order to overcome the above defects, the present invention provides a community detection method and system based on probability distribution hierarchical graph clustering network. Summary of the invention

[0005] The present invention provides a community detection method and system based on a probability distribution hierarchical graph clustering network, which is used to pre-process the processed graph data by adopting a graph convolutional network, so as to accurately and effectively determine the node features in the processed graph data, and determine the similarity between the nodes by the node features, so as to determine the probability distribution of each node according to the similarity, thereby providing convenience for community detection. Secondly, the nodes are clustered by probability distribution, so as to lock the belonging communities corresponding to each node, and effectively construct a multi-level community dendrogram according to the belonging communities. Finally, the communities of the multi-level community dendrogram are refined and divided by a model loss function, so as to achieve accurate and reliable community detection of each node, thereby ensuring the accuracy of community detection, greatly improving the effect of community detection, and providing great convenience for understanding the functions and characteristics of the network.

[0006] The present invention provides a community detection method based on a probability distribution hierarchical graph clustering network, comprising:

[0007] Step 1: Preprocess the graph data to be processed based on the graph convolutional network, extract the node features of each node in the graph data to be processed, and determine the similarity between the nodes based on the node features;

[0008] Step 2: Determine the probability distribution of each node based on the similarity, cluster each node into the corresponding community based on the probability distribution, and construct a multi-level community dendrogram based on the clustering results;

[0009] Step 3: Based on the model loss function, the communities of the multi-level community tree diagram are refined and divided, and the community detection of each node in the processed graph data is completed based on the refined division results.

[0010] Preferably, a community detection method based on a probability distribution hierarchical graph clustering network, in step 1, before preprocessing the to-be-processed graph data based on a graph convolutional network, comprises:

[0011] Obtain the graph data to be processed sent by the management terminal, and traverse the graph data to be processed based on the processing index to obtain the index value of the graph data to be processed under the processing index;

[0012] Determine the image representation features of the image data to be processed based on the index value, and compare the image representation features with the preset requirements;

[0013] When the image representation features meet the preset requirements based on the difference comparison results, the image data to be processed is judged to be qualified, and the community detection mechanism for the image to be processed is triggered;

[0014] Otherwise, the graph data to be processed is returned.

[0015] Preferably, a community detection method based on a probability distribution hierarchical graph clustering network, in step 1, preprocessing the graph data to be processed based on a graph convolutional network, extracting node features of each node in the graph data to be processed, including:

[0016] Obtain the image data to be processed and divide the image to be processed into regions;

[0017] Based on the regional division results, the local region representation of each node in different regions is simultaneously determined, and the edge features, texture features and relative connection relationships between each node are obtained based on the local region representation;

[0018] Based on the edge features, texture features of each node and the relative connection relationship between each node, the node features of each node in the graph data to be processed are obtained, and the node features are vector-converted to obtain the feature vector corresponding to the node features of each node.

[0019] Preferably, a community detection method based on a probability distribution hierarchical graph clustering network, in step 1, determining the similarity between nodes based on node features, comprises:

[0020] Obtain the basic configuration parameters of the graph convolutional network, and determine the target number of layers of the graph convolutional network based on the basic configuration parameters;

[0021] Aggregate the node features of each node in the processed graph data and the node features of adjacent nodes in turn based on the target number of layers, and perform low-dimensional space transformation on the node features of each node based on the aggregation results to obtain node embedding features of the node features of each node;

[0022] The node features of a node and the node features of adjacent nodes are aggregated according to the following formula:

[0023]

[0024] Z v =H (L) ;

[0025] Among them, H (l+1) The target number of layers in the graph convolution network is the feature obtained by aggregating the node features of each node in the processed graph data and the node features of the adjacent nodes in turn; σ(·) represents the activation function; l represents the label value of the current target number of layers in the graph convolution network; represents a diagonal matrix; H represents the sum of the weighted adjacency matrix plus the adjacency matrix of the current layer; (l) Represents the feature matrix of the l-th layer node in the graph convolutional network; W (l) H represents the weight of the lth layer in the graph convolutional network; (L) Indicates H (l+1) The features obtained by gradually aggregating from the first layer to the L layer; Z v The node embedding matrix represents the node features of each node; v refers to the ordinal value of each node in the graph data to be processed;

[0026] The node embedding features between different nodes are analyzed to obtain the similarity between each node.

[0027] Preferably, a community detection method based on a probability distribution hierarchical graph clustering network performs a low-dimensional space transformation on the node features of each node based on the aggregation result to obtain node embedding features of the node features of each node, including:

[0028] Determine a weighted adjacency matrix of each node in the to-be-processed graph data based on the operation standard of the management terminal, and determine a feature loss value of the embedding feature of each node based on the weighted adjacency matrix of each node and the node embedding feature of each node;

[0029]

[0030] Among them, μ represents the feature loss value of each node embedding feature; i and j represent the ordinal value of the node, and the value range is [1, N]; N represents the total number of nodes; A ij represents the weighted adjacency matrix between node i and node j; σ(·) represents the activation function; H represents the transposed matrix of the node embedding matrix of node i; j is the node embedding matrix of node j;

[0031] The feature loss value of each node is compared with the preset threshold, and when the feature loss value is greater than the preset threshold, the node features of the current node and the node features of the adjacent nodes are re-aggregated until the feature loss value is less than the preset threshold.

[0032] Preferably, a community detection method based on a probability distribution hierarchical graph clustering network analyzes the node embedding features between different nodes to obtain the similarity between the nodes, including:

[0033] Obtain feature vectors corresponding to node embedding features of different nodes, and perform similarity calculation on the feature vectors based on a preset calculation formula. The execution steps include:

[0034]

[0035] Where sim(i,j) represents the cosine value of the angle between the feature vectors of node i and node j; Represents the feature vector obtained by gradually aggregating node i from the first layer to the L layer; represents the feature vector obtained by gradually aggregating node j from the first layer to the L layer; ‖·‖ represents the norm of the vector; Represents the modulus value corresponding to the feature vector obtained by gradually aggregating node i from the first layer to the L layer; Represents the modulus of the feature vector obtained by gradually aggregating node j from the first layer to the L layer;

[0036] The similarity between nodes is obtained based on the cosine value of the angle between the feature vectors of each node.

[0037] Preferably, a community detection method based on a probability distribution hierarchical graph clustering network, in step 2, the probability distribution of each node is determined based on the similarity, and each node is clustered into a corresponding belonging community based on the probability distribution, and a multi-level community dendrogram is constructed based on the clustering results, including:

[0038] Obtain the similarity between the obtained nodes, and determine the node pairs between the nodes in the graph data to be processed based on the similarity;

[0039] At the same time, the similarity of the node pairs is mapped to a preset function for processing to obtain the sampling probability of each node pair, and the node distance of each node pair is determined based on the sampling probability;

[0040] Based on the similarity between nodes and the node distance between each node pair, the probability distribution of each node is obtained, and each node is clustered based on the probability distribution result to obtain the corresponding community of each node;

[0041] Based on the clustering results, the similarities between different communities are determined in turn, and the communities with similarities greater than a preset threshold are merged to obtain a new community;

[0042] Based on the community merging results, the communities corresponding to each node are iteratively merged for a target number of times, and the target distance between the new community and the remaining communities after each iterative merging process is updated, and a multi-level community dendrogram is obtained based on the updated results.

[0043] Preferably, a community detection method based on a probability distribution hierarchical graph clustering network, in step 3, the community of the multi-level community dendrogram is refined based on the model loss function, and the community detection of each node in the processed graph data is completed based on the refined division result, including:

[0044] The obtained multi-level community dendrogram is obtained, and initial parameters of modularity and single adjustment parameters are configured for the multi-level community dendrogram based on community detection requirements;

[0045] Based on the single adjustment parameter, the initial parameter of the modularity is incrementally adjusted, and based on the incremental adjustment result, the local structure of each belonging community in the multi-level community dendrogram is refined and divided;

[0046] Based on the refined division results, the final community detection results of the graph data to be processed are obtained, and the community detection of each node in the graph data to be processed is completed.

[0047] The present invention provides a community detection system based on a probability distribution hierarchical graph clustering network, comprising:

[0048] A similarity determination module is used to pre-process the graph data to be processed based on the graph convolution network, extract the node features of each node in the graph data to be processed, and determine the similarity between the nodes based on the node features;

[0049] A community detection module is used to determine the probability distribution of each node based on similarity, cluster each node into the corresponding community based on the probability distribution, and construct a multi-level community dendrogram based on the clustering results;

[0050] The community refinement module is used to refine the communities of the multi-level community dendrogram based on the model loss function, and complete the community detection of each node in the processed graph data based on the refined division results.

[0051] Preferably, a community detection system based on a probability distribution hierarchical graph clustering network, a similarity determination module, comprises:

[0052] An image preparation unit is used to obtain the to-be-processed graph data sent by the management terminal, and traverse the to-be-processed graph data based on the processing index to obtain the index value of the to-be-processed graph data under the processing index;

[0053] Image verification unit, used for:

[0054] Determine the image representation features of the image data to be processed based on the index value, and compare the image representation features with the preset requirements;

[0055] When the image representation features meet the preset requirements based on the difference comparison results, the image data to be processed is judged to be qualified, and the community detection mechanism for the image to be processed is triggered;

[0056] Otherwise, the graph data to be processed is returned.

[0057] Compared with the prior art, the present invention has the following beneficial effects:

[0058] By using a graph convolutional network to preprocess the graph data to be processed, the node features in the graph data to be processed can be accurately and effectively determined, and the similarity between the nodes can be determined through the node features, so that the probability distribution of each node can be determined according to the similarity, which provides convenience for community detection. Secondly, the nodes are clustered through probability distribution to lock the corresponding communities of each node, and the multi-level community dendrogram can be effectively constructed according to the communities. Finally, the communities of the multi-level community dendrogram are refined and divided through the model loss function, so that accurate and reliable community detection can be achieved for each node, which ensures the accuracy of community detection, greatly improves the effect of community detection, and provides great convenience for understanding the functions and characteristics of the network.

[0059] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures specifically pointed out in this application document.

[0060] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0062] Figure 1 It is a flow chart of a community detection method based on a probability distribution hierarchical graph clustering network in an embodiment of the present invention;

[0063] Figure 2 This is a flowchart of step 1 in a community detection method based on a probability distribution hierarchical graph clustering network in an embodiment of the present invention;

[0064] Figure 3 It is a structural diagram of a community detection system based on a probability distribution hierarchical graph clustering network in an embodiment of the present invention. DETAILED DESCRIPTION

[0065] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0066] Embodiment 1:

[0067] This embodiment provides a community detection method based on a probability distribution hierarchical graph clustering network. Figure 1 As shown, including:

[0068] Step 1: Preprocess the graph data to be processed based on the graph convolutional network, extract the node features of each node in the graph data to be processed, and determine the similarity between the nodes based on the node features;

[0069] Step 2: Determine the probability distribution of each node based on the similarity, cluster each node into the corresponding community based on the probability distribution, and construct a multi-level community dendrogram based on the clustering results;

[0070] Step 3: Based on the model loss function, the communities of the multi-level community tree diagram are refined and divided, and the community detection of each node in the processed graph data is completed based on the refined division results.

[0071] In this embodiment, the graph convolutional network is set in advance and is used to analyze the graph data to be processed, so as to determine the node features in the graph data to be processed.

[0072] In this embodiment, preprocessing refers to performing processing such as region division on the graph data to be processed, so as to facilitate determination of node features of each node in the graph data to be processed.

[0073] In this embodiment, the node features refer to the shape of each node in the graph data to be processed, the edge features, and the connection relationship between each node.

[0074] In this embodiment, the probability distribution is a parameter used to characterize the similarity between different nodes. That is, the higher the similarity, the greater the probability value between the nodes, that is, the greater the possibility that they belong to the same community.

[0075] In this embodiment, the belonging community refers to a group of nodes with a close association relationship. The nodes within the group are closely connected, while the connections with nodes outside the group are relatively sparse.

[0076] In this embodiment, the multi-level community tree diagram starts from a root node and branches downward to form multiple levels. Each node can have multiple child nodes, and the child nodes can further generate their own child nodes, and so on. In the multi-level tree diagram, different levels represent different classifications or levels of refinement of nodes, which can clearly show the hierarchical relationship, classification system or logical structure between nodes.

[0077] In this embodiment, the model loss function is set in advance and is used to refine the communities of the multi-level community dendrogram, that is, to further refine the community to which each node belongs.

[0078] The working principle and beneficial effects of the above technical solution are: by using a graph convolutional network to preprocess the graph data to be processed, the node features in the graph data to be processed can be accurately and effectively determined, and the similarity between the nodes can be determined by the node features, so that the probability distribution of each node can be determined according to the similarity, which provides convenience for community detection. Secondly, the nodes are clustered through probability distribution to lock the corresponding communities of each node, and the multi-level community tree diagram can be effectively constructed according to the belonging communities. Finally, the communities of the multi-level community tree diagram are refined and divided through the model loss function, so that accurate and reliable community detection of each node can be achieved, which ensures the accuracy of community detection, greatly improves the effect of community detection, and provides great convenience for understanding the functions and characteristics of the network.

[0079] Embodiment 2:

[0080] Based on Example 1, this example provides a community detection method based on a probability distribution hierarchical graph clustering network, such as Figure 2 As shown, in step 1, before preprocessing the graph data to be processed based on the graph convolutional network, it includes:

[0081] Step 101: Obtain the graph data to be processed sent by the management terminal, and traverse the graph data to be processed based on the processing index to obtain the index value of the graph data to be processed under the processing index;

[0082] Step 102: determining image representation features of the image data to be processed based on the index value, and comparing the image representation features with preset requirements;

[0083] Step 103: When it is determined based on the difference comparison result that the image representation features meet the preset requirements, the image data to be processed is determined to be qualified, and a community detection mechanism for the image to be processed is triggered;

[0084] Step 104: Otherwise, the graph data to be processed is returned.

[0085] In this embodiment, the processing index is known in advance and is used to analyze the image data to be processed, for example, it can be used to determine whether there is a relationship between the nodes included in the graph data to be processed.

[0086] In this embodiment, the indicator value refers to the analysis result corresponding to each processing indicator obtained after analyzing the graph data to be processed through the processing indicator.

[0087] In this embodiment, the image representation feature refers to the image effects and characteristics currently reflected by the image data to be processed.

[0088] In this embodiment, the preset requirements are set in advance.

[0089] The working principle and beneficial effects of the above technical solution are: by adopting processing indicators to analyze the graph data to be processed, the image representation features of the graph data to be processed can be accurately and effectively determined; secondly, the image representation features and preset requirements are compared for differences, so as to accurately and effectively determine whether the graph data to be processed is qualified based on the comparison results, which provides convenience and guarantee for accurate and reliable community detection of the graph data to be processed, thereby ensuring the accuracy of community detection of the graph data to be processed.

[0090] Embodiment 3:

[0091] On the basis of Example 1, this embodiment provides a community detection method based on a probability distribution hierarchical graph clustering network. In step 1, the graph data to be processed is preprocessed based on a graph convolutional network to extract node features of each node in the graph data to be processed, including:

[0092] Obtain the image data to be processed and divide the image to be processed into regions;

[0093] Based on the regional division results, the local region representation of each node in different regions is simultaneously determined, and the edge features, texture features and relative connection relationships between each node are obtained based on the local region representation;

[0094] Based on the edge features, texture features of each node and the relative connection relationship between each node, the node features of each node in the graph data to be processed are obtained, and the node features are vector-converted to obtain the feature vector corresponding to the node features of each node.

[0095] In this embodiment, the image to be processed refers to an image that needs to be subjected to community detection.

[0096] In this embodiment, the local region representation refers to the image representation of different regions in the image to be processed (the type and state of the object contained in the image).

[0097] In this embodiment, the node features refer to the connection relationship, edge features and texture features between nodes in the graph data to be processed, so as to facilitate community detection on the image to be processed.

[0098] In this embodiment, the feature vector refers to feature information in vector form obtained by format conversion of node features of each node.

[0099] The working principle and beneficial effects of the above technical solution are: by analyzing the image to be processed, the node features of each node in the image to be processed are effectively determined, and the obtained node features are formatted, so that it is convenient to determine the similarity between each node according to the feature vector obtained after the format conversion, which provides convenience for community detection.

[0100] Embodiment 4:

[0101] Based on Example 1, this embodiment provides a community detection method based on a probability distribution hierarchical graph clustering network. In step 1, the similarity between nodes is determined based on node features, including:

[0102] Obtain the basic configuration parameters of the graph convolutional network, and determine the target number of layers of the graph convolutional network based on the basic configuration parameters;

[0103] Aggregate the node features of each node in the processed graph data and the node features of adjacent nodes in turn based on the target number of layers, and perform low-dimensional space transformation on the node features of each node based on the aggregation results to obtain node embedding features of the node features of each node;

[0104] The node features of a node and the node features of adjacent nodes are aggregated according to the following formula:

[0105]

[0106] Z v =H (L) ;

[0107] Among them, H (l+1) The target number of layers in the graph convolution network is the feature obtained by aggregating the node features of each node in the processed graph data and the node features of the adjacent nodes in turn; σ(·) represents the activation function; l represents the label value of the current target number of layers in the graph convolution network; represents a diagonal matrix; H represents the sum of the weighted adjacency matrix plus the adjacency matrix of the current layer; (l) Represents the feature matrix of the l-th layer node in the graph convolutional network; W (l) H represents the weight of the lth layer in the graph convolutional network; (L) Indicates H (l+1)The features obtained by gradually aggregating from the first layer to the L layer; Z v The node embedding matrix represents the node features of each node; v refers to the ordinal value of each node in the graph data to be processed;

[0108] The node embedding features between different nodes are analyzed to obtain the similarity between each node.

[0109] In this embodiment, the basic configuration parameters refer to the basic information corresponding to the graph convolutional network, including corresponding data processing rules and corresponding metering rates, etc.

[0110] In this embodiment, the target number of layers refers to the number of computing layers included in the graph convolutional network, so as to facilitate multi-level analysis of the image to be processed and achieve accurate and effective determination of the similarity between nodes.

[0111] In this embodiment, the node embedding feature refers to the result obtained after the node feature of each node is transformed into a low-dimensional space, so as to facilitate the effective determination of the similarity between different nodes based on the node features.

[0112] The working principle and beneficial effects of the above technical solution are: by analyzing the graph convolutional network, the target number of layers can be locked, so that the node features of each node in the processed image can be effectively aggregated according to the target number of layers in the graph convolutional network, which facilitates the determination of the similarity between different nodes.

[0113] Embodiment 5:

[0114] On the basis of Example 4, this embodiment provides a community detection method based on a probability distribution hierarchical graph clustering network, characterized in that the node features of each node are converted into a low-dimensional space based on the aggregation result to obtain node embedding features of the node features of each node, including:

[0115] Determine a weighted adjacency matrix of each node in the to-be-processed graph data based on the operation standard of the management terminal, and determine a feature loss value of the embedding feature of each node based on the weighted adjacency matrix of each node and the node embedding feature of each node;

[0116]

[0117] Among them, μ represents the feature loss value of each node embedding feature; i and j represent the ordinal value of the node, and the value range is [1, N]; N represents the total number of nodes; A ij represents the weighted adjacency matrix between node i and node j; σ(·) represents the activation function; H represents the transposed matrix of the node embedding matrix of node i; j is the node embedding matrix of node j;

[0118] The feature loss value of each node is compared with the preset threshold, and when the feature loss value is greater than the preset threshold, the node features of the current node and the node features of the adjacent nodes are re-aggregated until the feature loss value is less than the preset threshold.

[0119] In this embodiment, the operating standard is known in advance and is used to characterize the operating conditions defined by the management terminal.

[0120] In this embodiment, the weighted adjacency matrix is ​​a matrix form used to represent the graph structure. In the graph, if the edges have different weights (for example, representing distance, connection strength, etc.), then a matrix is ​​used to represent the connection relationship between these nodes and the corresponding weights. This matrix is ​​called a weighted adjacency matrix. The elements in the matrix represent the weights of the edges between the corresponding two nodes. If there is no edge between the two nodes, the element is usually a specific value (such as 0 or infinity). The weighted adjacency matrix can be used to conveniently analyze and process graphs with weights.

[0121] In this embodiment, the feature loss value refers to whether the features represented by the node embedding features between the nodes meet the aggregation parameters.

[0122] In this embodiment, the preset threshold is set in advance and can be adjusted.

[0123] The working principle and beneficial effect of the above technical solution are: by determining the feature loss value of the embedded features of each node, the aggregation situation between the nodes in the processed image can be effectively determined, so that when the aggregation requirements are not met, the nodes can continue to be aggregated, ensuring the accuracy of the similarity determination between the nodes, thereby improving the reliability of community detection.

[0124] Embodiment 6:

[0125] On the basis of Example 4, this embodiment provides a community detection method based on a probability distribution hierarchical graph clustering network, which analyzes the node embedding features between different nodes to obtain the similarity between the nodes, including:

[0126] Obtain feature vectors corresponding to node embedding features of different nodes, and perform similarity calculation on the feature vectors based on a preset calculation formula. The execution steps include:

[0127]

[0128] Where sim(i,j) represents the cosine value of the angle between the feature vectors of node i and node j; Represents the feature vector obtained by gradually aggregating node i from the first layer to the L layer; represents the feature vector obtained by gradually aggregating node j from the first layer to the L layer; ‖·‖ represents the norm of the vector; Represents the modulus value corresponding to the feature vector obtained by gradually aggregating node i from the first layer to the L layer; Represents the modulus of the feature vector obtained by gradually aggregating node j from the first layer to the L layer;

[0129] The similarity between nodes is obtained based on the cosine value of the angle between the feature vectors of each node.

[0130] The working principle and beneficial effect of the above technical solution are: by calculating the feature vectors corresponding to the node embedding features of different nodes, the similarity between each node can be accurately and effectively determined through the cosine value of the angle, thereby providing reliable data support for community detection and ensuring the accuracy and reliability of community detection.

[0131] Embodiment 7:

[0132] On the basis of Example 1, this embodiment provides a community detection method based on a probability distribution hierarchical graph clustering network. In step 2, the probability distribution of each node is determined based on the similarity, and each node is clustered into a corresponding community based on the probability distribution, and a multi-level community dendrogram is constructed based on the clustering results, including:

[0133] Obtain the similarity between the obtained nodes, and determine the node pairs between the nodes in the graph data to be processed based on the similarity;

[0134] At the same time, the similarity of the node pairs is mapped to a preset function for processing to obtain the sampling probability of each node pair, and the node distance of each node pair is determined based on the sampling probability;

[0135] Based on the similarity between nodes and the node distance between each node pair, the probability distribution of each node is obtained, and each node is clustered based on the probability distribution result to obtain the corresponding community of each node;

[0136] Based on the clustering results, the similarities between different communities are determined in turn, and the communities with similarities greater than a preset threshold are merged to obtain a new community;

[0137] Based on the community merging results, the communities corresponding to each node are iteratively merged for a target number of times, and the target distance between the new community and the remaining communities after each iterative merging process is updated, and a multi-level community dendrogram is obtained based on the updated results.

[0138] In this embodiment, a node pair refers to nodes having a similar relationship.

[0139] In this embodiment, the preset function is set in advance and is used to determine the sampling probability of each node.

[0140] In this embodiment, the sampling probability is used to characterize the probability of each node pair being successfully detected when performing community detection, so as to facilitate determination of the community corresponding to each node pair.

[0141] In this embodiment, the belonging community refers to the specific community structure corresponding to each node pair in the image to be processed.

[0142] In this embodiment, the preset threshold is set in advance and is a measurement indicator used to measure whether the public security of different communities is similar, and can be adjusted.

[0143] In this embodiment, the target number of times is set in advance.

[0144] In this embodiment, the purpose of updating the target distance between the new community and the remaining communities after each iterative merging process is to further refine the similarity relationship between different communities, thereby effectively determining the multi-level community dendrogram, that is, effectively determining the association relationship between different nodes in the image to be processed.

[0145] The working principle and beneficial effects of the above technical solution are: by accurately and effectively determining the probability distribution of each node according to the similarity of each node, clustering each node according to the probability distribution, so as to accurately and effectively determine the community to which each node belongs. Secondly, the similarity between each belonging community is locked, and each belonging community is merged according to the similarity between each belonging community. Finally, a multi-level community tree diagram is accurately and effectively constructed, and accurate and reliable community detection is achieved for each node, thereby ensuring the accuracy of community detection and greatly improving the effect of community detection.

[0146] Embodiment 8:

[0147] On the basis of Example 1, this embodiment provides a community detection method based on a probability distribution hierarchical graph clustering network. In step 3, the community of the multi-level community dendrogram is refined based on the model loss function, and the community detection of each node in the processed graph data is completed based on the refined division result, including:

[0148] The obtained multi-level community dendrogram is obtained, and initial parameters of modularity and single adjustment parameters are configured for the multi-level community dendrogram based on community detection requirements;

[0149] Based on the single adjustment parameter, the initial parameter of the modularity is incrementally adjusted, and based on the incremental adjustment result, the local structure of each belonging community in the multi-level community dendrogram is refined and divided;

[0150] Based on the refined division results, the final community detection results of the graph data to be processed are obtained, and the community detection of each node in the graph data to be processed is completed.

[0151] In this embodiment, the basic idea is to move nodes one by one to other communities, and repeat this process until the modularity cannot be further increased. The value of this modularity is the maximum modularity, which is defined as:

[0152]

[0153] Based on the conversion relationship between the adjacency matrix, node weight and node pair sampling probability, its modularity can be expressed by probability distribution as follows:

[0154]

[0155] Among them, δ C (i,j) is an indicator function. If i and j belong to the same community, the value is 1. i =C j When C (i,j) = 1, otherwise 0;

[0156] It can be seen that the Q value actually reflects the difference between the number of edges actually connected to the node and the expected number of edges. However, any clustering of the maximization module has certain resolution limitations, and the use of fixed resolution parameters also makes the algorithm lose some universality.

[0157] The introduction of the multiplication factor γ improves the accuracy and robustness of community detection;

[0158] γ is the resolution parameter. Dynamically adjust the resolution parameter γ to optimize the modularity, ensuring that the community division can reflect both the local structure and the overall graph structure. The parameter modularity is:

[0159]

[0160] Corresponding to the modularity at the cluster level:

[0161]

[0162] The dynamic adjustment parameters mainly increase γ when the modularity increases to promote further subdivision of the community and reveal a more detailed structure; when the modularity decreases, γ is reduced to prevent over-segmentation and maintain the community structure, based on the following:

[0163]

[0164] Where δ is a small positive constant with a default value of 0.05, and γ = γ0 is initialized, starting from 1 for each iteration t, so as to continuously update the γ of the next iteration;

[0165] The introduction of this resolution parameter avoids the limitation of local optimum and improves the adaptability of the algorithm.

[0166] The working principle and beneficial effects of the above technical solution are: the communities of the multi-level community tree diagram are refined and divided through the model loss function, so as to realize accurate and reliable community detection of each node, ensure the accuracy of community detection, greatly improve the effect of community detection, and provide great convenience for understanding the functions and characteristics of the network.

[0167] Embodiment 9:

[0168] This embodiment provides a community detection system based on a probability distribution hierarchical graph clustering network. Figure 3 As shown, including:

[0169] A similarity determination module is used to pre-process the graph data to be processed based on the graph convolution network, extract the node features of each node in the graph data to be processed, and determine the similarity between the nodes based on the node features;

[0170] A community detection module is used to determine the probability distribution of each node based on similarity, cluster each node into the corresponding community based on the probability distribution, and construct a multi-level community dendrogram based on the clustering results;

[0171] The community refinement module is used to refine the communities of the multi-level community dendrogram based on the model loss function, and complete the community detection of each node in the processed graph data based on the refined division results.

[0172] The working principle and beneficial effects of the above technical solution are: by using a graph convolutional network to preprocess the graph data to be processed, the node features in the graph data to be processed can be accurately and effectively determined, and the similarity between the nodes can be determined by the node features, so that the probability distribution of each node can be determined according to the similarity, which provides convenience for community detection. Secondly, the nodes are clustered through probability distribution to lock the corresponding communities of each node, and the multi-level community tree diagram can be effectively constructed according to the belonging communities. Finally, the communities of the multi-level community tree diagram are refined and divided through the model loss function, so that accurate and reliable community detection of each node can be achieved, which ensures the accuracy of community detection, greatly improves the effect of community detection, and provides great convenience for understanding the functions and characteristics of the network.

[0173] Embodiment 10:

[0174] On the basis of Example 9, this embodiment provides a community detection system based on a probability distribution hierarchical graph clustering network, and a similarity determination module, including:

[0175] An image preparation unit is used to obtain the to-be-processed graph data sent by the management terminal, and traverse the to-be-processed graph data based on the processing index to obtain the index value of the to-be-processed graph data under the processing index;

[0176] Image verification unit, used for:

[0177] Determine the image representation features of the image data to be processed based on the index value, and compare the image representation features with the preset requirements;

[0178] When the image representation features meet the preset requirements based on the difference comparison results, the image data to be processed is judged to be qualified, and the community detection mechanism for the image to be processed is triggered;

[0179] Otherwise, the graph data to be processed is returned.

[0180] The working principle and beneficial effects of the above technical solution are: by adopting processing indicators to analyze the graph data to be processed, the image representation features of the graph data to be processed can be accurately and effectively determined; secondly, the image representation features and preset requirements are compared for differences, so as to accurately and effectively determine whether the graph data to be processed is qualified based on the comparison results, which provides convenience and guarantee for accurate and reliable community detection of the graph data to be processed, thereby ensuring the accuracy of community detection of the graph data to be processed.

[0181] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A community detection method based on probability distribution hierarchical graph clustering network, characterized in that: include: Step 1: Preprocess the graph data to be processed based on the graph convolutional network, extract the node features of each node in the graph data to be processed, and determine the similarity between the nodes based on the node features; Step 2: Determine the probability distribution of each node based on the similarity, cluster each node into the corresponding community based on the probability distribution, and construct a multi-level community dendrogram based on the clustering results; Step 3: Based on the model loss function, the communities of the multi-level community tree diagram are refined and divided, and the community detection of each node in the processed graph data is completed based on the refined division results.

2. A community detection method based on probability distribution hierarchical graph clustering network according to claim 1, characterized in that: In step 1, before preprocessing the graph data to be processed based on the graph convolutional network, it includes: Obtain the graph data to be processed sent by the management terminal, and traverse the graph data to be processed based on the processing index to obtain the index value of the graph data to be processed under the processing index; Determine the image representation features of the image data to be processed based on the index value, and compare the image representation features with the preset requirements; When the image representation features meet the preset requirements based on the difference comparison results, the image data to be processed is judged to be qualified, and the community detection mechanism for the image to be processed is triggered; Otherwise, the graph data to be processed is returned.

3. A community detection method based on probability distribution hierarchical graph clustering network according to claim 1, characterized in that: In step 1, the graph data to be processed is preprocessed based on the graph convolutional network to extract the node features of each node in the graph data to be processed, including: Obtain the image data to be processed and divide the image to be processed into regions; Based on the regional division results, the local region representation of each node in different regions is simultaneously determined, and the edge features, texture features and relative connection relationships between each node are obtained based on the local region representation; Based on the edge features, texture features of each node and the relative connection relationship between each node, the node features of each node in the graph data to be processed are obtained, and the node features are vector-converted to obtain the feature vector corresponding to the node features of each node.

4. A community detection method based on probability distribution hierarchical graph clustering network according to claim 1, characterized in that: In step 1, the similarity between nodes is determined based on node features, including: Obtain the basic configuration parameters of the graph convolutional network, and determine the target number of layers of the graph convolutional network based on the basic configuration parameters; Aggregate the node features of each node in the processed graph data and the node features of adjacent nodes in turn based on the target number of layers, and perform low-dimensional space transformation on the node features of each node based on the aggregation results to obtain node embedding features of the node features of each node; The node features of a node and the node features of adjacent nodes are aggregated according to the following formula: Z v =H (L) ; Among them, H (l+1) The target number of layers in the graph convolution network is the feature obtained by aggregating the node features of each node in the processed graph data and the node features of the adjacent nodes in turn; σ(·) represents the activation function; l represents the label value of the current target number of layers in the graph convolution network; represents a diagonal matrix; H represents the sum of the weighted adjacency matrix plus the adjacency matrix of the current layer; (l) Represents the feature matrix of the l-th layer node in the graph convolutional network; W (l) H represents the weight of the lth layer in the graph convolutional network; (L) Indicates H (l+1) The features obtained by gradually aggregating from the first layer to the L layer; Z v The node embedding matrix represents the node features of each node; v refers to the ordinal value of each node in the graph data to be processed; The node embedding features between different nodes are analyzed to obtain the similarity between each node.

5. A community detection method based on probability distribution hierarchical graph clustering network according to claim 4, characterized in that: Based on the aggregation results, the node features of each node are transformed into a low-dimensional space to obtain the node embedding features of the node features of each node, including: Determine a weighted adjacency matrix of each node in the to-be-processed graph data based on the operation standard of the management terminal, and determine a feature loss value of the embedding feature of each node based on the weighted adjacency matrix of each node and the node embedding feature of each node; Among them, μ represents the feature loss value of each node embedding feature; i and j represent the ordinal value of the node, and the value range is [1, N]; N represents the total number of nodes; A ij represents the weighted adjacency matrix between node i and node j; σ(·) represents the activation function; H represents the transposed matrix of the node embedding matrix of node i; j is the node embedding matrix of node j; The feature loss value of each node is compared with the preset threshold, and when the feature loss value is greater than the preset threshold, the node features of the current node and the node features of the adjacent nodes are re-aggregated until the feature loss value is less than the preset threshold.

6. A community detection method based on probability distribution hierarchical graph clustering network according to claim 4, characterized in that: Analyze the node embedding features between different nodes to obtain the similarity between each node, including: Obtain feature vectors corresponding to node embedding features of different nodes, and perform similarity calculation on the feature vectors based on a preset calculation formula. The execution steps include: Where sim(i,j) represents the cosine value of the angle between the feature vectors of node i and node j; Represents the feature vector obtained by gradually aggregating node i from the first layer to the L layer; represents the feature vector obtained by gradually aggregating node j from the first layer to the L layer; ‖·‖ represents the norm of the vector; Represents the modulus value corresponding to the feature vector obtained by gradually aggregating node i from the first layer to the L layer; Represents the modulus of the feature vector obtained by gradually aggregating node j from the first layer to the L layer; The similarity between nodes is obtained based on the cosine value of the angle between the feature vectors of each node.

7. A community detection method based on probability distribution hierarchical graph clustering network according to claim 1, characterized in that: In step 2, the probability distribution of each node is determined based on the similarity, and each node is clustered into the corresponding community based on the probability distribution, and a multi-level community dendrogram is constructed based on the clustering results, including: Obtain the similarity between the obtained nodes, and determine the node pairs between the nodes in the graph data to be processed based on the similarity; At the same time, the similarity of the node pairs is mapped to a preset function for processing to obtain the sampling probability of each node pair, and the node distance of each node pair is determined based on the sampling probability; Based on the similarity between nodes and the node distance between each node pair, the probability distribution of each node is obtained, and each node is clustered based on the probability distribution result to obtain the corresponding community of each node; Based on the clustering results, the similarities between different communities are determined in turn, and the communities with similarities greater than a preset threshold are merged to obtain a new community; Based on the community merging results, the communities corresponding to each node are iteratively merged for a target number of times, and the target distance between the new community and the remaining communities after each iterative merging process is updated, and a multi-level community dendrogram is obtained based on the updated results.

8. A community detection method based on probability distribution hierarchical graph clustering network according to claim 1, characterized in that: In step 3, the communities of the multi-level community dendrogram are refined based on the model loss function, and the community detection of each node in the processed graph data is completed based on the refined division results, including: The obtained multi-level community dendrogram is obtained, and initial parameters of modularity and single adjustment parameters are configured for the multi-level community dendrogram based on community detection requirements; Based on the single adjustment parameter, the initial parameter of the modularity is incrementally adjusted, and based on the incremental adjustment result, the local structure of each belonging community in the multi-level community dendrogram is refined and divided; Based on the refined division results, the final community detection results of the graph data to be processed are obtained, and the community detection of each node in the graph data to be processed is completed.

9. A community detection system based on probability distribution hierarchical graph clustering network, characterized in that: include: A similarity determination module is used to pre-process the graph data to be processed based on the graph convolution network, extract the node features of each node in the graph data to be processed, and determine the similarity between the nodes based on the node features; A community detection module is used to determine the probability distribution of each node based on similarity, cluster each node into the corresponding community based on the probability distribution, and construct a multi-level community dendrogram based on the clustering results; The community refinement module is used to refine the communities of the multi-level community dendrogram based on the model loss function, and complete the community detection of each node in the processed graph data based on the refined division results.

10. A community detection system based on probability distribution hierarchical graph clustering network according to claim 9, characterized in that: The similarity determination module includes: An image preparation unit is used to obtain the to-be-processed graph data sent by the management terminal, and traverse the to-be-processed graph data based on the processing index to obtain the index value of the to-be-processed graph data under the processing index; Image verification unit, used for: Determine the image representation features of the image data to be processed based on the index value, and compare the image representation features with the preset requirements; When the image representation features meet the preset requirements based on the difference comparison results, the image data to be processed is judged to be qualified, and the community detection mechanism for the image to be processed is triggered; Otherwise, the graph data to be processed is returned.

Images